Welcome to CellTracker

Fast image analysis is essential for high-throughput functional gene analysis. CellTracker is a unique tool for tracking gene expression levels of individual cells imaged by time-lapse microscopy. CellTracker integrates image browsing, processing and cell tracking. It features:

  • Powerful and flexible image processing
  • Compact cell tracking wizard and flexible boundary tracking
  • Straight forward visualisation
  • Supports multiple export formats, including TIFF image, AVI video, spreadsheet and XML files

To have a go, please download demo images from http://www.dbkgroup.org/celltracker.


Contents

Get started

Main controls

An overview of tracking algorithms

How to have a successful tracking

Menus

Demos

Release notes


Release note

Release notes

The CellTracker website, http://www.dbkgroup.org/celltracker /, has been updated. One may send feedback on-line.

1 New features

  • Mouse has four modes in this version, editing, labelling, selection and zoom. Each mode is applied for a whole series. For example, one may edit any boundary in any frame in the editing mode. The edited boundaries will be saved into CellTracker's workspace automatically during image browsing.

2 Modified features

  • The menus have been reorganised. Most boundary operators are put under the Edit menu. Copy and rename are listed in boundaries' context menu. Two new menus, Tools and Window, are added.

3 Limitation and known problems

  • CellTracker doesn’t support tiff files generated by its export facility, in which each file presents the image from only one channel.
  • The position of slider in the main window may not indicate the location of the current frame in the series.

4 Previous release notes

Release note of version 0.5


Main window

  1. Start up window
  2. Main window after loading data
  3. Toolbar
  4. Browsing tools
  5. Status bar

1 Start-up window

After start-up, CellTracker’s main window appears as below. One may load image series (menu File-Read time series image) or customise preference settings (menu File-Preference) here.

back

2 Main window after loading data

After loading an image series,you may get a window similar to the one shown below.The tools in upper toolbar are mainly used for boundary operations. Browsing tools at the bottom of the window are for image navigation. The lower status bar shows CellTracker running information.

back

3 Toolbar

Open a Carl Zeiss LSM file
Save current session
Undo
Redo
Draw a rectangular ROI
Draw a closed smooth curve
Draw a freehand closed curve
Lasso for boundary selection
Selection mode switch
Delete boundaries in current frame
Boundary editing mode switch
Boundary labelling mode switch
Zoom mode switch
Reset image scale
Change boundary style

Draw freehand curves: Click [image] and hold left mouse button to draw a free hand curve. Release the left button to close the curve by connecting the last point to the starting point.

Draw smooth spline curves: Click [image] to draw a closed spline line. In this mode, click key boundary points with the left mouse button and right click to finish. One may use the context menu to convert it into cytoplasmic and nuclear boundary.

Label boundaries: Click [image] to turn on labelling mode. Click it again to turn off. In the labeling mode, click boundary to create a label. After the label appears, one may click and hold it with the left button to move it elsewhere in the image. Remove a label use its context menu (right click). Use the boundary's context menu Rename to change boundary label.

Edit boundaries: Click [image] to turn on editing mode. Click it again to turn off the mode. In this mode, a few red circles appear when a boundary with control points is clicked. Click and move red circles to change the boundary shape. Click elsewhere in the boundary to move the whole boundary. In this version, one can’t add/delete control points. Boundaries without control points can only be moved as a whole. One may generate control points of a curve by converting it into spline form (menu-Boundary-transform into spline form or via its context menu). Note that boundary after the conversion is usually not the same as the original one.

Select boundaries:The lasso button [image] is for selecting boundaries. The selected boundaries will be highlighted. If the boundaries can not be selected using the lasso tool. One may turn on selection mode by clicking [image], and then click boundaries to select. Click the button [image] again to turn off the mode. To select all boundaries in the current frame, one may used menu Edit-Select all.

The button [image] is for deleting selected boundaries in the current frame. Use button [image] to undo the last operation. Use menu under View to show/hide overlay menu to show/hide certain types of overly.

back

4 Browsing tools
Show image series as a video
A slider for browsing images
Stop play Select display channels
Next
Previous

Click the channel selection drop down menu in the lower left corner of the CellTracker main window to browse each channel. Select All channels if you wish to display all channels in one image, which will make browsing slightly slow. The item Every Channel is to display each channel in one image (a maximum of four images can be shown in the display area). One can also go to a frame using menu Edit-Go to menu.

back

5 Status bar
A
B
C
D
A
Zoom mode status
B | Selection mode status
C | Current frame no
D | CellTracker status

back


An overview of tracking algorithms

Basic tracking algorithms

There are mainly four algorithms for cell boundary tracking under the menu Boundary-Tracking.

  1. Tracking by edge using template affine transform
  2. Seeded watershed
  3. Tracking by edge using key frame templates
  4. Tracking by colour

The simple cell tracking and advanced cell tracking under the Cell menu are implemented as the combinations of the four key algorithms above. According to the feature detection approaches used in the algorithms, the tracking methods can be classified as below.

According to the number of shape templates used for tracking each boundary, the tracking method can also be classified as below

The general interface for the four algorithms is shown below. One has to select the specific parameters for each algorithm, tracking range and the tracking targets (boundaries). Only the boundaries in the ROIs will be listed, since the CellTracker confines tracking and image processing in ROIs.

I Tracking by edge using template affine transform

This method assumes that the cellular boundaries undergone translation, rotation and scale changes over the time. One may use the current boundaries as templates. If the boundaries are close to ellipse, one may also use ellipse as the shape templates, which often generates robust results than the former.

In order to find the boundary positions, the algorithm generates Nsample contour randomly within a possible ranges of translation, rotation and scale variation. In order to evaluate how well a boundary fit in the image, the CellTracker generates Nnormal normal line along the boundary, and then checks if there are edges along normal lines within a range of m. The method then uses the position of edges to give each boundary a weight (score). The estimated position of a cellular boundary is calculated as the weighted sum of all the Nsample boundaries. In most processes, the shape of nucleus has rather limited change in shape, which may be simplified as affine transform between frames. This method is therefore often used for tracking nuclei. The procedure, under the menu Boundary-Tracking-Tracking by edge using template affine transform, uses the method for tracking the boundaries in the ROIs. The parameter window is show below.

Note that one should control the size variation under a low level.

II Seeded watershed

The method requires a seed for each tracking target and a detection limit to find the region covered by cells. Once the region is obtained, it classifies the foreground pixels using the distance to seeds and the image gradient. There is another parameter, regulation factor, to balance the weight between the distance and image gradient for image segmentation. In this method, there are no constraints on the shape and size of the boundaries. It is therefore often used to track cytoplasmic boundaries using nuclei as seeds.

This methods has appeared a couple of times in the CellTracker

  1. Boundary-Boundary detection-Seeded watershed
  2. Boundary-Tracking-Seeded watershed

In the former case, the CellTracker uses the current boundaries in ROIs as seeds. In the procedure under Cell-Simple cell tracking, the methods I and II are combined to track nuclear and cytoplasmic compartments. Depending on cells motion type, there are some variations on how to set seeds in Simple cell tracking. In most case, nuclei are chosen as seeds. In Advanced cell tracking, the combination is also available (choose one shape template for nucleus and no template for cytoplasm i.e. No way to be presented by a few templates). In the advanced mode, the users are also allowed to choose different channels for tracking nuclei and cytoplasms, respectively.

One problem of this method is that overlapping cells may also be labelled as the tracking objects. In that case, one may mark the interfering cells as user defined boundaries, and tracking the cells over the overlapping period again.

In the Simple cell tracking and the advanced tracking, the nuclei are used as seeds. There are only two parameters, i.e. detection limit and the regulation factor.

III Tracking by edge using key frames

If cellular shapes changes gradually in a process, one may select a number of key frames to model the cellular shapes between frames. The motion in the current version is limited to translation only. The advantage of this method is that there is no limitation on the number of key frames to be used, however, due to the limitation of motion, i.e. no rotation, it has limited applications. One main application is for post-processing. As mentioned earlier, one problem of the seeded watershed method is that overlapping cells may also be labeled as the tracking objects. In that case, one may use the boundaries before and after the overlapping period as key frames and do tracking over the period again.

IV Tracking by colour

This method assumes that the histograms of the tracking are very similar between frames. It uses the colour appearance for tracking. Compared with edge based tracking, the algorithm is robust to regions with low signal-to-noise ratio. Compared to method I, the motion is limited to translation and rotation only. In many cases, the histograms of cellular compartments vary lot in a biological process, and due to the limit in motion, the method is mainly suitable for tracking the regions with little change in appearance, e.g. nuclei stained by dyes.

Cell Tracking

In the simple cell tracking, we use tracking by edge using template affine transform to track nuclei and seeded watershed for cytoplasms. In the advanced cell tracking, because the very limited use of tracking by colour, the CellTracker combines three methods for tracking, i.e. method I,II and III. In addition, one may use different channels for nuclei and cytoplasm tracking. In the nuclei tab, Tracking by edge using template affine transform will be applied if user selects Need only one shape template. If the user chooses Need more than one shape templates, the method Tracking by edge using key frame templates will be applied. In cytoplasm tracking, that no way to be presented by a few templates means we choose seeded watershed for tracking.

Nuclei tab in advanced cell tracking

Cytoplasms tab in advanced cell tracking

There are about nine combinations for tracking cells as below.


How to have a successful tracking

How to make a successful tracking

Every algorithm has its own limitations or assumptions. To get a successful tracking, at least the three conditions have to be fulfilled.

  • Suitable tracking method
  • Proper image pre-processing
  • Reasonable motion parameters

Even so, boundary post-processing may still be required in a complex process.

I Choose a tracking method

If you are not sure which method to use, please use the cell tracking wizard, i.e. Advance cell tracking under the Cell menu. Choose proper options based on the cell shape variation in your application.

II Choose proper pre-processing

In the overview of tracking algorithms, the tracking methods are classified as three types as below.

In the advanced tracking window, the tracking procedures depend on image edge or intensity features.

In the nuclei tab, both algorithms for Need only one shape template and Need more than one shape templates are based on edge detection. In the cytoplasms tab, Need only one shape template and Need more than one shape templates are based on edge features. The third option No way to be presented by a few shape templates is based on image intensity features.

Colour based tracking

In the tracking based on colour, it usually does not require any pre-processing to improve the signal-to-noise ratio of image.

Intensity base tracking

Normally, a smoothing operation, e.g. average and median filter is required. If the image signal-to-noise ratio is very low. One may use texture information for tracking, e.g. an image of entropy.

Edge base tracking

In edge based tracking, smoothing operation (methods under the menu Image-Blur), e.g. average and median filter, is required. If a large blur window is required, the median filter is often preferred. The flowchart of common image pre-processing is shown below. The steps with dotted borders are optional.

No pre-processing other than smoothing is required if the contrast of cell compartments are almost the same over time. In many cases involving translocation between nucleus and cytoplasm, the pre-processing, local normalisation, may be required. The procedure corrects image intensities so that the weak edges could be detected.

Image pre-processing 1 original image (left) 2 smoothed image (middle) 3. After local normalisation (right).

III Estimate tracking parameters

In most tracking windows, one needs to input the cell motion information, i.e. motion type and parameters. The default motion parameters are saved as cell info (File-Cell information). The form is shown below.

One may first track cells using the default parameters. In most case, a number of consecutive frames (5 at least), could be successfully tracked. One may then estimate the motion parameters using the sequence. To do this, click the Default button and then select a range of frame for training.

IV Boundary post-processing

1. Refine cell boundaries

After tracking using methods under the Cell menu, one may refine the boundaries using the active contour method (Boundary-Tracking-Refine boundaries by active contour). The parameters of active contour are shown below. If the refinement is too much using the default parameters, one may reduce the number of iterations.

2. Processing

Using menu-Boundary-Processing one may make boundary convex, of the ellipse shape, and expand/shrink.

3. Retrack a sub-range

If there are interfering objects near tracking targets, they may be labeled as targets cytoplasm regions as well. To correct this, one may draw user defined boundaries to cover the interfering objects, and do the tracking over the sub-range again.

4. Modify boundaries manually

In the worst case, use the manual tools, which is obvious useful if there is problem with just a few frames.


Menus

1 File

2 Edit

3 View

4 Image

5 Boundary

6 Cell

7 Graph

8 Tools

9 Windows

10 Help


File menu

  • Read time series image
  • Load
  • Save
  • Close current file
  • Imaging information
  • Cell information
  • Dye information
  • Export
  • Print preview
  • Print
  • Optimise memory
  • Preference
  • Exit

Read time series image

Carl Zeiss LSM, tiff and Matlab mat files can be read directly into the CellTracker. Use the link to get an explanation of the interface for loading images.

Carl Zeiss LSM file

Import a Carl Zeiss LSM file of time series image. In the snapshot below, the list box shows the LSM files in current folder. The right image is the thumbnail of the image file highlighted in the left. Press Browse button to change the current folder and the Preview button for preview the whole image series. Click Load button to open the image series.

Tiff image folder

Read all the tiff images in a folder as a time series image. The image sequence is ordered by file names. One may also select invidual tiff files in CellTracker.

Tiff image files

Read and organise an image series from a group of tiff files. Users may select and order tiff file sequence manually. Click Browse button in the window to change the current folder and Add tiff files to make an image series. One may also Remove files from the list box. Use Up and Down buttons to change the order of tiff files in the image series.

Matlab mat file

Read an image series saved in Matlab format. The mat file contains three matrices, X, T and W. They are image data, time stamps, and image channel names, respectively.

X: a cell array,in which each element represent a multi-frame image series at an imaging channel.

T: a vector of time stamp of each frame

W: a cellular string in which each element is channel name.

Suppose we have an image with 50 frames and three channels, A, B and C. In Matlab, the variables T and W can be written as

W={'A', 'B', 'C'};

T=[1:50]'; % a column vector

And each element of X is a three dimensional array with 50 frames.

back

Load

Saved session

Load saved session, which includes pre-processing and cellular boundaries and ROIs.

Cell motion model

Load cell motion model built by CellTracker

Image pre-processing procedure

Load image pre-processing procedure and parameters saved in Save menu. The procedure loaded may be appended to or replace the current one.

back

Save

Cell motion model

Save a motion model

Current session

Save image pre-processing and the boundaries of the current image series, which may be loaded later. With the boundaries, the CellTracker can calculate cell properties such as nuclear average intensity.

Image pre-processing procedure

Save pre-processing parameters for future use.

back

Close current file

Close the current image sequence. CellTracker interface back to start-up window.

back

Imaging information

Show the imaging information if the original image series is generated by Carl Zeiss Microscope. An example is shown below.

back

Cell information

A dialogue for user to enter cell motion information. Currently, cellular and nuclear information is used in tracking and boundary detection methods (Level set, Direct threshold). If the cell and nucleus area threshold are too large, the original result from these methods may be regarded as artefacts and will be discarded. Click default to estimate the default values. Note that if there are cell and nucleus marked in current frame, the default values are the average size of cell and nucleus. Otherwise, the default cell area is about 1/200 of image area. For an image of size 512x512, the default value will be 512x512/200=1310. The nuclear area is about 1/3 of the cell size.

back

Dye information

This is a window for inputting the dye information of each image channel. This information has NOT been used at the moment. CellTracker classifies image channels into six classes according dye information. Click a relevant buttons below Dye information to set dye information for each channel. In the case shown below, Channel Ch2-T1 and Ch1-T2 are channels with functional dyes.ChD-T1 is a bright field channel without any dye.

back

Export

Image data and tracking result can be exported in several formats, i.e. Excel spreadsheet, AVI video, Tiff image sequence and XML file. One may have an overview of the CellTracker export by data types.

Tracking result to Excel

Export result into a Microsoft Excel file. In the snapshot below, Range refers to the range of frames. One may Select a range or input a valid range expression. Use the Add and Remove buttons to change the export channels. The selected channels are shown in the upper list box. The lower list box shows the chosen properties for export. Please use Add and Remove button to change properties in export. Result will be exported into an Excel sheet (If Save into an existing file is checked, the CellTracker will add a new sheet into the existing file) Check the first check box if the user want to export the image information as well (only valid if the image source is a Carl Zeiss LSM file). Use the link to get the meanings of the export properties.

back

Images as AVI video:

Export image series as an AVI video. One may export image data only or snapshots of current image as well as overlays. Note that the latter mode is much slower than the former.

back

Image as tiff files

back

Result in XML

Export cell boundary properties in XML format. Compared with the Tracking to Excel, CellTracker may export the coordinates of cellular boundaries as well. In the snapshot below, Range refers to the range of frames. One may Select a range or input a valid expression. Use the Add and Remove buttons to modify export channels. The selected channels are shown in the upper list box. The lower list box shows the properties selected for export. Please use Add and Remove button to change properties in the export list. Use the link to get the meanings of the export properties. One may view the output schema here.

back

Print preview

Print preview of the current image

back

Print

Print the current image

back

Optimise memory

Memory may become fragmented after using CellTracker for a while. This menu will save all the data in the hard disk and load them again so that the memory is rearranged.

back

Preference

Preference in CellTracker, data load mode, tips display mode and user’s folders. If the user’s PC has low memory, one may choose to load data frame by frame. Otherwise, choose to load all frames into memory in one go. Select show all tips if the user is not familiar with CellTracker’s interface (mainly buttons). In Custom mode, CellTracker will show all tips at first, then whether the tips window will be shown or not next time depends on the user’s choice for each tips.

(Preference window)

(Tips window)

back

Exit

Close CellTracker. CellTracker will check whether there are ROIs and pre-processing procedure available. If there are, user will be prompted to save the result before closing CellTracker.

back


Edit menu

  • Undo
  • Redo
  • Copy image into a new window
  • Copy image to clipboard
  • Clear boundary labels
  • Copy
  • Delete
  • Delete ROIs
  • Rename
  • Select all
  • Invert selection
  • Go to
  • Refresh

Undo

List of operations applied recently. One may undo a number of steps in one go. In the window below, select the second item ‘Draw free hand…’ will undo the first two operations. One can not an intermediate steps only.

back

Redo

Redo the steps that have been undone. The interface is the same as the Undo list.

back

Copy image into a new window

The contents, including boundaries and the labels, in the current image will be copied into a new window, where user may print and change image/overlay display mode. If there are multiple images on display in the main window, please click an image first to make it the current image.

back

Copy image into clipboard

The contents of the current image will be copied into the windows clipboard, which may be pasted into another application. If there are multiple images, please click an image first to make it the current image.

back

Clear boundary labels

Remove boundary labels in the current frame. The boundary names however remain unchanged.

back

Copy

Copy selected boundaries. One may select boundaries first, and run this operation. The selection in the list box corresponds to the select boundaries in the current frame. One may also select boundaries directly from the listbox. The operation is based on the boundary ID. The existing boundary will be overwritten if a boundary of the same ID exists in the destination frame.

back

Delete

Delete selected boundaries. One may select boundaries first, and run this operation. The selection in the list box corresponding the select boundaries in the current frame. One may also select boundaries directly from the listbox. One may select specific ones (above the separator) or a certain types (below the separator). One may also combine both selection criteria, e.g. delete boundary cell unamed1 + any nuclear curves. To delete all the boundaries in the image, choose any cellular curves + any nuclear curves+ any user defined curves. After each operation, the contents in the interface will be updated immediately. Click the button Close to finish. The window will also be closed automatically if there is no boundary left in the current frame. The CellTracker only lists all the visible cell boundaries. If users select boundaries using the options below the separator in the deletion mode, the related hidden boundaries will be deleted as well.

back

Delete ROIs

Delete all the ROIs. Use ROI's context menu to delete individual ROIs.

back

Rename

Rename boundaries on a number of frames. One may also change the boundary type.

back

Select all

Select all the boundaries in the current image

back

Invert selection

Invert the selection of the boundaries in the current image

back

Go to

Input a frame number,so that CellTracker navigates from the current frame to another.

back

Refresh

Update the display in the CellTracker’s main window.

back


View menu

  • Image display form
  • Image colour
  • Show/hide overlay
  • Overlay style

Image display form

There are two submenus, Image (original) and Image. In the former mode, the CellTracker displays the original image data; in the latter mode, the image after processing. One may have to use Image mode in a tracking process. For the sake of fast browsing, one may use Image (original) mode, in which the CellTracker skips the pre-processing steps.

back

Image colour

The snapshot is shown blow. Click the rectangle region near a channel name to change the channel’s colour. CellTracker sets the default colour is white (Gray level image), i.e. the higher image intensity, the brighter image becomes.

back

Show/Hide overlay

Check on/off the corresponding box to hide or display overlays. CellTracker tracks all the visible cell boundaries in the current ROIs. If only a certain type of boundaries is to be tracked, e.g. nuclear boundaries, one may hide a certain type of boundaries first with this menu. For the methods under the Boundary-Tracking menu, one may select boundaries for tracking.

back

Overlay Style

This menu corresponds to the tool [image] to change overlay and label styles. In the interface, one may use the context menu of labels and curves to modify styles. One may change the colour, line width and line style of overlays, and the font style, colour and the background colour of labels.

back


Image menu

This menu provides common tools for image pre-processing. The processing is applied for the regions inside ROIs of the whole image series. If there is no ROI marked, CellTracker takes the whole image for processing. For most methods, there is a preview window so that users may tweak parameters. The normal preview window is shown below. The left shows what the image and boundary look like after processing. The upper right tools are used for parameter tweaking. One must tick the preview on to get a preview. Clicking Apply will close the preview window and impose the pre-processing on the image series. Click Cancel to close the window without any action. Use Reset button to set the parameters the default values, and tick off the preview check box.

  • Blur
  • Bright/contrast
  • Cleaning
  • Convert into binary image
  • Edge detection
  • Gradient
  • Invert
  • Local normalisation
  • Morphological transform
  • Texture
  • Apply pre-processing on whole images
  • Remove pre-processing

Blur

It is mainly used for image smoothing. One may smooth using a Gaussian, average or median filter. One may preview result when tuning parameters. For the tracking methods based on edge information, smoothing is essential for the success of tracking cells with low signal-to-noise ratio image data. If the background is clean, one may use averaging for smoothing. If the background region has strong noise like salt and pepper, one may use median filter to suppress the noise.

back

Bright/Contrast

Brightness adjusting manually or adaptively. The latter approach often gives better result. In addition, no parameter is required. The interface for manual adjustment is given below.

back

Cleaning

Fill holes: Fill holes in the image. An example before (left) and after hole filling is given below.

Clear border: Suppress object bordering with the ROIs. An example before (left) and after the clearing (right) is given below.

back

Convert into binary image

Transform image into only two levels, foreground and background. One is asked to input an adjustment factor. The larger the factor, the higher the threshold. For the left image below, an adjustment factor 0.5 gives the result in the middle. When it is increased to 1.5, the most cellular region will be thresholded as background.

back

Edge detection

Edge detection is one of the most important operators. Canny edge detection is suitable for both fluorescent and bright field channel. The other method, direct edge detection, first transforms the image into binary image and removes small artifacts using default cell information. In the canny edge detection, there are two parameters, one is a threshold adjustment factor, and the other is the standard deviation of the Gaussian filter for smoothing. If there are too many edges arising from noise, one may increase the sigma value.

The edge from the binary image is usually very clean, but this is only suitable for fluorescent images. In the direct edge detection method, image is first transformed into binary form by adaptive thresholding, and followed by edge detection. If the threshold is too high, those objects with low intensities won’t be detected as edges. On the other hand, the background will be regarded as foreground if the threshold is too low. The area threshold is to remove small artifacts.

(direct edge detection)

back

Gradient

In the CellTracker, two methods are presented for gradient calculation. One is used in canny edge detection, in which the image is first smoothed.

The other is called gradient vector flow described at http://iacl.ece.jhu.edu/projects/gvf/. This is computed as a spatial diffusion of the gradient of edges derived from the image. This computation causes diffuse forces to exist far from the object, and crisp force vectors near the edges.

back

Invert

Invert image intensities, making background into foreground and the foreground into background. An example is shown below. The right image is the complement to the image in the left.

back

Local normalisation

The local normalization of f(x,y) is computed as follows:

where:

f(x,y) is the original image

mf(x,y) is an estimation of a local mean of f(x,y)

sf(x,y) is an estimation of the local standard deviation

g(x,y) is the output image

The estimation of the local mean and standard deviation is performed through spatial smoothing. In the snapshot below, neighbour size is determined by the standard deviation of normal distribution. After the local normalisation, cells with local intensities can be seen clearly.

back

Morphological transform

Note that the menu provides common morphological operations which are seldom required in CellTracker. Ordinary user may skip the text about this menu.

Morphology is a broad set of image processing operations that process images based on shapes. Morphological operations apply a structuring element (SE) to an input image, creating an output image of the same size. The most basic morphological operations are dilation and erosion. In a morphological operation, the value of each pixel in the output image is based on a comparison of the corresponding pixel in the input image with its neighbours. In the CellTracker, we only use a structure element with ‘disc’ shape. By choosing its size, you can construct a morphological operation that is sensitive to specific object in the input image.

Structuring Elements

In CellTracker, all structuring elements (SE) are two dimensional, consisting of a matrix of 0's and 1's, typically much smaller than the image being processed. The centre pixel of the structuring element, called the origin, identifies the pixel of interest -- the pixel being processed. The pixels in the structuring element containing 1's define the neighbourhood of the structuring element. These pixels are also considered in morphological processing.

Dilation and erosion are two fundamental morphological operations. Dilation adds pixels to the boundaries of objects in an image, while erosion removes pixels on object boundaries. The number of pixels added or removed from the objects in an image depends on the size and shape of the structuring element used to process the image.

This example dilates a simple binary image containing one rectangular object.

0     0     0     0     0     0     0     0     0     0
0     0     0     0     0     0     0     0     0     0
0     0     0     0     0     0     0     0     0     0
0     0     0     1     1     1     1     0     0     0
0     0     0     1     1     1     1     0     0     0
0     0     0     1     1     1     1     0     0     0
0     0     0     0     0     0     0     0     0     0
0     0     0     0     0     0     0     0     0     0

To expand all sides of the foreground component, the example uses a 3-by-3 square structuring
element object. After dilation,
Close
 dilates an image and then erodes the dilated image using the same structuring element for both operations.
Opening
 erodes an image and then dilates the eroded image using the same structuring element for both operations
Top-hat filtering
 subtracts a morphologically opened image from the original image. This can be used to enhance the contrast in an image
Bottom-hat filtering
 subtracts the original image from a morphologically closed version of the image. This can be used to find intensity troughs in an image
H-maxima transform
 suppresses all maxima in image whose height is less than h, where h is a scalar
H-minima transform
 suppresses all minima in image whose depth is less than h, where h is a scalar
Regional maximum
 find the regional maxima of image. It returns a binary image that identifies the locations of the regional maxima in image. BW is the same size as the original image. In the image BW, pixels that are set to 1 identify regional maxima; all other pixels are set to 0
Regional minimum
 computes the regional minima of image. The output binary image BW has value 1 corresponding to the pixels that belong to regional minima and 0 otherwise. The image BW is the same size as the original image.
Extended maximum transform
 compute the extended-maxima transform, which is the regional maxima of the H-maxima transform. Here h is a nonnegative scalar
Extended minimum transform
 compute the extended-minima transform, which is the regional minima of the H-minima transform. Here h is a nonnegative scalar. Morphological reconstruction processes one image, called the marker, based on the characteristics of another image, called the mask. The high points, or peaks, in the marker image specify where processing begins. The processing continues until the image values stop changing.

Note that reconstruction-based opening and closing are more effective than standard opening and closing at removing small blemishes without affecting the overall shapes of the objects. In Open by reconstruction, the output image are reconstructed using the eroded image as marker and the original image as mask.For an image expressed by fluorescent protein, the visible effect of morphological processing is given below.

Dilation | make image brighter
Erosion | make image darker
Opening | darken bright details
Closing | brighten dark details

back

Texture

In most cases, we look at objects according to their intensities. That is not helpful for objects of a texture pattern. CellTracker provides three measurements of texture, i.e. local entropy, local range and local standard deviation. One has to choose a proper structuring element as neighbourhood. For pixels on the borders, CellTracker uses symmetric padding. In symmetric padding, the values of padding pixels are a mirror reflection of the border pixels in image. A snapshot of local entropy is shown below.

Entropy filer: each output pixel contains the entropy value of its neighbourhood around the corresponding pixel in the input image.

Range filter: each output pixel contains the range value (maximum value - minimum value) of the neighbourhood

Standard deviation: each output pixel contains the standard deviation of the neighbourhood around the corresponding pixel in the input image

back

Apply pre-processing on whole images

In the CellTracker, all the pre-processing is done on real time, i.e., pre-processing is done frame by frame when you browse the image. That could increase the browsing time significantly. You may apply the imposed pre-processing on the whole images immediately, which can not be undone. To reduce the browsing time, one may also use the Image (original) mode (under View-Image menu) skipping image pre-processing. The operation requires lots of memory.

back

Remove pre-processing

Remove all the pre-processing imposed on certain image channels. Tick off the channels you wish to keep the pre-processing. If you are going to remove the last pre-processing only, please use the button [image] in the top toolbar.

back


Boundary menu

Each boundary has the following properties

ID:
 a unique integer as boundary ID
Data:
 a two column matrix with x, y coordinates
Control Point:
 spline control point to control data
Tag:
 boundary type
Label:
 boundary name
Label Position:
 one may label a boundary at a couple of positions
Children:
 sub-boundary IDs
Parent:
 parent boundary IDs
  • Boundary detection
  • Boundary properties
  • Convert into
  • Transform into spline form
  • Processing
  • Tracking

Boundary detection

This menu includes several methods to generate boundaries from image data. They could be used to initialise cell boundaries. CellTracker includes the following methods

  • Active contour
  • Direct thresholding
  • Normal level set
  • Seeded watershed
Active contour

Parameters of active contour are shown below. If the result contour is very noisy, one may increase the number of iterations and the values of Beta. However, the key is to get as clear edges as possible.

back

Direct thresholding

This method gets object boundary by thresholding. There is only one parameter, threshold adjustment factor. The method has no constraints on the object shape, but the default cell and nuclear size information (menu-File-Cell information) may affect the result. One may decrease/increase the size criteria if necessary.

back

Normal level set

This methods works on ordinary image data assuming that that image intensities inside an object can be simplified as constant. Good result may be expected if image is blurred in pre-processing step. The method may be understood as segmenting image according to the image intensity. There are two parameters. The first parameter is the number of phases in the image. Usually, image has two classes (phases), i.e. background (low intensity) and foreground (high intensity).

back

Seed watershed

In seeded watershed, a marker is required, which tells where the genuine cells are. Nuclei are often used as markers. One may also draw an approximate boundary as marker. In the former case, we wish to keep the existing marker, the first check box will be checked on. In the latter case, we may use the estimated boundary to replace the marker. In the seeded watershed algorithm, a binary image above detection level is used to estimate cellular region. If the boundary obtained is smaller than the real one, decrease the detection level; otherwise, increase the level. Use Preview to check the result of thresholding.

back

Boundary properties

Show the properties of each boundary in the current frame

back

Convert into

Convert the selected type of boundary into another type, this applied to all boundaries with the same ID.

back

Transform into spline form

Normally, boundary is a collection points. Boundaries in spline form have a number of control points. One may change boundary by adjusting these control points only. This operation extracts key points from boundaries, which will be used as control points

back

Processing

Post processing boundary operations are listed in the following figure. One may set the extent of expansion or shrinkage by entering a multiplication factor. For the boundary size refinement, one may click the button Apply for several times until the results are satisfactory.

In this mode, one may choose a frame range and the boundaries for processing.

back

Tracking

Tracking by edge using template affine transform: Suitable for tracking boundaries with limited shape variation.

Tracking by edge using key frame templates: suitable for boundaries with mild shape changes.

Tracking by colour: use a fixed contour to track a region, e.g. region stained by nuclear dye.

Seeded watershed: Suitable for tracking boundaries with lots of changes in shape and size

The general interface is shown below. Select boundaries and a frame range for tracking and then set Parameters for each method.

Tracking by edge using template affine transform

This method assumes that the cellular boundaries undergone translation, rotation and scale changes over the time. One may use the current boundaries as templates. If the boundaries are close to ellipse, one may also use ellipse as the shape templates, which often gives robust results. In order to find the position of the boundaries, the algorithm generates Nsample contour randomly within a possible ranges of translation, rotation and scale change. To estimate how well a boundary fit in the image, the CellTracker generates Nnormal normal line along the boundary. These normal line then detection if there are edge within a range of m. The method then uses the position of edges to give each boundary a weight. The estimated position of cellular boundary is the weighted sum of all the boundaries generated. In most processes, the shape of nuclei has rather limited change in shape, which may be simplified as affine transform between frames. This method is therefore often used for tracking nuclei.

Tracking by edge using key frame templates

User may select a few key frames as templates. The boundaries between key frames is regarded a linear combination of those on key frames. The motion in the current version is limited to translation only. The algorithm also uses edge information for tracking, therefore, smoothing is required for image with low signal-to-noise ratio. The advantage of this method is that it works if there are interfering cells nearby, i.e. cell overlapping. Another advantage is that there is no limitation on the number of key frames, however, due to the limitation of motion, i.e. no rotation, it has a limited number of applications. One main application of the method is for post-processing. Overlapping cells may also be labeled as the tracking objects. In that case, one may use the boundary before and after the overlapping period as key frames and do tracking over the period.

Tracking by colour

This method assumes that the histograms of the tracking are very similar between frames. It uses the colour appearance for tracking. Compared with edge based tracking, the algorithm is robust to regions with low signal-to-noise ratio. Compared to method I, the motion is limited to translation and rotation only. In many cases, the histograms of cellular compartments vary lot in a biological process, and due to the limit in motion, the method is mainly suitable for tracking the regions with little change in appearance, e.g. nuclei stained by dyes.

Seeded watershed

The method requires a seed for each tracking target and a detection limit to find the region covered by cells. Once the region is obtained, it classifies the foreground pixels using the distance to seeds and the gradient of image. There is another parameter regulation parameter to balance the weight between the distance and image gradient. In this method, there is no limitation on the shape and size of the boundaries. It is therefore often used to track cytoplasmic boundaries using nuclei on each frame as seeds.

back

Refine by active contour

Refine boundaries using active contour (snakes). Active contour is often not suitable for tracking even moderate motion, however, it may be used for refining the boundaries after tracking, since tracking often gives a reasonable initiation. Parameters of active contour are shown below. If the result contour is very noisy, one may increase the number of iterations and the values of Beta. However, the key is to get as clear edges as possible.


Cell menu

  • Cell detection
  • Group cell boundaries
  • Label cells
  • Simple cell tracking
  • Advanced cell tracking

Cell detection

Detect cell boundaries in the ROIs. It assumes that nuclear region is significantly darker or brighter than cytoplasmic region if there is no channel stained with nuclear dye. In that case, problem will arise if the two kinds of cells overlapping in the ROIs. If there is a channel stained by nuclear dye, please tick the corresponding option in the window.

Group cell boundaries

This menu is to match a pair of cytoplasmic and nuclear boundaries. Boundaries drawn manually are independent. To calculate properties involving both types of boundaries, the boundaries have to be matched to make a cell complete. For methods to track both the nuclear and cytoplasmic boundaries at the same time, one may apply the grouping on the starting frame. The boundaries produced by the CellTracker will keep the relations during the tracking process. If the nuclear and cytoplasmic boundaries are obtained in several different steps, one may have to group the boundaries over the tracking range afterwards.

Label cells

Display cell labels in current image. To change labels in a number of frames, use Edit-Rename. One may delete the labels in the current frame using Edit-Delete-Boundary labels.

Simple cell tracking

It asks a few questions about the motion and variation of nuclei and cytoplasms. In the procedure, it assumes that nuclear shape doesn’t change very much; it is subject to translation, slight scale change and rotation. It has no limitation on the size and shape of cytoplasmic region. If there are interfering cells, the cytoplasmic boundaries may be large than it should be. In this case, one may mark the interfering cells as user defined boundaries before hand. The method is suitable for separated cells, or overlapping cells without much motion. The tracking of nuclei is based on edge information. Smoothing is required for image with low signal-to-noise ratio. The approach can be applied for gray level or binary edge images.

Advanced cell tracking

This is a cell tracking wizard. Depending on the cell motion and shape variation, it gives users an integrated interface for cell tracking. Compared with the Simple cell tracking, it offers much more procedures (see the chart below). It features at

  • Support tracking nuclei and cytoplasm using different channels
  • Support tracking using nuclear dye information
  • Support tracking nuclei using both edge and appearance information, if there is a channel with clear nuclear appearance available.

The wizard gives a combination of tracking algorithms. It includes all the combinations listed below.


Graph menu

  • Cell properties
  • Image contour
  • Image 3D surface
  • Profiles

Cell properties

Plot the cellular boundary properties. The CellTracker will generate a graph window for each cell. In the snapshot below, ‘Frame range’ refers to the range of frames for display. One may Select a range or input a valid range expression. Use the Add and Remove buttons to change channels for export. The selected channels are shown in the upper list box. The lower list box shows the properties for export. Add (Remove) properties into (from) the lower list box for plot.

Image contour

Generate a contour plot of the current image. An example is shown below.

Image 3D surface

Generate a 3D plot using image intensity. One may zoom and rotate the 3D plot.

Profiles

Use mouse draw a curve, the CellTracker will generate the intensity profiles beneath the curve (the dashed line)

The corresponding profile is shown below


Tools menu

Draw

The three items, rectangle, spline curve and freehand curve, under the menu corresponding the button [image], [image] and [image].

Edit

This menu has the function as the tool [image], i.e. Editing mode switch. The menu is shown as 'Edit on' when the editing mode is off, and 'Edit off' when it is on.

Label

This is the same as the tool [image] in CellTracker's toolbar. It is shown as 'Label on' when the labelling mode is off, and 'label off' when the mode is on

Selection

The two submenus, Lasso and Object selector, correspond to the tools [image] and [image]. The former is used to select boundaries in a region.The latter could be used to select individual ones.

Zoom

This is the same as the tool [image] in CellTracker's toolbar. It is shown as Zoom on' when the labeling mode is off, and 'Zoom off' when the mode is on


Window menu

CellTracker will produce N figures of cell properties after tracking N cells (Graph-Cell properties). This could be a problem if there are quite a few figures. With the Window, one may tile or cascade, or close all the figures.

  • Tile
  • Cascade
  • Close all

Tile

The CellTracker will decide the optimal layout of N figures. There will be no overlapping between figures.

Cascade

Cascade CellTracker figures.

Close all

Close all the CellTracker figures (not windows).


Help Menu

  • Contents
  • Demos
  • Image processing wizard
  • About

Contents

A list of help information available.

Demos

A set of tutorials for users to get familiar with main functions in CellTracker. In the current version, each demo is zipped into a file, containing raw data, can be downloaded at the http://www.dbkgroup.org/celltracker. The corresponding result (session file) is in the subfolder examples of the CellTracker. One may load the result from the menu File-Load-Saved session.

Image processing wizard

Advise user on which image pre-processing method to use.

About

About the CellTracker.


Demos

  1. Basic tracking procedure

  2. Cell boundary operations

  3. Track overlapping cells using tracking wizard

  4. Track cells using key frames

  5. Tracking cells with low signal-to-noise ratio

  6. Track cells with nuclear dyes

  7. Track cells using edge information and boundary refinement


Basic tracking

Basic concepts

Boundaries: there are three types of boundaries in CellTracker, i.e. cellular, nuclear and user defined. Each boundary can be converted into the other types. The style of boundaries can be modified using the tool [image] in the top toolbar. Each boundary has the following properties

ID: | a unique integer as boundary ID
Data: | a two column matrix with x, y coordinates
Control Point: | spline control point to control data
Tag: | boundary type
Label: | boundary name
Label Position: | one may label a boundary at a couple of positions
Children: | sub-boundary IDs
Parent: | parent boundary IDs

ROI: region of interest. CellTracker constrains image processing and tracking in ROIs in order to reduce computation time. Unlike boundaries, ROIs are for the whole time series. There may be multiple ROIs in a time series, which are supposed not to overlap with each other, although it may not a problem in most cases. One may draw a rectangle ROI using the tool [image] in the CellTracker toolbar. One may also draw a user defined boundary and convert it into a ROI later, however, ROI can not be converted into other types. This Image menu provides common tools for image pre-processing. The processing is applied for the regions inside ROIs of the whole image series.

Basic tracking

In many cases of cell tracking, the shape of nuclear does not change very much while the shape of cytoplasm varies a lot. One may use the information of one channel to tracking cell boundaries. In this demo, we demonstrate the tracking using simple cell tracking procedure. In the image series, no cells are stained with nuclear dye.

1 Click the CellTracker icon [image] on your desktop. The start-up window of CellTracker is shown below.

If it is the first time for you to run the CellTracker, click menu File-Preference to choose a proper data load mode and tip display mode. Click Data Folder in the list box to set the default data folder. Click Result Folder to set the default folder for storing results.

2 Click File—Read time series image-Carl Zeiss LSM file and select the file demo1.lsm. A message will pop up telling whether the file has been loaded successfully or not.

The CellTracker interface after loading the data is shown below. Follow the link (main window) to have an idea of main controls in the window.

3 Browse the image sequence using the button [image]; switch channel using the control [image]. Suppose we are going to track the cells in the lower left corner. It seems that the cell S/N is higher in Channel Ch1-T2. Here we switch to the channel Ch1-T2 first, then draw a ROI using the tool [image]. The cell tracking will be limited in the ROI. To produce cell boundaries automatically, select the menu Cell-Detection. With the default parameters, it is expected to get the cellular and nuclear boundaries as shown in the figure below. In the cell detection programme, it is assumed that nuclei are holes or white bright spots in the fluorescent images if the nuclei have not been stained. Cell detection fails if the nuclei are very close to the cell edges.

Note that one may improve the signal-to-noise ratio using average or median filter (menu Image-Blur-Average/Median). The Image menu provides common tools for image pre-processing. The processing is applied for the regions inside ROIs of the whole image series. If there is no ROI marked, the CellTracker takes the whole image for processing. For most methods, there is a preview window so that users may preview the result while tweaking parameters. The preview window of average filter is shown below. The left shows what the image and boundary look like after processing. The upper right controls are for parameter tweaking. One must tick the preview on to get a preview. Clicking Apply will close the preview window and impose the pre-processing on the image series. Click Cancel to close the window without any action. Use Reset button to set the parameters the default values, and tick off the preview check box.

image pre-processing

4 If the cell detection fails, one may use the drawing tool to mark its boundary manually. The tool [image] is for drawing a closed spline curve. With the drawing mode, click key boundary points with the left mouse button and right click to finish. Draw a nuclear boundary with the same procedure. If necessary, click [image] and [image] to zoom in and out, respectively. Note that the default boundary type is user defined. Right click each boundary to convert it into cellular boundaries. Each boundary is given a name started with unnamedxxx. One may use boundary context menu to rename. To show boundary name, one may first turn on labeling mode using [image] in the top toolbar. In the labeling mode, click each curve to produce a name label. After the labels appear, one may click and drag it with the left mouse button to move the label to elsewhere in the image. Right click the label to delete. The nuclear and cytoplasmic boundaries produced by the cell detection have been grouped automatically according to cells. The nuclear and cytoplasmic boundaries drawn manually are independent. They need to be matched in advance to calculate the properties involving both types of boundaries, e.g. nuc/cyto ratio. One may use the menu group cell boundaries under the cell menu to do so. Once a pair of nuclear and cytoplasmic boundaries have been grouped, the relation will be kept in the followed tracking processes. For a simple cell tracking, it is only required to match cellular boundaries on the starting frame.

5 Click the main menu Cell-Simple cell tracking. The algorithm tracks nuclear boundaries and then use the nuclei as the seeds to find cytoplasmic boundaries. There are no constraints on the shape and size of cytoplasmic boundaries. The cytoplasmic boundaries obtained by this algorithm are prone to interfering components. One may draw the interfering cells as user defined boundaries before tracking, which works only for cells with mild motion. There are three tabs in the simple tracking interface. Select all the frames (1-10) in the Readme tab. Choose motion parameters under the Nuclei tab, and set the Parameters (the lower figure below). Here we use the default parameters except that the nuclear motion type is translation, rotation and size variation (you may try other options as well). Click OK to finish. Note that if there are only nuclear boundaries visible (cytoplasmic boundary unavailable or hidden), the procedure will track the nuclear boundaries only). In the Cytoplasms tab, choose a suitable motion type for the cell. Use preview to check whether the detection level is proper, above which the region will be regarded as foreground. Click Start to track the cells over the image sequence.

Nuclei tab of simple cell tracking

Set parameters in nucleus tab in the simple cell tracking

Here CellTracker uses the default cell motion parameters for the image sequence, i.e. size variation, displacement and rotation. The full default parameters may be viewed using the menu File-Cell information. See appendix for more details on how to set default parameters. Changes in the above window will saved as the default motion parameters for the future use of the current image sequence.

Cytoplasms tab in simple cell tracking

Parameters for cytoplasm tracking

6. Use the controls in the main window to browse the tracking result. To track other cells in another ROI, please delete the current ROI first to avoid tracking the cells in the old ROI again. Cell tracking is by default applied for all the marked cells inside ROIs and cells outside ROIs will not be considered. Note that CellTracker does support tracking using multiple ROI simultaneously.

7 Click the menu File-Export-Tracking result into Excel. Select all frames but remove the channel ChD from the list box, as it is a bright field channel without any gene expression information. Click OK and then input a proper output file name. Use Add button to export more properties if necessary.

The spreadsheet layout of tracking results is expected to be like this. They are sorted by cell, property and image channel. The first column contains the file information, which is only available for Carl Zeiss images. One may view the information from the menu File-Imaging information.

8. Click menu Edit-Copy image into clipboard. All the elements in the current image will be copied into the windows clipboard, and then paste it into MS Excel. This step is not essential, only for the user’s record.

Note that the results produced in this demo are saved in the files demo1.ssn and demo.xls in the examples folder. To load the result, click the menu File-Load-Saved session, which will overwrite the result produced by the user. To save your current session, click the menu File-Save-Current session.

9. One may also have a look at the result inside the CellTracker. Click menu Graph-Cell properties. The interface is very similar to the export-to-excel window. The tracker will produce a figure window for each cell. Each property is shown as a subplot of the window. One may Remove the unwanted properties before clicking OK.

There are quite a few tools in the figure toolbar. The function of each tool is shown below

zoom in

zoom out

rotate 3-D

show /hide colour bar, not useful for this kind of graph.

pan

data cursor

*

show/hide legend

Data cursor is probably the most useful tool. If the tracking results of some frames get wrong, you may use the data cursor tool to find out the frame number (X coordinates). When the data cursor mode is on, right click the current axis to see other options in the data cursor mode.

Appendix: Cell information window

Here is a form of default cell information, clicking the Default button to generate cell information.

If there is no boundary over the whole series, the cell tracking will produce the default ones itself.

If the cell boundaries of the current frame have been marked, the CellTracker will use them for the parameter estimation. Obviously, there is no way to estimate the parameters such as velocity and rotation with only one frame.

If a series of cell boundaries have been available (more than five frames at least), the tracker will ask the user to select a range for motion parameter estimation.

To track an image sequence, one may first use the default motion parameters to track a number of frames, and then choose a range with acceptable tracking results to estimate the motion parameters. One may use the estimated parameters to track the rest frames. In this demo, click menu File-Cell information and then Default button after the tracking. A window pops up asking a range. Using all the frames, one may get the estimated parameters as shown below.

Select frame range for estimating motion parameters

The estimated motion parameters.


Demo2

Objective: Boundary and the related operations including select, copy, deletion, editing and processing.

1. Click the icon [image] on your desktop to start CellTracker. Click File—Read time series image-Carl Zeiss LSM file and select the file demo2.lsm. A message will pop up telling whether the file has been loaded successfully or not.

2. There are only two channels in the image. The channel Ch2-T1 has little information about the cell signalling process. Switch the current channel from Ch2-T1 to Ch1-T2 using the channel pop menu [image]. You may get a main window like this.

3. Suppose we are interested the two cells in the middle of the image. Browsing the sequence, we notice the cells are almost stationary. In this case, there is no need to obtain cell boundaries using any tracking algorithms. Draw a ROI first, and then detect cell boundaries using cell detection (menu Cell-Cell detection). One may get a result as below.

4. One may copy the boundaries to the other frames one by one using boundary context menu. one may also select a group of boundaries, and copy them by the menu Edit-Copy. The interface is shown below.

To copy boundaries, one must choose the specific boundaries in the list box. Here we copy all the boundaries, select all frames and click OK.

The interface of deletion is

Note that the window won’t disappear after this action since one may do more about the boundaries. After each operation, the contents in the interface will be updated immediately. Click the button Close to finish. The window will also be closed automatically if there is no boundary left in the current frame. The CellTracker only lists all the visible cell boundaries. If users select boundaries using the options below the separator in the deletion mode, the related hidden boundaries will be deleted as well.

5. The CellTracker name boundaries in the series as unamed1, unnamed2 and so on. If you don’t like the default boundary names, use boundary context menu to change their names. The labels of the same boundary will be modified over the whole series. One may also rename using the menu Edit-Rename, with which you may change both boundary type (Tag) and name for a frame range. Select a boundary by combining the boundary type and name filters first, and then select the new boundary type and enter a new name, click the button Apply to change. After that, the window will be updated immediately to reflect the change. The menu Cell-Label cells will display the labels of cytoplasmic boundaries in the current frame.

6. To display cell compartment properties in the CellTracker, click the menu Graph-Cell properties. Choose the properties you are interested in, and click Ok to get a graph window for each cell. If you suspect there is something wrong in one frame, turn on the data cursor mode to find out the corresponding frame number. One may return that frame to improve boundaries later.

Here we go to frame 20 (Click the menu Edit-Go to (Ctrl+F) ). We found the nuclear boundary smaller than that it should be. Turn on editing switch using [image], and then click the boundary you wish to edit, there are red circles (control points) coming out. Click and drag the control points to change the boundary shape. Click and drag anywhere other than the control points to move the whole boundary. Click [image] to finish editing. You may undo your last operation using the button [image] in the top toolbar.

7. Select boundaries

One can select a group of boundaries for processing (e.g. transform into spline form, convert into a certain type, delete or processing). There are several ways to select boundaries

  1. Menu Edit-Select all and Edit- Invert selection
  2. Use [image] to select boundaries in a region.
  3. Use [image] to turn the selection mode on, click individual boundaries to select. One may find out the selection mode ( [image] or [image]) by inspecting the status bar at the bottom of CellTracker’s main window.
  4. If you wish to deal with all the boundaries, one may click the corresponding function menu (button) directly. A message will pop up asking if you wish to apply the operation on all the boundaries in the current frame.

8. Transform into spline.

Normally, all the boundary produced in CellTracker is a smooth spline curve with control points except the boundaries produced by [image], i.e. freehand closed curve. You may select it and transform it into spline form using the menu Boundary-Transform into spline form. Note that only non-spline boundary has a context menu ‘transform into spline form’.

9. Processing

You may refine a boundary by expansion, shrinkage, fitting into an ellipse form or converting into convex form in the CellTracker. Select menu Boundary-Processing. One has to select an action, frame range and the specific boundaries for processing.One may select boundaries using the selection methods in section 7 or choose directly from the listbox. Enter a percentage (multiplication factor) in the edit box for expansion/shrinkage, e.g 150% means the boundary after the operation is 1.5 times of the current one. In this case, one may expand the nuclear boundaries a bit.

Boundary list

As a matter of fact, one may also use the tracking interface to copy the cell boundaries in the ROIs. In the simple cell tracking mode (menu Cell-Simple Cell Tracking), choose the options as below.

In the cell tracking wizard (Cell-Advanced tracking), one can simply choose Both nuclei and cytoplasms are almost stationary and Click the Start button to finish the copy operation.

The result of demo is saved as demo2.ssn.


Demo3

Objective: Cell tracking using a certain motion type

1. Click the icon [image] on your desktop to start CellTracker.

2. Click File—Read time series image-Carl Zeiss LSM file and select the file demo3.lsm. A message will pop up telling whether the file has been loaded successfully or not.

3. Browse the image series. Draw a ROI to cover the active region of the three cells as below. Here we track the three cells in the ROI from frame 1 to 130 only, since the cells divide soon after that. As one can see, there is frequent translocation between nucleus and cytoplasm for the channel Ch3-T1. Compared with the other channels, it is more suitable to tracking nuclei. The channel Ch2-T2 correspond the regions of the whole cells. Try the cell detection (menu cell-cell detection) using the two channels. If it fails, use the manual tools to draw nuclear and cytoplasmic boundaries. Make sure to group the cell boundaries (menu Cell-Group cell boundaries). Note that we start from frame 2 in this demo.

4. Here we use ellipse to approximate the nuclear shape, which often gives robust results. The tracking procedure makes use of edge feature for boundary detection. To enhance image, we first smooth image with a window of 5 and then apply local normalisation (image-local normalisation) with the default parameters. The result is shown below.

5. Click the menu advanced unsupervised tracking. Select all the frames of 1-130 for tracking. Click the Parameters button in the nucleus tab and select the motion type translation +rotation +size variation. In the cytoplasms tab, preview the result using different detection limit. The default parameters are suitable for most cases.

Nuclei tab and the corresponding parameter window.

Cytoplasms tab and the corresponding parameter window.

6. Check the tracking result. It seems that the tracker fails to capture the cell motion some times. This is because the motion parameter is improper. Click menu File-Cell information and then the Default Button. Select a range of good tracking result, e.g. 90-130. The CellTracker will estimate a set of parameters using the previous result. Note that one may have to control the size variation at a low level for a robust tracking. Use the advanced cell tracking wizard to do tracking again with the updated parameters as shown below.

7. The result is saved in demo3_ellipse.ssn. Generally, the tracker captures the motion of the three cells. The cell properties of the three cells are shown blow.

The properties of cell unnamed1

The properties of cell unnamed2

The properties of cell unnamed3

8. Notice that the nuclear appearance doesn’t change very much in channel Ch2-T2. One may use it as the reference of nuclei (shown below). Do tracking again. The result is saved in demo3_ellipse_templates.ssn. One may also try to use the bright field channel as nuclear references.


Demo4

Objective: tracking cells using key frame boundaries.

In some cases, cellular boundaries go through gradual changes with little rotation. The boundary in the middle frames may be written as a linear combination of the key frames. There is no limitation on the number of key frames. The method is suitable for cells with limited changes in shape.

To get boundaries of key frames, one may

  • Draw cell boundaries of one frame and group them using group cell boundaries.
  • Copy the boundaries to the rest key frames. Use editing tools to make them fit into the cell boundaries of that frame.

In this way, the boundaries in different key frames have the same ID, Label and boundaries relations.

1. Click the CellTracker icon [image] on your desktop to start CellTracker.

2. Click File—Read time series image-Carl Zeiss LSM file and select the file demo4.lsm. A message will pop up telling whether the data file has been successfully loaded or not. The data have been used in the David Nelson’s paper published in the Science journal.

3. Switch the current channel to Ch3-T2 and browse the image sequence. In this demo, we track cells from frame 2, since the cell position in frame 1 is far from those in the rest frames. Draw a ROI around the cell.

4. Draw the cell boundaries of the cell in frame 2 as shown below and group cell boundaries. Copy the boundaries into the frame 300. Use editing tools to modify the boundaries.

Snapshot of frame 2

Snapshot of frame 300

5. Smooth image using the median filter of size 5 (menu Image-Blur-median).

6. Click Cell-Advanced unsupervised cell tracking. Select tracking range of frames 2-300. In the Nuclei tab, we choose Need more than one shape templates. Since the key frames for both nuclear and cytoplasmic boundaries are the same, we check Use the same frames for cyto tracking. The parameter button in Cytoplasms is thus disabled. One may get the same function using the Boundary-Series-Tracking-Tracking by edge using key frame templates. It is possible to use different key frames for tracking nuclear and cytoplasmic boundaries. In that case, check off Use the same frames for cyto tracking and set independent parameters for cytoplasm tracking.

Nuclei tab

Cytoplsams tab

Parameters window of the nuclei tab

7. The tracking result is saved as demo4.ssn. Click Graph-Cell a property, and choose the frame range 2-300, channel ch2-t1& ch3-t2 and the properties as shown below, one may get a result like this

The interface for drawing cell properties

Cell properties


Demo5

Objective: Track cells with low S/N. Use different pre-processing for nucleus and cytoplasm tracking, respectively.

1. Click the icon [image] on your desktop to start CellTracker. Click File—Read time series image- Carl Zeiss LSM file and select the file demo5.lsm. A message will pop up telling whether the file has been loaded successfully or not.

2. Switch to the channel Ch3-T2. Here we are going to track the four cells in the middle of the image, draw a ROI.

3. Browse the image series. Note that the shape of nuclei has little change while the cytoplasm regions vary a lot. In this demo, we are going to use edge based tracking using template affine transform to track nuclei and seeded watershed for the cytoplasms. The signal-to-noise ratio of the image is low; it is hard to get a proper pre-processing working for both edge based and intensity base tracking (seeded watershed) (see the review of tracking algorithms). For tracking based on edge features, it is ideal to get a clear binary edge image. For tracking based on image intensity features, it is ideal to have constant image intensities within objects. A binary image is an extreme case, which rules out the threshing problem. Here we transform image for cytoplasm tracking first. Note that there is huge difference in cell intensities, it is therefore difficult to find a good threshold over the time series. Since our targets have low signal-to-noise ratio, one may transform the image using texture filter, e.g. entropy. In addition, all the object intensities will then be within a narrow range even if there is sharp difference in the original image intensities. Note that it is only suitable for cells with low S/N if using the image after entropy filtering. The pre-processing is described as follows

  1. Click Image-Texture-Entropy to get an image with entropy values. Here we use a structure element disk of size 4.

    2. Image after entropy filter

  2. Use Image-Convert into binary image to get a binary image. Here a threshold adjustment factor of 0.8 used

  3. Draw used define boundaries for those interfering objects. This is to prevent interfering objects from being labeled as our targets, since the seeded watershed algorithm has no shape and size constraints.

  4. Track using the seeded watershed algorithm (menu Boundary-series-tracking-seeded watershed) with the boundaries on the current frames as the initiation. Here we select both cellular and user defined boundaries for tracking.

4. Browsing the result, there is a concave in the boundary unnamed 11. Here we use Boundary-Series-processing to make the boundary convex. The boundary after processing is shown below.

Image after seeded watershed (frame 21)

Boundary processing: convert the cytoplasm boundary unnamed11 into convex form

5. Remove all the pre-processing on the Ch3-T2 since the pre-processing for nuclear tracking is very different. Here we do the following pre-processing make cell edges clear

  1. Apply median filter of size 5 to make image smooth.

  2. Apply local normalisation using the default parameters.

  3. Apply the canny detection to get cell edges. For image after local normalisation, the procedure direct edge detection could produce false positive edges.

6. Tracking nuclear boundary using the parameters. Here we use low motion parameters. One may try larger parameters and estimate new motion parameters (see demo 1 for more details). After tracking, one may have to group all the cell boundaries if the boundaries on the start frame have not been grouped (menu cell-group cell boundaries)

7. Plot the cell properties. The result is shown below. Note that the tracker is unable to locate objects accurately when the fluorescent intensities inside and outside nuclei are almost at the same level.


Demo6

Objective: Track cells stained by nuclear dye using colour information and wizard

1. Click the icon [image] on your desktop to start the CellTracker.

2. Click File—Read time series image-Carl Zeiss LSM file and select the file demo6.lsm. A message will pop up telling whether the data file has been successfully loaded or not. Switch the current channel to channel CH2-T3, in which cells are stained by a nuclear dye.

3. Draw the nuclear boundaries as below using the tool [image]. Here one may look the image with all channels to find cells for tracking.

4. Click the menu boundary–Series-tracking-Tracking by colour. Here we track the region from 1-95 using the default parameters

Tracking by colour and its parameters

In this tracking algorithm, there are only two motion type, translation and translation & rotation. Whether the method succeeds of course depends on the default translation (and rotation parameters) in the advanced menu. One should not set the parameters too small. Here only cell velocity is relevant regarding the motion of translation. A velocity of 5 is used in this demo. The so-called colour template is in fact a histogram of the region encircled by the object boundary. One may set the number of bins used in histogram. In the algorithm, the CellTracker gets new estimation of the histogram for each object. If the updating factor is 1, we replace the template by the current estimation. If the factor is zero, the tracker will use the initial templates over the whole image sequence. The parameter weight is used to evaluate the difference between the histograms corresponding to each boundary and the template.

5. The result is saved in demo6_color.ssn. Since the motion doesn’t cover the shape change of nuclei, the result won’t be accurate. Generally, this is a robust method to tracking nuclei stained by dye.

Part 2: tracking cells using the cell tracking wizard

1. Browse the image series. Draw a ROI as below. Here we are going to use the channel Ch2-T3 for tracking nuclei and CH2-T2 for cytoplasms.

2. Smooth images using a median filter for both channel Ch2-T2 and Ch3-T3 with a window of 13, respectively.

3. Click Cell-Cell detection using Ch2-T3 and Ch2-T2. Check the option Channel stained with nuclear dye. The initial cellular boundaries are generated as below.

Initial boundaries generated by cell detection. Ch2-T2 (left) and Ch3-T3 (right)

4. Click the menu Cell-Advanced unsupervised tracking. Set the tracking range of 1-95.

Nuclei tab (left) and the corresponding parameter window (right).

Cytoplasm tab (left) and the corresponding parameter window (right).

5. Save the result as demo6_nudye_detection. Plot the cell properties, the result is shown below

Cell labels

Cell properties of cell unnamed1

Cell properties of cell unnamed2


Demo7

Objective: Track both nuclear and cytoplasmic boundaries using one shape template based on edge features; refine boundaries using active contour

For methods based edge features, the tracking result depends on how successful the software finds cellular edges. It will be very helpful to get clear cellular edges after image pre-processing. It is recommended to read how to get a successful tracking before this demo. To find image edges, one may use Canny edge detection under Image menu, which works both for bright field and fluorescent/luminescent channels, however, there may be quite a lot unwanted edges detected. For images in fluorescent/luminescent channel, one may try edge detection using direct edge detection, with which clean edges are often obtained.

1. Click the icon [image] on your desktop to start CellTracker.

2. Click File—Read time series image-Carl Zeiss LSM file and select the file demo7.lsm. A message will pop up telling whether the file has been loaded successfully or not. Browsing the image sequence, one can clearly see nuclear and cell boundaries except frame 1, and that there is little change in the shape of nucleus and cytoplasm over the time course.

3. Draw the boundaries using the tool [image]. If necessary, one may look the image with all channels to find cells locations. Here we get boundaries as below. Make sure to group cell boundaries (menu Cell-Group cell boundaries) before tracking.

4. Click Image-Edge detection-Direct edge detection. Change the intensity threshold so that the binary edges align with the cell boundaries marked above. Here a threshold adjustment factor of 0.7 is chosen in this demo.

5. Click the menu boundary–tracking-by edge using template affine transform. One may also get the same function using the advanced unsupervised cell tracking by choosing ‘need one shape template’ for nuclei and cytoplasm respectively. Here we track the cell from the frame 2-240. Because the boundaries are quite close to ellipse shape, we use ellipse as shape template, which will be applied for all the boundaries in the ROI. We then use motion model of translation + rotation + size variation. In the current version, using ellipse for template generates quite stable results. If you use the current contour as the shape template, the result with a motion model of translation + rotation + size variation may become poor after a number of frames. In that case, one may reduce the percentage of size variation to a very low level, e.g. 1%.

If the tracking result apparently lags behind the cell motion, one may increase the variation range of displacement, rotation and size variation. One may also increase the number of normal lines and normal line length of the Advanced parameters if the current shape templates significantly deviate from the cell boundaries.

6. The result is saved in demo7_ellipse.ssn. The tracking using ellipse shape template is quite successful. In the next, one may use active contour to refine cell boundaries. Click Boundary-Series-Refine by active contour. Here we apply active contour on all boundaries from frame 2-240 using the default parameters. The result is saved in demo7_ellipse_ac.ssn.

(parameters of active contour)

The tracking result may look like this. The boundaries estimated align better with cellular ones except failures on a handful of frames.

More about the demo

A couple of other pre-processings and models may be used for tracking.

  • Apply the median filter of size 13, then tracking using one template (current contour or ellipse) affine transform (saved as demo7_contour_affine_grayimage.ssn and demo7_ellipse_affine_grayimage.ssn)
  • Apply the median filter of size 13 and then local normalisation, track the cell using one template (current contour or ellipse) affine transform (saved as demo7_contour_affine_grayimage_localnorm.ssn).

One may try cell detection after the median filtering. In this case, one has to increase both the nuclear and cytoplasmic detection limit to get a reasonable result.

Cell detection result (right) if the detection limits (0.03) are too low.

Cell detection result (right) with the increased detection limits (0.1) for both nucleus and cytoplasm.


Boundary

 Boundaries: there are three types of boundaries in Cell Tracker, i.e. cellular, nuclear and user defined. Each boundary can be converted into the other types. The style of boundaries can be modified using the tool in the top toolbar. Each boundary has the following properties
ID | a unique integer as boundary ID
Data | a two column matrix with x, y coordinates
Control Point | spline control point to control data
Tag | boundary type
Label | boundary name
Label Position | one may label a boundary at a couple of positions
Children | sub-boundary IDs
Parent | parent boundary IDs

Related: ROI


Export

In Tracker, we track cellular boundaries from image data, then we are then able to estimate the cell properties from boundaries. The relationship is illustrated in the following diagram. Boundaries are overlaid on image. We can get a snapshot of boundary and image data.

Tracker supports a couple of formats. The export can be classified as the above levels.

  1. Image data output
  2. Boundaries
  3. Cell properties
  4. Snapshot of image and boundaries

Data output

  • Export image data as a tiff image sequence: click File-Export-Image as tiff file. Choose image data in the popup menu (Figure 1).
  • Export image data as an AVI video: click File-Export-Image as AVI video Choose image data in the popup menu (Figure 2).

Figure 1: Dialog for export data in tiff image files

Figure 2: Dialog for export data in AVI video files. FPS (frame-per-second) is used to set play speed.

Boundaries

Menu File-Save current session

Cell property output

  • Menu File-Export-Tracking to Excel
  • Menu: File-Export-Result in XML

Snapshot output

Snapshot of a time series

  • Export raw data in tiff file: File-Export-Image as tiff file. Choose snapshot in the popup menu.
  • Export raw data in AVI format: File-Export-Image as AVI video Choose snapshot in the popup menu.

Snapshot of an image

  • Menu Edit-copy image into a new window: copy the current image and boundaries into another figure
  • Menu Edit-copy image into clipboard: copy the current image and boundaries into clipboard


Import

Tracker current supports four types of raw data

  • LSM file
  • Tiff files
  • Tiff file folder
  • Matlab mat file

LSM file

Menu File-Read time series image-Carl Zeiss LSM file. The interface and illustration is shown below:

Tiff files

Menu File- Read time series image -Tiff image files

Tiff file folder

Menu File- Read time series image - Tiff image folder

Matlab mat file

Menu File- Read time series image -Open Matlab file. The Matlab file contains only three variable, X, T,W

X: a cell array, in which each element represent a multi-frame image series at an imaging channel.

T: a vector of time stamp of each frame

W: a cellular string in which each element is channel name.

Suppose we have an image with 50 frames and three channels, A, B and C. In Matlab, the variables T and W can be written as

W={'A', 'B','C'};

T=[1:50]'; % a column vector

And each element of X is a three dimensional array with 50 frames.


Dilation and erosion are two fundamental morphological operations

Structuring Elements

In Cell Tracker, all structuring elements (SE) are two dimensional, consisting of a matrix of 0's and 1's, typically much smaller than the image being processed. The centre pixel of the structuring element, called the origin, identifies the pixel of interest -- the pixel being processed. The pixels in the structuring element containing 1's define the neighbourhood of the structuring element. These pixels are also considered in morphological processing.

Morphological operators

Dilation and erosion are two fundamental morphological operations. In dilation, the value of the output pixel is the maximum value of all the pixels in the input pixel's neighbourhood. In a binary image, if any of the pixels is set to the value 1, the output pixel is set to 1. In erosion, the value of the output pixel is the minimum value of all the pixels in the input pixel's neighbourhood. In a binary image, if any of the pixels is set to 0, the output pixel is set to 0. For example

A SE is 3x3 and looks like:

0 1 0
1 1 1
0 1 0

The centre of the SE is at (1,1).

The source image, A, is 4x4 and looks like:

0 0 0 0
0 1 1 0
0 1 1 0
0 0 0 0

At the first pixel (0,0), with the SE centred on (0,0) none of the SE pixels overlay any source pixels where both are 1. Note: in the implementation, when the SE is off the edge, the nearest edge pixels stand-in for the pixels off the edge. At the second source pixel (1,0), there is a match (SE pixel (1, 2) matches image pixel (1,1)) and so, the intersection of A and SE when SE is at (1,0) is non-empty. So, the output pixel at (1,0) is set to “1”.

After processing the entire image, the output will look like:

0 1 1 0
1 1 1 1
1 1 1 1
0 1 1 0

Dilation and erosion are often used in combination to implement image processing operations. For example, the definition of a morphological opening of an image is an erosion followed by a dilation, using the same structuring element for both operations. The related operation, morphological closing of an image, is the reverse: it consists of dilation followed by an erosion with the same structuring element. For a gray scale image, Opening tends to darken bright details while closing tends to brighten dark details.

Morphological operators in pre-processing

For an image expressed by fluorescent protein,

Dilation: make image brighter

Erosion: make image darker

Opening: darken bright details

Closing: brighten dark details


Abbreviations

Avg: average

Cyto: cytoplasmic

Int: integrated

Nuc: nuclear

Properties

Cell boundary: the boundary covers the whole cell

Cell centroid: centroid of cell boundary

Cell intensity(Int): integrated intensity of the region covered by cell boundary

Cell intensity(Avg): average intensity of the region covered by cell boundary

Cyto intensity(Int): integrated intensity of the cytoplasmic region only

Cyto intensity(Avg): average intensity of the cytoplasmic region only

Nuc intensity(Avg): average intensity of the nuclear region only

Nuc boundary: nuclear boundary

Nuc centroid: centroid of the nuclear boundary

Nuc intensity(Int): integrated intensity of the nuclear region only

Nuc/Cyto ratio(Avg): the ratio between Nuc intensity(Avg) and Cyto intensity(Avg).

Nuc/Cyto ratio(Int): the ratio between Nuc intensity(Int) and Cyto intensity(Int).


Range expression

In the package, it is quite often that user is asked to input a range of frames for analysis. One may Select it from list box. If the input range is irregularly spaced and has a large number of elements, it becomes a tedious job to select from the list box. It is necessary to learn how to generate a valid expression. One may generate a range sequence using colon operator. The expression 1:10 is a row vector containing the integers from 1 to 10, i.e.

To obtain non-unit spacing, specify an increment. For example, 100:-7:50 is 100 93 86 79 72 65 58 51

For a vector can not be presented in a simple expression, one may input a vector by

  • Separate the elements of a row with blanks or commas.
  • Surround the entire list of elements with square brackets, [ ].

For example

[1 3 4 7] is 1 3 4 7

[1 2:5 11] is 1 2 3 4 5 11

Two examples are given below with the corresponding selection in the list box.

Example 1

Example 2


Release note

Release notes

A website, http://www.dbkgroup.org/celltracker/, is created for the CellTracker. Information regarding the package will be post there in the future.

1 New features

  • A new tracking wizard, advanced cell tracking, is added under the Cell menu. It covers most tracking algorithms in the package. It supports tracking using more than one image channels.

2 Modified features

  • Demos have been rewritten to reflect the lastest changes.
  • The tracking algorithms under the Boundary-Tracking have been renamed.

3 Bug fixes

  • Lots of bugs have been identified and removed.

4 Limitation and known problems

  • CellTracker doesn’t support tiff files generated by its export facility, in which each file presents the image from only one channel.
  • CellTracker may crash when double clicking a folder in the preference window.
  • The position of slider in the main window may not indicate the location of the current frame in the series.


Untitled Document

ROI: region of interest. Cell Tracker constrains image processing and tracking in ROIs in order to reduce computation time. Unlike boundaries, ROIs are for the whole time series. There may be multiple ROIs in a time series, which are supposed not to overlap with each other, although it may not a problem in most cases. One may draw a rectangle ROI using the tool in the Cell Tracker toolbar. One may also draw a user defined boundary and convert it into a ROI later, however, ROI can not be converted into other types. This Image menu provides common tools for image pre-processing. The processing is applied for the regions inside ROIs of the whole image series.

Related: boundaires