2.1.

Yeast Culture

The W303 strain of yeast cells were used for this study. The following procedure was used for the cell culturing: 1. Frozen permanents are streaked out on media plates and incubated overnight at 37 °C until the colonies are visible. 2. A single colony is picked from the plate using a sterile pipette tip and used to inoculate a 10 ml culture of liquid media. 3. Incubate 10 ml culture overnight until saturated. 4. Inoculate saturated culture into fresh liquid media. Incubate overnight, and then dilute in the YPD media to a concentration of approximate 10⁷ cells/ml. (The YPD media contains 1% Bacto Yeast extract, 2% Bacto Peptone, and 2% glucose) This cell solution is constantly agitated using a magnetic stirrer during the experiments to keep the cells from clumping.

2.2.

Stationary Cell Images

A drop of the cell culture media was placed on a glass slide and observed using an Olympus BX51 microscope with a 50×/0.5 objective that has a depth of field of 1 μm. Images of the cells were captured using a CCD camera interfaced with ImagePro software as 8-bit TIFF and later converted to type double in MATLAB. The exposure time was set to 90 min for regular cell images. One image with no-cells in the viewing area was taken to serve as the background image. 100×100 pixel image clips containing only one cell (referred to as cell-clip) were cropped from the raw images to form a data set, and for each cell-clip, the same-size image clip was cropped at the same location from the background image. This clip (background-clip) serves as an estimation of the true background of the cell-clip. Each of these cell-clips were labeled according to the relative size of its bud as class 1 (phase G₁: no bud), class 2 (phase S: small bud), and class 3 (phases G₂/M: large bud). In total, 240 stationary cell-clips were collected with 70 samples each in class 1 and class 2, and 100 samples in class 3.

3.1.

Image Enhancement Sub-Algorithm

This sub-algorithm uses the prior knowledge of the background noise in the microscope field of view to eliminate noise and improve the contrast of cell images, to help aid in the image segmentation. The approach taken by this algorithm is to examine each individual pixel of the image clip and then map the intensity of the pixel toward the mean intensity value if the pixel intensity is within the noise range of the background, or map it away from the mean intensity if the pixel intensity is outside the noise range of the background.

First, a clip is cropped from a blank microscope field of view, and is averaged using a 20×20 mask to produce a background-mean matrix: μ_bg, and the variance of each 20×20 region is also computed to form a variance matrix: σ_bg ² of the background. Then, for the intensity of each pixel in the cell clip, its generalized euclidean distance (GED, also known as Mahalanobis distance) to the background mean is computed:¹⁷

To convert GED^′ back to pixel values, the reversion of Eq. 1 is used, while setting the new mean intensity to 0.5 and hard limiting the intensity range to between 0 and 1. Fig. 2b shows a cell-clip after applying the enhancement algorithm and its histogram. It is obvious that the enhanced clips have lower noise, and the contrast is significantly improved.

Fig. 2

(a) Cell clip and histogram prior to enhancement. (b) Enhanced cell clip and histogram.

This method and homomorphic filters achieve similar results because both are able to increase contrast and normalize the brightness of an image.²³ However, homomorphic filters achieve illumination correction by assuming the inhomogeneities as low frequency components,²⁴ while the report method can completely remove the background information, which include both the nonuniform illumination and random high frequency inhomogeneities. This is especially important, since for the microfluidic cell sorting application of this algorithm, there will likely be unwanted objects in the microscope field of view, including channel walls, debris from fabrication, and dust particles. These objects are stationary and could be completely removed by the algorithm. One drawback with the enhancement algorithm is that the mapping depends solely on the pixel intensity, and certain orientations of the cell could result in some of the cell boundary/interior pixels having intensity values within the noise range. This will result in visible gaps in the cell boundary/interior after enhancement (Fig. 3). A more robust enhancement algorithm would take into account the spatial distribution of the pixels as well as pixel intensities.

Fig. 3

Shape identification steps. Direct threshold could produce a cell boundary region with gaps. An estimate of the cell edge can be obtained with several iterations of pixel dilation and erosion. This boundary estimate can then be added back onto the cell boundary region to fill any gaps in order to obtain the final cell shape.

3.2.

Shape Identification Sub-Algorithm

The enhanced images now have clearly defined intensity differences between the background, cell boundary, and cell interior. By observing the enhanced cell-clips, it was noted that the interior edge of the dark cell boundary represents the shape of the cell very well, therefore a simple threshold method can be used to obtain this boundary shape. However, for some cell images, such an edge may not exist if the cell boundary does not close completely, for example the clip in Fig. 3. A heuristic method is determined to connect and close an open boundary by obtaining an estimate of the boundary and then add it back to the original image. The steps of the shape identification algorithm are shown in Fig. 3.

By enhancing the image before segmentation, the proposed algorithm overcomes the traditional poor performance of threshold based segmentation methods under noisy environments. Another major limitation of this segmentation method—also a common limitation of threshold methods—is the inability to handle multiple cells sticking to each other.¹⁶ However, for the specific application of cell sorting, the cells must be isolated from each other. The segmentation algorithm has the benefits of the threshold-based methods in terms of speed and simplicity.At the same time, for the specific application of microfluidic cell sorting, the general limitations of threshold methods does not limit the algorithm's performances.

3.3.

Feature Extraction Sub-Algorithm

With the cell shapes successfully extracted from the original images, it is now possible to extract specific features from the shapes to classify between cells in different cell division phases. Manual classification of yeast cells is done by first looking for a visible bud (G₁ or other) and then examines the size of the bud (S, G ₂, and M). The automated classifier will also follow this two-step guideline. The major difference between budding and nonbudding cells is their shape: nonbudding cells look like compacted circles while budding cells have a peanut-like shape. Thus, features must be extracted from the cell images that can reflect the shape and bud size of the cells.

The compactness is a geometric feature that is measured by the ratio of a shape's area to its perimeter squared.²⁵ A nonbudding cell should have a lower compactness measurement while a budding cell will have a higher compactness measurement.

The axis ratio (shown in Fig. 4) is the ratio of the shape's major axis to its minor axis. A nonbudding cell, looking more like a circle, will have an axis ratio closer to 1, while cells with larger buds will have larger axis ratios. The quantities needed for calculating these two features can all be obtained using MATLAB's binary region analysis tool.

Fig. 4

Description of features. Left: Major and minor axis. Right: Mother cell and bud.

To extract the bud size feature from the cell shapes, the bud must be isolated from the parent cell. An existing method is look for convex cell boundaries that represent the location of the bud neck.⁸ This method is very efficient but its accuracy can be unreliable. The proposed bud isolation algorithm assumes that the parent cell is circular and tries to locate it using circle detection techniques. First, the area of the cell shape A _c is determined (in pixels), and then its equivalent radius is calculated as √A _c/ π rounded up to an integer (pixel) value. Then, a circular mask of the same radius is created and slid across all the pixels that belong to the cell shape in order to find the location where the mask best matches the cell shape. All the pixels that are covered by the circular mask are then removed and the remaining pixels represent the bud. Then, the bud-to-area ratio feature can be obtained by dividing the size of the bud in pixels to the size of the entire cell.

Of the three features mentioned above, the axis ratio and compactness can be obtained with the shortest time, while the bud ratio is the most complicated feature to obtain and requires more computational time.

5.1.

Image Enhancement

5.1.1.

Threshold parameter

As mentioned in Sec. 3.1, the threshold value n in Eq. 2 has a critical role in the entire algorithm. The value of n represents the cut-off value between the background noise and cell features, and the effect of various values of n is shown in Fig. 6:

Fig. 6

Enhancement result for different threshold value n.

As n increases, more background noise is eliminated by the enhancement algorithm and the edge of the cell membrane becomes sharper. However, a high n value results in unclosed cell membrane boundaries, and subsequently the failure to extract features from the image since the segmentation algorithm only tries to find the inner edge of the cell membrane. A low n value will also result in incorrect features due to background noise.

An experiment is conducted where the image enhancement, segmentation, and feature extraction algorithms are applied to the entire set of cell clips with different values of n. Two criteria were used to evaluate the performance of these algorithms: the number of clips (out of 240) that failed to return a cell shape and the ratio of interclass distance (Sb) to intraclass distance (Sw) of each feature space. The resulting performance is tabulated in Table 2. For n>1, the number of failed clips increase dramatically and thus is not recommended. n = 0.8 has the highest inter- to intraclass ratio, meaning that the classes are densely compact and have good separation; thus, n = 0.8 was chosen as the preferred threshold value.

Table 2

Comparison of image segmentation performance for different n values.

		Interclass distance (Sb) to intraclass distance (Sw) ratio
n	Fail to extract feature	Feature 1	Feature 2	Feature 3	Average classification accuracy
0.5	0	0.384	0.383	0.0017	49%
0.8	1	0.814	1.699	1.656	82%
1	5	0.669	1.556	0.631	81%
1.5	23	0.360	0.886	0.483	49%

5.1.2.

Segmentation and bud isolation

Figure 7 shows the result of the segmentation and bud isolation on several cell-clips. In general, the segmented shapes closely represent the actual cell shape, although the bud sizes are slightly enlarged due to the segmentation algorithm. The segmentation algorithm is not completely fail-proof: an image with a very large gap (4th clip in Fig. 7) in the cell boundary cannot be closed completely, thus resulting in the algorithm returning an empty shape. The bud isolation algorithm overestimates the size of the parent cell, but since the area of a circle is proportional to the square of the radius, the overestimation is fairly consistent even with different bud sizes. This overestimation is introduced purposely to attempt to negate the enlargement of the bud during the segmentation process, and to make sure that small defects on the boundary are not identified as buds.

Fig. 7

Examples of cell clips. (a) original images. (b) segmented images. (c) bud separation result.

5.2.

Optimal Feature Space and Classifier

For each classification method, 120 cells, 40 from each class, were randomly chosen to form a training set and the classifier is obtained using the training set. Then, the classifier is used to classify the other 120 cells, the results of which are compared with their actual class labels below. For cross-validation, each classifier is tested 10 times in each set of feature spaces by selecting 10 random training and testing sets from the original data set. Table 3 shows the confusion matrices of the three classifiers averaged over different feature spaces, and the average classification accuracies of each classifier for each feature space are shown in Table 4.

Table 3

Confusion matrices: average classifier performances classifier results.

Table 4

Summary of classifier accuracies.

The performances for different feature sets show that set 2 results in the most accurate classification. Set 1 is inaccurate because it is 1D in nature, making it difficult to separate overlapping classes. Set 3 results in similar accuracy to set 2.

The performances of different classifiers are very similar, with kNN having slightly lower accuracy compared to the other methods. All three methods can classify samples in class 1 with near 90% accuracy; this was expected, since good separation was observed between class 1 and class 2/3 in most feature spaces.

The Mahalanobis distance method missclassifies many of the class 2 samples. This is mainly due to the inaccuracy in the covariance estimation of the classifier, since class 1 and class 3 have more outliers compared to class 2. The support vector machine classifier is less prone to outliers compared to the Mahalanobis distance method. It can classify class 2 samples more accurately. However, it is still inaccurate when classifying between class 2 and 3. This is expected because these two classes have major overlaps.

kNN was expected to be the most accurate classifier overall, since kNN does not assume any class/boundary shape. However, the results do not show any improvement in accuracy, in fact the kNN results are slightly worse than the Mahalanobis distance method and SVM. The likely reason for the poor performance

is the lack of data points. If more data points were available, the actual shape of the class would be shown more clearly.

Table 5 shows the computational speeds of each classifier algorithm, not including the image analysis. Although kNN requires a much longer time (20 ms) to make the classification decision compared to SVM and Mahalanobis distance, it will not be the bottleneck of the entire image analysis system, since the image enhancement and segmentation require a much longer computational time.

Table 5

Computational speeds of different classifiers.

Based on these results, it can be concluded that feature Set #2, which uses the axis ratio and compactness feature space, and compactness and bud size feature space sequentially, is the optimal feature set for this application. Regarding the most optimal classifier, any one of them could be chosen since they all demonstrate very similar performances. The author suggests using the support vector machine since it has the fastest speed, and if abundant training data is available, use kth nearest neighbor since it does not make assumptions on class or boundary shapes.

5.3.

System Performance Under Different Conditions

5.3.1.

Effect of intensity

In addition to the original set of cell clips, another set of clips were taken with half the exposure setting as the original, as shown in Fig. 8.

Fig. 8

Cell images under different exposure setting (images brightened to shown dark content).

The image enhancement and segmentation algorithms were performed on these clips with the same parameters, and then classified using the kNN classifier obtained using the original clips as a training set. A comparison of the image enhancement results for full exposure and half exposure settings is shown in Fig. 9, and the performances of the algorithms for half exposure setting are shown in Table 6.

Fig. 9

Image enhancement under different exposure.

Table 6

Performance of algorithm under half exposure setting

	Inter-to-intra class ratio
Fail to extract feature	Feature 1	Feature 2	Feature 3	Average kNN accuracy
0	0.610	1.521	1.134	67%

It was noticed that the enhanced image under half exposure conditions has a lower contrast compared to the full exposure image, since the unenhanced image already has a lower contrast due to insufficient exposure.

At the low exposure setting the inter to intraclass ratios are slightly lower to those with regular exposure settings, due to lower contrast and high noise ratio. Despite this, the image enhancement and segmentation algorithms were able to extract meaningful features from the cell clips. The kNN classification accuracy is lower than for the properly exposed images (67% accuracy compared to 77% in Table 4). These observations show that the higher signal-to-noise ratio of low exposure setting has a negative impact on the performance of the system.

5.3.2.

Effect of focusing

Due to the ellipsoidal shape and refractive index difference between the cell content and surrounding liquid media, yeast cells act similar to convex lenses and have significantly different appearances when observed in-focus and out-of-focus.²⁶ The original cell images are taken with the microscope lens focused near the foci of the cells (about 10 μm from the actual cell position), resulting in the bright spot in the middle of the cell.²⁶ This focus setting is referred to as standard focus, and has many optical advantages. It results in bigger cell sizes, a more circular cell shape, and higher contrast between the cells and the background. To test the performance of the imaging algorithms under different focusing conditions, two more sets of cell clips were taken. One set is taken at 5 μm further away from the cell (overfocus), and the second set is taken with the microscope focused on the cell (underfocus, 10 μm from the original focal plane) resulting in the images shown in Fig. 10.

Fig. 10

Appearance of cells under different focal setting.

The overfocused cells generally look larger than the original, especially the buds. The cell membranes are also blurry. The overfocused condition is not an ideal environment for both manual and automated classification, as the bud information is difficult to infer from a blurry image. Some preliminary testing using the algorithm has also shown difficulty to obtain bud size. In general, it is not recommended to use the current algorithm to identify overfocused images.

The underfocused cells generally look darker, and do not show a bright interior spot. This appearance is very different to the standard focused cells. The image enhancement algorithm can still improve the contrast of the cell clips, shown in Fig. 11, but the image segmentation algorithm, which tries to find an interior boundary of the darker regions, fails completely. However, the enhanced images can easily be segmented using a different algorithm. Then the feature extraction and classification algorithms can still be used for underfocused cells.