When a cell is illuminated, light scattering happens due to the interaction of the cell’s complex biological structures with the optical radiation. It has been proven that the rich features in the scattered light distribution are closely related to the morphologies and inhomogeneous refractive index of the external membrane, internal nucleus, and other organelles.1 As a result, the light scattering pattern (LSP) contains the cell’s “fingerprint,”2 which is expected to provide an exquisitely sensitive approach for noninvasive and label-free determination of cellular morphology and, consequently, its use in cytopathic discrimination.3–7 As a prominent example of its application, the forward and side scattering intensities in the LSP are applied in the well-known flow cytometry for cell sorting and identification. Lymphocytes in human peripheral blood are a type of white blood cell generated by the immune system to defend the body against cancerous cells, pathogens, and foreign matter. The morphology changes of lymphocytes deduced from LSP can be used to predict the onset of diseases.8,9 The research in which cells of different types or pathological conditions are identified by their specific features extracted from the LSPs is called the LSP inversion study. The inversion study, a powerful tool to reveal variations of the cell’s morphology and composition from information contained in the scattered light, is a label-free method for cell identification. The key scientific problem in the LSP inversion study is to figure out the specific and sensitive scattering light features and their statistical distributions among the samples. It is important to know in advance the cellular structures that scatter the light in LSP inversion study. Thus, to identify the key features of light scattering from clinical lymphocytes, a three-dimensional (3-D) shape model is set up with the membrane, nucleus, and proper optical structures to mimic their pathological conditions.7,10–20 The LSP of the model is simulated by some numerical methods, such as the finite-difference time-domain (FDTD)21,22 and discrete dipole approximation.23,24 Finally, specific LSP features are extracted according to the statistical principles by analytical methods, such as machine learning.25–28 Above all, an accurate and suitable model of lymphocytes is vital. A simple model is to consider the membrane and nucleus of lymphocytes as ideal spheres or ellipsoids.7,18–20 Recent research29 proposes more precise modeling method to construct 3-D structures from a stack of two-dimensional (2-D) confocal images, in which the real shapes of fluorescence-labeled membrane and nucleus are recorded. To the best of our knowledge, this method gives the best result for obtaining the cell’s 3-D morphology at present. In this work, both manual labeling for different cellular structures and imaging by fluorescence confocal microscopy are used. The labeling quality is rather compromised as about 80% to 90% of cells cannot be constructed directly from the stack of 2-D confocal images because the images of the top and bottom cellular structures are normally incomplete. Some algorithms for constructing 3-D cell structure from its image stack have been reported.29 But these algorithms are focused on how to build up the 3-D structures for well-labeled cells. The stack with incomplete cell structures has to be discarded even though only a small number of image layers in the stack are invalid. As a result, LSP inversion study with the current construction methods cannot meet the requirements for statistical investigations. It is the original intent of our research to improve on the current method. If the invalid image layers in the discarded stack can be recovered by some structural relations with the valid layers, then an approximate constructed morphology with acceptable accuracy could be retrieved from discarded image stacks. There are two problems herein: one is how to determine invalid layers in stack and the other is how to build up the structural relations between the valid and invalid layers. A set of algorithms called the area and shape constraint method (ASCM) is developed by our group to solve these two problems. In this method, the image entropy is applied to determine the invalid layers, and the area and termination shape recursive relation is applied to build up the relation between the invalid and valid image layers. The samples used for verifying the method are clinical lymphocytes of human peripheral blood in suspension and standard spheres with known structural parameters. Three kinds of methods are designed to evaluate the performance of ASCM. The results proved that ASCM is a promising method in 3-D cell modeling for light scattering inversion study and its use in statistical investigations.
The morphologies of membrane and nucleus of clinical lymphocytes are being investigated in our research. Clinical lymphocytes are always in a suspension during the whole process of sample preparation, even during sample imaging by fluorescence confocal microscopy so as to maintain their primitive morphologies. A key problem is how to select the suitable fluorescence dyes. On the one hand, it can select to specifically bind the components of the cells under research; on the other hand, after fluorescence confocal imaging, the dyeing cellular components must be conveniently separated from others often through digital image processing. To meet the requirements, instead of labeling cytoplasm and nucleus of lymphocytes as done in previous research,13–17,29 the membrane marker CM-DiI V22888 (Life Technologies Co., Ltd.) and the nucleus marker Dapi S36968 (Life Technologies Co., Ltd.) are selected to effectively distinguish the edges of nucleus–cytoplasm and cytoplasm membrane. The absorption spectral peak and the exciting fluorescence peak for CM-DiI V22888 are at 553 and 570 nm, respectively, and for Dapi S36968 they are at 364 and 460 nm, respectively. The speed and duration of centrifugation, the temperature for lymphocyte separation and labeling, the washing times after labeling should be well controlled and optimized to prevent morphology damage of the clinical lymphocytes. Our procedure is as follows. Fresh clinical lymphocyte cells, collected within 2 h, are separated from human peripheral blood with lymphocyte separation liquid (Tian Jing Haoyang Biological Manufacture Co., Ltd., China). The separated lymphocytes are then diluted with phosphate-buffered saline (PBS, pH 7.4) to a concentration of to . CM-DiI V22888 is added into 1-mL diluted lymphocyte sample and the sample is incubated for 30 min at 37°C. After incubation, the labeled sample suspension is centrifuged for 5 min with 400 centrifugation force at 37°C. The supernatant is then removed and the leftover is washed gently with PBS (37°C) twice. Another sample is diluted again to a concentration of to with PBS (37°C). Finally, the sample is incubated for 5 min after adding 2 to 3 drops of Dapi S36968. To ensure spatial stability of lymphology during fluorescence confocal imaging, a 20 to drop is dripped on to a 60-mm glass bottom dish with a 14-mm bottom well (In Vitro Scientific Co., Ltd.). The sample drop is controlled empirically with a hemispherical surface in the bottom well. The suspended lymphology is therefore kept almost still by surface tension during the confocal scanning and imaging.
The samples are scanned along the -axis by the laser scanning confocal microscope (LMS780, Zeiss) using a oil immersion objective lens. The membrane (red) and nucleus (blue) are recorded in an RGB image with the resolution of . The step size of -scan is set at . One result of our confocal image stack is shown in Fig. 1.
Area and Shape Constraint Method
Due to the incomplete morphology constructed directly from the image stacks, as shown in Figs. 1(b) and 1(c), the ASCM is proposed. The 3-D morphology generator program of ASCM has been developed, which consists of three steps: image preprocessing, layer restoration, and 3-D morphology construction.
The goal of image preprocessing is to extract the contours of the membrane and nucleus in each image layer. Taking layer #68 in Fig. 1 as an example, the image preprocessing flowchart is shown in Fig. 2(a). Because the membrane and nucleus are labeled in our experiments, it is not difficult to extract the contours of the membrane (red color) and the nucleus (blue color). Normally, there is an unstained cytoplasm region (black color) between them. We first separate the red and blue components from image pixels in the format. For either the membrane or nucleus, only the or pixel channel is retained. The grayscale images for and pixel channel are obtained separately. A Gaussian 2-D window average filtering algorithm is adopted to remove the speck noise. The filtering is carried out via the convolution of the intensity reading of a pixel over a grid of , centered on the pixel, with a kernel of , , and . For the threshold segmentation, the Otsu method, the minimum error method, or the maximum entropy method30 can be applied to separate the membrane and nucleus from the background, and then their corresponding binary images are generated. Due to the similar results for different threshold segmentation methods, we selected the Otsu method with its threshold parameter satisfying the following conditions:Fig. 2. Finally, the contours of the membrane and nucleus are filled in by image filling operations. And the max connected region processing is adopted to remove abnormal signals shown in Fig. 2(b). The reason of abnormal signals is that some unknown reasons make unwanted regions dyed by fluorescence dyes. They sometime occur and cannot be removed the filtering processing. In the case, the max connected region processing becomes necessary.
The purpose of layer restoration is to determine and restore the invalid layers, in which the contours of the membrane or nucleus are imaged unrecognizably or not recorded at all due to the poor quality of dyeing and microscope imaging. In the ASCM method, to determine the invalid layers, the image entropy value method is selected. The area recursive relation and the termination shapes of the valid images are extracted and applied to restore the profiles of membrane and nucleus in the invalid layers. The detailed procedure for layer restoration is as follows:
It is known that the image layers close to the maximum area cross section have better image quality. But the image layers close to the cell’s top and bottom layers are normally fuzzy and not easily identifiable, such as image #1 to #15 at the top and image #118 to #135 at the bottom of the cell as shown in Fig. 1. The main reason is that in the region near the maximum area cross section there are more labeled substances. For layers near the maximum area cross section, the gray values of the membrane and nucleus are close to 255. However, for layers near to top and bottom of the cell, the gray values are gathered in a small range with low pixel values, which are approximately close to those of background noise. According to the Shannon information theory,30 the entropy is a statistical description for the randomness of data and can be used to characterize image textures. An image with a large entropy value contains more information. The definition of image entropy is given by the following equation:31Fig. 1 and display the results in Fig. 3. The abscissa and ordinate in Fig. 3 relate to the image layer number and image entropy, respectively. , the entropy of image #68, equals to 6.86 (arb. unit). Following the definition of an invalid layer, image layer #1 to #24, and #94 to #135 are considered invalid and are highlighted by green frames in Fig. 1. The maximum area cross sections for both the membrane and nucleus shown in Fig. 1 are in the same layer (image #68). But this does not always happen. Therefore, in practice, the invalid layers for the membrane and nucleus should be determined separately.
Three characteristics are required to restore the contours in an invalid layer: the centroid, the area, and the shape.
The position of the centroid is the criterion to align contours in each image layer. Suspended cell movement during microscope scanning can offset the centroid for different layers and can even cause cells to vanish in the field of view. With the surface tension described in Sec. 2.1, centroid offset caused by cell movement is restricted and can be calibrated. The centroids of the contours of the valid image layers are aligned by image processing algorithms (see details in Sec. 2.3.3).
The area of each valid image layer is calculated to establish the area constraint recursive relation. For clinical lymphocytes in human peripheral blood, despite the varying degrees of roughness due to ruffles and folds, the morphology for their membranes or nuclei is basically like an ellipsoid, which can be expressed mathematically as , as shown in Fig. 4(a). The area in the ’th image layer satisfies the equation , where and (). The height of each layer satisfies the equation , where is the scanning distance interval of the confocal microscope. With computed from , , and , the area constraint can be described by the following equation: , where the parameters , , and can be determined from a curve fitting of the areas of the membrane or nucleus of the valid image layers. That is, we assume the area satisfies a quadratic dependence with the layer number . The membrane shown in Fig. 1 is used as an example. After image preprocessing as described in Sec. 2.3.1, images of #25 to #93 are determined as valid layers. The pixel area of the membrane in these layers is obtained [the dotted curve shown in Fig. 4(b)] and then fitted with the quadratic area curve , ( to 93) [the solid curve in Fig. 4(b)]. Thus, in this case, ,, and are found to be equal to , 1578.74, and , respectively. To verify the quadratic relation is a good fit to the data, the relative error between the fitted curve and the data is calculated and shown in Fig. 4(c). It can be seen that their relative errors are small and all , which proves that for lymphocytes a quadratic relation of and is reasonable.
The termination layers in the last valid image at the top and bottom of the cell are used to approximate the areas of the invalid layers by ASCM. For the membrane shown in Fig. 1 as an example, the termination layers are image #25 and #93 (microscope scan from the cell’s bottom to top layer). After deciding the termination layers, its shape is shrunk until its area equals to the value calculated by the area constraint obtained from the valid layers. This is shown in Fig. 4(d). Figure 4(e) shows the reconstructed shape using centroid alignment and applying the area and shape constraint described in step 2.
Three-dimensional morphology construction
To achieve 3-D morphology construction, the centroids of the contours in both the valid and restored invalid layers are calculated and aligned strictly. For the confocal image stack, the resolutions in -, -, and -direction are normally different. In our experiments, the resolutions in -, -, and -direction are 0.04, 0.04, and , respectively. Therefore, interpolation is implemented to improve the resolution in the -direction by inserting an extra layer. First, we fitted the contours in each valid and the restored invalid layer by Fourier series fitting method. The origin of the polar coordinate () is set up at the centroid of each layer. Second, the extra layer inserted is interpolated from the neighboring layers. The known in each layer is used to establish the Lagrange interpolation function32 for determining the interpolated values. The results are shown in Fig. 4(f). After the interpolation, the resolution in -axis is improved to 0.05 μm. For the lymphocyte shown in Fig. 1(a), the final reconstructed membrane and nucleus obtained by ASCM are shown in Fig. 4(g).
Experimental Results and Discussions
Characteristics of Image Entropy Curves for Membrane and Nucleus
To verify that the image entropy curves for different clinical lymphocytes are similar to that shown in Fig. 3, 74 clinical lymphocytes from 15 donors are tested. Their entropy curves for both membranes and nuclei are shown in Fig. 5 for comparison. Figures 5(a-1) and 5(b-1) show the good distribution agreement of image entropy curves for different clinical lymphocytes. Figures 5(a-2) and 5(b-2) are the statistical histogram for 74 clinical lymphocytes. According to Shannon theory,30 the image entropy increases with the value of as can be seen in Eq. (3). Both and are determined from the 2-D confocal images. Therefore, it is necessary to discuss the gray values in images in order to be able to analyze the characteristics of their entropy curves. Using the lymphocyte shown in Fig. 1 as an example, the image layers #15, #30, #45, #60, #75, #90, #105, and #120 are selected. The gray value distributions for both the membrane and nucleus are analyzed. The background noise is normally in the grayscale range from 0 to 50. The gray histograms of the membrane and nucleus after removing background noise are shown in Figs. 6(a-1) and 6(b-1). The entropy curves are shown in Figs. 6(a-2) and 6(b-2). It can be seen that the distribution range of grayscale values is wider for layers adjacent to the maximum area cross section due to the better labeling and imaging, such as image #60. For layers close to the maximum area cross section, their value is larger than those far from the maximum cross-section layer (such as image #120). This is the reason that the entropy in Eq. (3) is larger for layers close to the maximum cross section.
Validation of the Three-Dimensional Constructed Morphology
To validate the precision of the constructed morphology by ASCM, two types of experiments are designed by our group. One is called the valid image layer comparison (VILC) method and the other is the LSP method, which will be discussed in Sec. 3.3.
In the VILC method, a valid lymphocyte image stack is chosen and two neighboring layers are constructed by applying the area constraint. The constructed morphology is compared with the original contours of the valid image layers. The comparison is used to evaluate the precision of the constructed morphology by ASCM. In the range of valid image layers, the contour of the newly constructed morphology is labeled as , and the original known contour is labeled as . The centroids of and are aligned for each image layer. Subsequently, the overlapping area [labeled as Area ()] and the total area [labeled as Area ()] are calculated, respectively. The misclassified error (ME)30 is calculated from the mathematical expression in Eq. (4). Zero percent shows the best accuracy of ASCM
The 74 clinical lymphocytes analyzed in Fig. 5 are used for the comparison. One of their ME results is shown for both the membrane [seeing the subplot on the top of Fig. 7(a)] and the nucleus [seeing the subplot on the top of Fig. 7(b)]. Due to the different number of valid image layers for each lymphocyte, the maximum area cross-section layer for each lymphocyte is labeled as the zeroth layer. The abscissa of Fig. 7 is obtained after the number of the maximum area cross-section layer is subtracted the number of original image layer. From our experience, the invalid layers at the top and bottom of the lymphocyte account for about 10% of the cell size. In our experiments, the confocal microscope’s step size for the -scan is and sometimes .
For the lymphocyte membrane with diameters 9 to and for the nucleus with diameters 6 to , there are on average 10 invalid image layers for the membrane and 8 for the nucleus for step size -scan. For step size, the number of invalid layers is halved. For one of the 74 clinical lymphocytes, its ME of the th and the th image layer for the membrane shown in the subplot of Fig. 7(a) and the th and the th image layer for the nucleus shown in the subplot of Fig. 7(b) is analyzed.
The box-whisker plots of ME results for all the 74 clinical lymphocytes are shown in Figs. 7(a) and 7(b), respectively. The ME mean values of the 74 samples are listed. It can be seen that all the mean values are , and for th and th image layers of the membrane and nucleus are . For the nucleus, the ME is larger than that of the membrane. The reason is that the number of image layers for the nucleus is less than that of the membrane due to its size; the layers far from the one that determines the shape constraint have lower precision. Thus, compared with the membrane, the th and the th image layers of the nucleus are further from its maximum cross section and thus their MEs are larger. For example, in Figs. 7(a) and 7(b), 8.88% is obtained for the th nucleus image layer, which is larger than 6.57% for the th membrane image layer. It is worthy to note that the VILC method for calculating ME gives an upper bound because only the information in the maximum area cross section and the neighboring two layers are applied to set the area and shape constraint. For the ASCM used in practice, the construction precision is much better than the VILC evaluation results. Therefore, if ME of VILC method can satisfy the construction precision requirements (), the ASCM is undoubtedly more precise in constructing 3-D morphology of the confocal image stack.
Precision of Light Scattering Pattern
In the optical scattering inversion study, the LSP is the key criterion from which 3-D morphology is constructed. Therefore, the LSP is a convincing way to evaluate the precision of the constructed morphology by ASCM. With the help of the commercial software (FDTD solutions, Lumerical Inc, Canada), the LSP is calculated by our programs to construct the 3-D morphology. LSP of ideal spheres with radius , 0.5, and is used to construct the morphology and compared with the actual spheres. The results are shown in Fig. 8. The match is very good over the full range of the scattering angles, which proves our programs are correct and reliable.
The standard fluorescence spheres with diameters 6.56, 9.44, and 10.00 μm [shown in Figs. 9(a-1)–9(c-1), respectively] are selected for ASCM evaluation. The morphologies of standard fluorescence spheres are scanned by a confocal microscope and then constructed without applying ASCM [labeled as “original model,” shown in Figs. 9(a-2)–9(c-2)] and with ASCM [labeled as “constructed model,” shown in Figs. 9(a-3)–9(c-3)]. First, from the constructed morphologies, the constructed models look more like ideal spheres than the original models. For original models, there are serious deformations in the models’ top and bottom surfaces. But for the constructed models, these deformed sections are well restored by ASCM. Second, the original and constructed models are compared with that of ideal spheres as shown in Figs. 9(a-4)–9(c-4). These spheres only differ in their volumes due to the different diameters. As we all know, the scattering intensity in the forward direction is most sensitive to the sample’s volume, and with current commercial flow cytometers, 2 deg to 5 deg and 2 deg to 19 deg are commonly selected to detect the forward scattering light. Thus, the LSP relative errors in the range 2 deg to 5 deg and 2 deg to 19 deg are compared for the original and constructed models with that of ideal spheres. The results are shown in Figs. 9(a-5)–9(c-5). It can be seen that the relative errors for the original models with diameter 6.56, 9.44, and are larger than those of constructed models in both the 2-deg to 5-deg and 2-deg to 19-deg range. Especially for the standard sphere, the relative errors of original model shown in Fig. 9(a-5) are as high as 105.49% and 97.09% in the forward scattering 2-deg to 5-deg and 2-deg to 19-deg range, respectively. On the other hand, the errors of constructed model are only 19.10% and 18.34% in the range 2 deg to 5 deg and 2 deg to 19 deg, respectively. For the standard sphere, the errors decrease from 8.79% of original model to 1.89% of constructed model in the 2-deg to 5-deg range and from 7.82% to 1.86% in the 2-deg to 19-deg range. For the standard sphere, the errors decreased from 19.60% of the original model to 8.07% for the constructed model in the 2-deg to 5-deg range and from 12.00% to 2.20% in the 2-deg to 19-deg range.
In conclusion, ASCM is an effective method for constructing morphology and can be applied to the image stacks that are usually considered as invalid or commonly discarded when using current methods. ASCM therefore would become a promising approach to process abundant clinical lymphocytes data, especially for statistical scattering inversion study.
There are still some other sources of error arising from sample preparation and hardware. In sample preparation, the high quality of cell fluorescence staining is the important step to obtain a clear and accurate 3-D constructed morphology. The specific and sensitive fluorescence dyes should be selected carefully with consideration to cell types, cellular organelles, and excitation light wavelength. During sample dyeing, the operations should follow the procedure recommended by manufactures but need to be adjusted and optimized according to the character and type of cells used. Normally, the excitation light peak wavelength used for fluorescence confocal microscopy may not exactly match with the manufacturer’s recommended peak wavelength of the dye. This can compromise the efficiency of fluorescence imaging for labeled cells. Additionally, in confocal microscopy, the refractive index matching problem and the sample movement along the scanning direction should also be considered. For the refractive index matching problem, if there is a large refractive index difference between samples and solution, for example with polyester samples () in the gelose solution (), the microscope focus position during scanning will be in error. It may cause serious deformation in the constructed morphology. Therefore, calibration is needed before measurements. Another problem is the movement of the sample during imaging. There are two kinds of sample movements. One is the sample moving in the plane perpendicular to the scanning direction. This kind of movement can cause a change in the measured sample location. The morphology distortion by this kind of movement can be corrected by an image processing method. The other sample movement is along the scanning direction, which may lengthen or shorten the sample morphology in the image stack. However, the distortion created by this kind of movement is difficult to correct. To avoid or minimize this kind of movement, we find that samples can be kept almost still by surface tension of the 20 to sample drop as described in Sec. 2.1. For our experiments, the numerical aperture and magnification of the microscope objective lens are 1.4 and (oil immersion), and the samples are confined in special bottomed wells for fluorescence confocal microscopy. The optical aberration of the objective lens and other microscope lenses, as well as the scanning depth due to the refractive indices of immersed oil, and the bottom well can be calibrated by relevant compensation methods. Finally, the larger image size and the smaller scanning step obviously can improve morphology precision, but they cause an increase in the scanning time so that the labeled fluorescents may be bleached. It should be traded-off according to specific circumstances. The image of the size of or and 0.1- or scanning step can be selected for clinical lymphocytes studies.
In the paper, ASCM is developed to construct 3-D morphology of clinical lymphocytes. The internal and external spatial morphology of lymphocytes are accurately constructed and digitized. This is very important in the light scattering inverse problem especially for its statistical research. To validate the precision of ASCM, first, we proved that image entropy can be an effectively standardized to determine invalid layers from valid ones in the confocal image stack. Then, the MEs of constructed morphology, as the worst case are found to be , which can satisfy the requirements of statistical measurements in practice. Finally, the 3-D morphologies of standard spheres constructed with and without ASCM are compared. With ASCM, the distorted sections in the model without ASCM are well restored. The LSPs of both models with and without applying ASCM are then analyzed with the LSPs of ideal spheres. The small relative errors prove the good performances of ASCM. We hope that this work may provide a useful tool in conducting the statistical experiments and investigations for the inversion problem of the clinical cell’s scattering features.
The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.
This study was funded by the National Natural Science Foundation of China (Grant No. 61405155), the China Postdoctoral Science Foundation (Grant No. 2016M602807), the Fundamental Research Funds for the Central Universities, and the Collaborative Innovation Center of Suzhou Nano Science and Technology (NANO-CIC).
Lu Zhang is an associate professor at Xi’an Jiaotong University, China. She received her bachelor’s degree in mechanical engineering in 2002. She received her PhD and master’s degrees in instrument science and technology from Xi’an Jiaotong University in 2010 and 2006, respectively. As a visiting scholar, she did the scattering research for small particles at the National University of Singapore in 2013. Her current researches are mainly focused on bio-optical detections for label free cells and small particles.