One of the most common noninvasive techniques to acquire information about transparent biological samples is differential interference contrast (DIC) microscopy. DIC microscopy is effective to study biomedical samples because it accentuates contrast on the edges of the structure in the sample and produces high-resolution images from samples such as tissue cells, eggs, and embryos.1
Thin live cells, essentially 2-D specimens, can be observed and studied with a single 2-D image. However, when imaging thick biological objects, whose complex structures have thickness values larger than the depth of field of the microscope,1 the use of 2-D images does not provide enough information about the morphology of the sample. For this type of objects, the use of 3-D images can provide important information.
Information about thick samples is acquired in 3-D DIC imaging in the form of a stack of images focused at different depths along the optical axis. Each image in the stack contains information from both in-focus and out-of-focus planes. Therefore, even if a stack of images is acquired through focusing, the image at a given focal plane is not simply a representation of a 2-D slice of the object.2, 3 The identification of different regions within the three-dimensional structure of the object consequently becomes a challenge, and further processing of the images may be required for this task.
Examples of such thick transparent samples are mouse and human embryos—both have a complex structure and thickness values that range from . Their study relies mostly on 2-D images from conventional microscopy modalities.4 In this paper, we will image mouse embryos using 3-D DIC microscopy and apply image processing algorithms to extract 3-D information that could be useful for embryo development studies.
An advantage of DIC microscopy as opposed to other wide-field microscopy modalities is its applicability at high numerical apertures. Larger apertures result in higher lateral resolution and better depth discrimination. Additionally, DIC provides high contrast in transparent objects and is sensitive to phase changes. High contrast offers clear edges, and phase changes produce variations in irradiance that appear as different textures in the image. These characteristics allow DIC imaging to be a suitable candidate for the application of texture extraction methods that are based on intensity changes.
Texture extraction methods have been successfully applied to wide-field microscope images of biological samples to both delineate and segment regions of interest. The detection of such regions has been the target of several automatic or iterative algorithmic methods.5, 6, 7, 8, 9, 10 The use of these algorithms varies with respect to the targeted application and microscope modality.
Previous texture detection methods in DIC microscopy images of biological samples have generally focused on the detection of cells or nuclei. Hamahashi 5 present a method for detection of nuclei in the process of cell division using a set of temporal 3-D DIC images of C. elegans embryos. The method combines local entropy and object tracking algorithms. The nuclei are successfully detected by the use of boundary and texture characteristics. Although these embryos are considered thick (thickness is about ); cell nuclei can be easily detected by simply looking at the image. However, in the case of more complex thicker objects such as mouse embryos (thickness is about ), nuclear appearance is not as detectable as in C. elegans embryos, and in many cases, it is undetectable by looking at the image.
Other contributions have focused on addressing both deconvolution and texture detection algorithms for cell detection. Kuijper and Heise’s method7 is able to detect cells properly as long as the cells are distinguishable by the human eye. As the authors pointed out, the method needs to be enhanced to address situations where the cells are not evidently visible. A solution to the cell detection problem that combines DIC and quantitative phase images from optical quadrature microscopy (OQM)11, 12, 13 has been used to develop a cell-counting method in live mouse embryos.14 (OQM noninvasively reconstructs the amplitude and phase of an optically transparent sample.) In that work, a 2-D DIC image was used to extract cell boundaries. Based on the detected boundaries, the method was capable of counting cell numbers ranging from 8 to 26.
Entropy is a characteristic of an image.15 The local entropy of a pixel in the image depends on information from its surrounding neighbors. Neighbor information can be extracted from a 3-D DIC image and used to generate a 3-D local entropy texture image that provides morphological information about the thick sample. Rapid changes of local entropy with focus in dense -stacks provide detailed information about the sample. This information is directly related to levels of texture roughness. Using the variation of texture roughness along the axis provides information about the morphological characteristics of the sample. Moreover, regions with different textures within the object can be detected based on the local entropy.
The current work is focused on extracting information from a dense -stack of DIC images using a local entropy-based texture detection method. Local entropy texture methods have been used as an alternative to classical segmentation methods,5 in part because of their low sensitivity to out-of-focus effects. We use this sensitivity with variation of focus to follow changes in texture within the 3-D positions of the different sample regions, rather than trying to delineate or extract a specific region shape. The extracted information may contribute to understanding the natural processes occurring in such samples. For example, in embryo development studies, where the morphological details are valuable information,16, 17, 18 a wide range of questions can be answered by using an accurate morphology recognition mechanism.
In this work, we focus on embryo development studies, particularly the selection of the most viable embryos for assisted reproduction techniques such as in vitro fertilization (IVF). Recent studies in IVF have shown high pregnancy rates when the transfer of embryos to the mother’s womb occur at the blastocyst stage.19, 20 The blastocyst is one of the later stages of preimplantation embryo development, and its structure is so complex that it defies available image processing algorithms. In this paper, we process a set of 3-D DIC images of blastocyst mouse embryos. Blastocysts are composed of a spherical encasement known as the zona pellucida that surrounds and contains the inner structure (see Fig. 1 ). The inner structure of the blastocyst consists of three primary regions: inner cell, trophectoderm, and blastocoele cavity. Characteristics of the inner regions such as size, expansion, and distribution are used by clinicians to evaluate how well developed the embryo is at a particular time post-fertilization.4, 21, 22
We extract DIC image textures using local entropy concepts to segment the inner cell, trophectoderm, and blastocoele cavity regions in the embryo. The obtained results are validated in two distinct ways, leveraging both fluorescence images and OQM phase images.11, 12, 13 In a direct approach, we compare our results to nuclei locations obtained through fluorescent staining. All nuclei should be clustered together in the inner cell and trophectoderm regions of the embryo. We also use a model-based validation approach computing 2-D quantitative phase images from the reconstructed embryo 3-D structure. We assign previously published indices of refraction23, 24 to the regions obtained from the texture analysis. Then, a forward model25 is applied to estimate phase images. The computed phase images are compared to measured OQM phase images. The texture detection procedure and the two validations methods are applied to DIC images of five different blastocyst embryos.
The rest of the paper is structured as follows: A description of the local entropy texture detection method and its application to 3-D DIC images is presented in Sec. 2. Experiments to validate the obtained information from local entropy images are presented in Sec. 2 as well. Sections 3, 4 present the results and conclusions.
The DIC images consist of 26 slices collected in increments along the optical axis from five blastocyst mouse embryos. The resulting -stack covers along the axis, which is enough to cover the complete volume of the embryo. Each slice is , with a pixel resolution of . Figure 2 shows DIC images of five different blastocyst embryos, and Fig. 2 shows slices at three representative focal planes ( , , ) of a single embryo. DIC image stacks are processed with the local entropy method (see Sec. 2.2) to differentiate regions that contain cells (inner cell and trophectoderm) from those that do not contain cells (blastocoele cavity and spaces between cells).
Next, OQM phase images were collected before staining. OQM is a noninvasive technique that provides amplitude and phase of an optically transparent sample.12, 14 These phase images were unwrapped with a 2-D -norm algorithm.26 Measured OQM phase images were used to validate the detected embryo structure from DIC images.
Last, a stack of 26 fluorescence images was collected after applying to the five blastocyst samples a Hoechst nucleus stain. The fluorescent dye (Hoechst 33342) binds fluorophores to the nucleus of a cell, giving more contrast to the nuclei in the embryo image. In fluorescence images, bright regions provide information about the shape and location of the cell nuclei. Fluorescence images are normalized using a histogram thresholding method, such that intensity values greater than zero (brighter regions) indicate the presence of the cell nuclei. In practice, this modality is not feasible for live embryo applications such as IVF, because the Hoechst stain permanently modifies the DNA of the nuclei. Therefore, this stack is used for validation purposes only.
Detection of Regions Using Local Entropy Image
Our region detection scheme is based on differentiating smoothness levels of image textures. The smoothness levels in a 3-D DIC image are quantified using local entropy.15 Local entropy is defined by the entropy of a pixel window surrounding a point of interest within the image.27 For a rough texture, the local entropy values are high, while lower values correspond to smooth texture regions. Because a smooth texture in blastocyst DIC images is characteristic of the blastocoele, we expect its local entropy to be lower than that in cellular regions, in which a rough texture is indicative of the organelles in the cytoplasm. Our analysis is focused on the distribution of the cells and blastocoele regions in the embryo. Thus, we have removed the zona pellucida from the images by applying an edge detector27 before computing the local entropy images.
We define texture using local entropy in the image as follows: Let represent the input image and an pixel window. The resulting local entropy image is given byis the number of gray levels in , and is the frequency of occurrence of gray-level in window . Each output pixel contains the entropy value of the neighborhood around the corresponding pixel in the input image.
A stack of DIC images is segmented by using a window of size. In other words, for a 2-D image located at , the window covers from to images along the axis. The top and bottom images of the stack were processed with a size window. As a result of this process, a 3-D local entropy image is obtained.
Two validation approaches were used to test the predicted blastocyst embryo structure. In the first approach, we compared the local entropy image with a fluorescence image of Hoechst-stained nuclei. The images were overlapped to identify common regions. In the second validation method, we used the local entropy images to create an embryo model and compute a quantitative 2-D phase image. Then, the 2-D phase image was compared to a measured OQM phase image. Results of these validation methods provide information about the accuracy of the local entropy texture method to predict embryo structure from 3-D DIC images.
Local entropy and fluorescence images
A histogram of the local entropy images was calculated to analyze the distribution of the local entropy values. Figure 3 shows a histogram of the 3-D local entropy images for all five embryos. We observe that the local entropy values follow a multimodal distribution. We apply thresholds to the histogram to divide the data in three different groups. These groups are labeled regions with high, medium, and low local entropy values. High entropy contains values greater than or equal to 1.2, medium values are between 0.8 and 1.1, and low values are those less than or equal to 0.8. Figure 4 shows an overlay of the local entropy (after thresholding) and DIC images.
Since each cell contains organelles with different optical properties, texture variation occurs in each cell. Then, medium and high local entropy values are assumed to be located in the inner cell region, while low values are in the blastocoele cavity. Nevertheless, since regions with high and medium local entropy values represent those occupied by the volume of the cells with their subcellular components, they are expected to be larger than the brighter regions identified as nuclei of the cells in the fluorescence images. Figure 5 shows an overlay of the local entropy (high and medium local entropy regions, after thresholding) and fluorescence images.
To validate the accuracy of the segmented regions, we calculate the percentage of the fluorescent areas that are outside the high and medium local entropy regions. We expect that since the high and medium local entropy areas contain the fluorescent areas, a highly accurate method should yield a low percentage value.
2-D quantitative phase image estimation
The obtained 3-D local entropy image is used in conjunction with our image formation forward model to estimate phase images.25 The calculated image is compared to a measured OQM phase image to validate the detected regions with the local entropy method. The procedure is described in Fig. 6 .
The embryo model is constructed using the regions with high, medium, and low entropy (obtained from the local entropy image histogram, Fig. 3). These regions are each assigned an index of refraction as obtained from published cell modeling work.23, 24 The cytoplasm of the cell has a reported index of refraction of 1.35. The average of the indices of refraction of the nucleus and the cellular organelles is equal to 1.37. Since the presence of the nucleus and different organelles in each cell result in high entropy values, we have assigned 1.37 to the high entropy region and 1.35 to the medium entropy region. The low local entropy region is composed of the blastocoele cavity. Prior embryo analysis has shown us that the blastocoele has an index of refraction similar to the immersion medium, which is 1.33; thus, we use this value for the blastocoele region. Last, we add the zona pellucida to the model, since it was removed from the DIC image to calculate the local entropy image. The zona pellucida is uniform and thin, and its index of refraction can be estimated from a measured OQM phase image. The index of refraction for this region has been estimated to be equal to 1.34.
We generate a phase image of the synthetic embryo described earlier using a forward model25 for thick objects. The calculated phase image is compared to a measured OQM phase image. To evaluate the similarities between these images, we use the Dice metric.28 Let denote a measured image set of pixels in a region , and then the Dice metric for region is defined as , where |.| denotes set size, and is the set of pixels of the calculated image. For the validation, we choose a region that contains phase values greater than . Since phase values greater than do not belong to the blastocoele cavity region, the comparison is performed for a region that contains cells in both the simulated and measured phase images.
Local entropy image stacks are computed from five DIC image stacks of the five blastocyst mouse embryos [see Fig. 2]. The embryos’ volume morphology is identified from the information contained in the local entropy images. This information is assessed with the two validation methods: first, using fluorescence images, and second, computing a phase image from the extracted morphology and performing a comparison between the computed image and a measured OQM image.
Using Fluorescence Images
Validation results using fluorescence images for embryo 1 are presented in Fig. 5. This figure shows an overlay of the fluorescence and the generated local entropy images. We can observe that the high entropy regions almost completely contain the cell nuclei area. The percentage of the region identified as nuclei in the fluorescence images that is outside the area with high local entropy is equal to 3%. This error means that the segmentation (local entropy-based) method failed when de-tecting those regions as low entropy regions. Then, we consider this percentage as an error of the method. Table 1 shows the percentage results from all five embryos.
Validation of local entropy segmentation of DIC versus fluorescence images. %Error is the fraction of fluorescence area outside the inncer cell region segmented from the DIC image.
The reported error values are less than 5% for four of the samples (embryos 1, 2, 3, and 5) and 12% for just one case (embryo 4). If we observe embryo 4 in particular [see Fig. 2], this sample is reaching the blastocyst stage but still has a very small and not well delineated blastocoele cavity with most of the embryo’s area being composed of the inner cell region. Thus, this case serves to show a near to worst case performance of our approach. Even though posing a more difficult case to extract the texture of the regions, our method is able to report only a 12% of error for this case. We can observe that most of the fluorescent areas (from fluorescent images) are contained in the high local entropy region. These results give us confidence that the method is highly accurate when extracting the region textures of a blastocyst mouse embryo.
Using an OQM Image
Estimated and measured phase images are compared quantitatively. The comparison and validation is performed using the Dice metric. The Dice metric is applied to a selected region in the image. Although the same criterion and threshold is used for each image, the size and location of the region varies for different embryo images. Computed and measured phase images for all five embryos and their corresponding selected region are presented in Fig. 7 . We can observe from the images that the estimated phase closely resembles the measured phase. Results of the Dice metric values for the five embryos are shown in Table 2 . Again, embryo 4 being the most difficult case showed a Dice value of 0.71, while the rest provide values of 0.80 or more. These results show that the 3-D object structure generated with the information extracted from DIC images closely resembles the true embryo structure.
Results of Dice metric. The Dice metric is used to compare calculated and measured phase images in a selected region. The region corresponds to phase values greater than 15rad .
This work has explored the extraction of 3-D morphological information from DIC images of thick objects. The application of a texture detection method based on local entropy has been presented as a noninvasive technique to analyze the structure of five blastocyst mouse embryos.
An important advantage of applying this method to DIC images is its low sensitivity to image quality; this allows the extraction of information about the morphology of a thick object. In conclusion, detecting the blastocoele and cell regions of mouse embryo at blastocyst stage using the local entropy method contributes to the understanding of the overall embryo morphology. This may positively improve the analysis of embryo development for assisted reproduction techniques such as IVF.
Additional work must be completed to provide a more general criterion to select a window size that improves the ease of interpretation of the results. The local entropy algorithm can be enhanced by including a thresholding mechanism to have more control in the tuning of the processed images. A detailed analysis of the effectiveness of different texture extraction methods could increase the accuracy of this approach as well.
Another area of future work is to include confocal microscopy images in the set of collected images to compare the extracted morphology from DIC images to the information provided by confocal images.
This work was supported in part by the Bernard M. Gordon Center for Subsurface Sensing and Imaging Systems (Gordon-CenSSIS), under the Engineering Research Centers Program of the National Science Foundation (Award No. EEC-9986821).