1.1.

Related Work

Previous texture detection methods in DIC microscopy images of biological samples have generally focused on the detection of cells or nuclei. Hamahashi ⁵ present a method for detection of nuclei in the process of cell division using a set of temporal 3-D DIC images of C. elegans embryos. The method combines local entropy and object tracking algorithms. The nuclei are successfully detected by the use of boundary and texture characteristics. Although these embryos are considered thick (thickness is about $23\mu \mathrm{m}$ ); cell nuclei can be easily detected by simply looking at the image. However, in the case of more complex thicker objects such as mouse embryos (thickness is about $100\text{to}120\mu \mathrm{m}$ ), nuclear appearance is not as detectable as in C. elegans embryos, and in many cases, it is undetectable by looking at the image.

Other contributions have focused on addressing both deconvolution and texture detection algorithms for cell detection. Kuijper and Heise’s method⁷ is able to detect cells properly as long as the cells are distinguishable by the human eye. As the authors pointed out, the method needs to be enhanced to address situations where the cells are not evidently visible. A solution to the cell detection problem that combines DIC and quantitative phase images from optical quadrature microscopy (OQM)^{11, 12, 13} has been used to develop a cell-counting method in live mouse embryos.¹⁴ (OQM noninvasively reconstructs the amplitude and phase of an optically transparent sample.) In that work, a 2-D DIC image was used to extract cell boundaries. Based on the detected boundaries, the method was capable of counting cell numbers ranging from 8 to 26.

Entropy is a characteristic of an image.¹⁵ The local entropy of a pixel in the image depends on information from its surrounding neighbors. Neighbor information can be extracted from a 3-D DIC image and used to generate a 3-D local entropy texture image that provides morphological information about the thick sample. Rapid changes of local entropy with focus in dense $z$ -stacks provide detailed information about the sample. This information is directly related to levels of texture roughness. Using the variation of texture roughness along the $z$ axis provides information about the morphological characteristics of the sample. Moreover, regions with different textures within the object can be detected based on the local entropy.

1.2.

Our Approach

The current work is focused on extracting information from a dense $z$ -stack of DIC images using a local entropy-based texture detection method. Local entropy texture methods have been used as an alternative to classical segmentation methods,⁵ in part because of their low sensitivity to out-of-focus effects. We use this sensitivity with variation of focus to follow changes in texture within the 3-D positions of the different sample regions, rather than trying to delineate or extract a specific region shape. The extracted information may contribute to understanding the natural processes occurring in such samples. For example, in embryo development studies, where the morphological details are valuable information,^{16, 17, 18} a wide range of questions can be answered by using an accurate morphology recognition mechanism.

In this work, we focus on embryo development studies, particularly the selection of the most viable embryos for assisted reproduction techniques such as in vitro fertilization (IVF). Recent studies in IVF have shown high pregnancy rates when the transfer of embryos to the mother’s womb occur at the blastocyst stage.^{19, 20} The blastocyst is one of the later stages of preimplantation embryo development, and its structure is so complex that it defies available image processing algorithms. In this paper, we process a set of 3-D DIC images of blastocyst mouse embryos. Blastocysts are composed of a spherical encasement known as the zona pellucida that surrounds and contains the inner structure (see Fig. 1 ). The inner structure of the blastocyst consists of three primary regions: inner cell, trophectoderm, and blastocoele cavity. Characteristics of the inner regions such as size, expansion, and distribution are used by clinicians to evaluate how well developed the embryo is at a particular time post-fertilization.^{4, 21, 22}

Fig. 1

DIC image of a live mouse embryo at blastocyst stage.

We extract DIC image textures using local entropy concepts to segment the inner cell, trophectoderm, and blastocoele cavity regions in the embryo. The obtained results are validated in two distinct ways, leveraging both fluorescence images and OQM phase images.^{11, 12, 13} In a direct approach, we compare our results to nuclei locations obtained through fluorescent staining. All nuclei should be clustered together in the inner cell and trophectoderm regions of the embryo. We also use a model-based validation approach computing 2-D quantitative phase images from the reconstructed embryo 3-D structure. We assign previously published indices of refraction^{23, 24} to the regions obtained from the texture analysis. Then, a forward model²⁵ is applied to estimate phase images. The computed phase images are compared to measured OQM phase images. The texture detection procedure and the two validations methods are applied to DIC images of five different blastocyst embryos.

The rest of the paper is structured as follows: A description of the local entropy texture detection method and its application to 3-D DIC images is presented in Sec. 2. Experiments to validate the obtained information from local entropy images are presented in Sec. 2 as well. Sections 3, 4 present the results and conclusions.

2.1.

Image Collection

The DIC images consist of 26 slices collected in $5\text{-}\mu \mathrm{m}$ increments along the optical axis $\left(z\right)$ from five blastocyst mouse embryos. The resulting $z$ -stack covers $125\mu \mathrm{m}$ along the $z$ axis, which is enough to cover the complete volume of the embryo. Each slice is $640\times 480\text{pixels}$ , with a pixel resolution of $0.3\mu \mathrm{m}$ . Figure 2 shows DIC images of five different blastocyst embryos, and Fig. 2 shows slices at three representative focal planes ( $z=-25\mu \mathrm{m}$ , $z=0\mu \mathrm{m}$ , $z=25\mu \mathrm{m}$ ) of a single embryo. DIC image stacks are processed with the local entropy method (see Sec. 2.2) to differentiate regions that contain cells (inner cell and trophectoderm) from those that do not contain cells (blastocoele cavity and spaces between cells).

Fig. 2

Collected DIC images of blastocyst mouse embryos: (a) five blastocyst mouse embryos (1–5 from left to right), and (b) DIC images at three different focal planes of a single blastocyst mouse embryo.

Next, OQM phase images were collected before staining. OQM is a noninvasive technique that provides amplitude and phase of an optically transparent sample.^{12, 14} These phase images were unwrapped with a 2-D $L^p$ -norm algorithm.²⁶ Measured OQM phase images were used to validate the detected embryo structure from DIC images.

Last, a stack of 26 fluorescence images was collected after applying to the five blastocyst samples a Hoechst nucleus stain. The fluorescent dye (Hoechst 33342) binds fluorophores to the nucleus of a cell, giving more contrast to the nuclei in the embryo image. In fluorescence images, bright regions provide information about the shape and location of the cell nuclei. Fluorescence images are normalized using a histogram thresholding method, such that intensity values greater than zero (brighter regions) indicate the presence of the cell nuclei. In practice, this modality is not feasible for live embryo applications such as IVF, because the Hoechst stain permanently modifies the DNA of the nuclei. Therefore, this stack is used for validation purposes only.

2.2.

Detection of Regions Using Local Entropy Image

Our region detection scheme is based on differentiating smoothness levels of image textures. The smoothness levels in a 3-D DIC image are quantified using local entropy.¹⁵ Local entropy is defined by the entropy of a pixel window surrounding a point of interest within the image.²⁷ For a rough texture, the local entropy values are high, while lower values correspond to smooth texture regions. Because a smooth texture in blastocyst DIC images is characteristic of the blastocoele, we expect its local entropy to be lower than that in cellular regions, in which a rough texture is indicative of the organelles in the cytoplasm. Our analysis is focused on the distribution of the cells and blastocoele regions in the embryo. Thus, we have removed the zona pellucida from the images by applying an edge detector²⁷ before computing the local entropy images.

We define texture using local entropy in the image as follows: Let $X$ represent the input image and $W$ an $m\times n\times p$ pixel window. The resulting local entropy image $Y$ is given by

A stack of DIC images is segmented by using a window of $7\times 7\times 3\text{pixel}$ size. In other words, for a 2-D image located at $z_k$ , the window covers from $z_{k-\Delta z}$ to $z_{k+\Delta z}$ images along the $z$ axis. The top and bottom images of the stack were processed with a $7\times 7\text{pixel}$ size window. As a result of this process, a 3-D local entropy image is obtained.

2.3.

Validation

Two validation approaches were used to test the predicted blastocyst embryo structure. In the first approach, we compared the local entropy image with a fluorescence image of Hoechst-stained nuclei. The images were overlapped to identify common regions. In the second validation method, we used the local entropy images to create an embryo model and compute a quantitative 2-D phase image. Then, the 2-D phase image was compared to a measured OQM phase image. Results of these validation methods provide information about the accuracy of the local entropy texture method to predict embryo structure from 3-D DIC images.

2.3.1.

Local entropy and fluorescence images

A histogram of the local entropy images was calculated to analyze the distribution of the local entropy values. Figure 3 shows a histogram of the 3-D local entropy images for all five embryos. We observe that the local entropy values follow a multimodal distribution. We apply thresholds to the histogram to divide the data in three different groups. These groups are labeled regions with high, medium, and low local entropy values. High entropy contains values greater than or equal to 1.2, medium values are between 0.8 and 1.1, and low values are those less than or equal to 0.8. Figure 4 shows an overlay of the local entropy (after thresholding) and DIC images.

Fig. 3

Histogram of the 3-D local entropy image of all five embryos. We have selected the threshold based on the distribution shapes. These thresholds (th1 and th2) divide the local entropy values into three groups. These groups are used to define high, medium (cells regions), and low (blastocoele region) local entropy regions for validation purposes.

Fig. 4

Visualization of detected regions using local entropy method in DIC images. Figure shows an overlay of the local entropy (after thresholding) and DIC images. Each slice was collected with an interval of $5\text{microns}$ along the $z$ axis. In the figure, the $z$ position increases row-wise from top-left to bottom-right. The figure at the top-left corresponds to the slice taken at $z=-60\mu \mathrm{m}$ , and the figure at the bottom-left corresponds to the slice at $z=60\mu \mathrm{m}$ .

Since each cell contains organelles with different optical properties, texture variation occurs in each cell. Then, medium and high local entropy values are assumed to be located in the inner cell region, while low values are in the blastocoele cavity. Nevertheless, since regions with high and medium local entropy values represent those occupied by the volume of the cells with their subcellular components, they are expected to be larger than the brighter regions identified as nuclei of the cells in the fluorescence images. Figure 5 shows an overlay of the local entropy (high and medium local entropy regions, after thresholding) and fluorescence images.

Fig. 5

Validation of detected regions using local entropy method in DIC images using fluorescence images. Figure shows an overlay of the local entropy (high and medium local entropy regions, after thresholding, in red) and fluorescence images (green). Brighter regions (yellow) are the common regions of the overlapped images. (Color online only.)

To validate the accuracy of the segmented regions, we calculate the percentage of the fluorescent areas that are outside the high and medium local entropy regions. We expect that since the high and medium local entropy areas contain the fluorescent areas, a highly accurate method should yield a low percentage value.

2.3.2.

2-D quantitative phase image estimation

The obtained 3-D local entropy image is used in conjunction with our image formation forward model to estimate phase images.²⁵ The calculated image is compared to a measured OQM phase image to validate the detected regions with the local entropy method. The procedure is described in Fig. 6 .

Fig. 6

Block diagram of embryo model and phase reconstruction from DIC images.

The embryo model is constructed using the regions with high, medium, and low entropy (obtained from the local entropy image histogram, Fig. 3). These regions are each assigned an index of refraction as obtained from published cell modeling work.^{23, 24} The cytoplasm of the cell has a reported index of refraction of 1.35. The average of the indices of refraction of the nucleus and the cellular organelles is equal to 1.37. Since the presence of the nucleus and different organelles in each cell result in high entropy values, we have assigned 1.37 to the high entropy region and 1.35 to the medium entropy region. The low local entropy region is composed of the blastocoele cavity. Prior embryo analysis has shown us that the blastocoele has an index of refraction similar to the immersion medium, which is 1.33; thus, we use this value for the blastocoele region. Last, we add the zona pellucida to the model, since it was removed from the DIC image to calculate the local entropy image. The zona pellucida is uniform and thin, and its index of refraction can be estimated from a measured OQM phase image. The index of refraction for this region has been estimated to be equal to 1.34.

We generate a phase image of the synthetic embryo described earlier using a forward model²⁵ for thick objects. The calculated phase image is compared to a measured OQM phase image. To evaluate the similarities between these images, we use the Dice metric.²⁸ Let $T_k$ denote a measured image set of pixels in a region $k$ , and then the Dice metric for region $k$ is defined as $2(|T_k\cap {\tilde{T}}_k|∕\left|T_k\right|+\left|{\tilde{T}}_k\right|)$ , where |.| denotes set size, and ${\tilde{T}}_k$ is the set of pixels of the calculated image. For the validation, we choose a region that contains phase values greater than $15\mathrm{rad}$ . Since phase values greater than $15\mathrm{rad}$ do not belong to the blastocoele cavity region, the comparison is performed for a region that contains cells in both the simulated and measured phase images.

3.1.

Using Fluorescence Images

Validation results using fluorescence images for embryo 1 are presented in Fig. 5. This figure shows an overlay of the fluorescence and the generated local entropy images. We can observe that the high entropy regions almost completely contain the cell nuclei area. The percentage of the region identified as nuclei in the fluorescence images that is outside the area with high local entropy is equal to 3%. This error means that the segmentation (local entropy-based) method failed when de-tecting those regions as low entropy regions. Then, we consider this percentage as an error of the method. Table 1 shows the percentage results from all five embryos.

Table 1

Validation of local entropy segmentation of DIC versus fluorescence images. %Error is the fraction of fluorescence area outside the inncer cell region segmented from the DIC image.

The reported error values are less than 5% for four of the samples (embryos 1, 2, 3, and 5) and 12% for just one case (embryo 4). If we observe embryo 4 in particular [see Fig. 2], this sample is reaching the blastocyst stage but still has a very small and not well delineated blastocoele cavity with most of the embryo’s area being composed of the inner cell region. Thus, this case serves to show a near to worst case performance of our approach. Even though posing a more difficult case to extract the texture of the regions, our method is able to report only a 12% of error for this case. We can observe that most of the fluorescent areas (from fluorescent images) are contained in the high local entropy region. These results give us confidence that the method is highly accurate when extracting the region textures of a blastocyst mouse embryo.

3.2.

Using an OQM Image

Estimated and measured phase images are compared quantitatively. The comparison and validation is performed using the Dice metric. The Dice metric is applied to a selected region in the image. Although the same criterion and threshold $\left(15\mathrm{rad}\right)$ is used for each image, the size and location of the region varies for different embryo images. Computed and measured phase images for all five embryos and their corresponding selected region are presented in Fig. 7 . We can observe from the images that the estimated phase closely resembles the measured phase. Results of the Dice metric values for the five embryos are shown in Table 2 . Again, embryo 4 being the most difficult case showed a Dice value of 0.71, while the rest provide values of 0.80 or more. These results show that the 3-D object structure generated with the information extracted from DIC images closely resembles the true embryo structure.

Fig. 7

Visual comparison of estimated phase image from processed DIC images to a measured phase image of five mouse embryos and selected regions with phase values larger than $15\mathrm{rad}$ for quantitative comparison using a Dice metric. From left to right: measured OQM image, region from measured image, estimated phase image, and region from estimated image. (a) embryo 1 with $\mathit{\text{Dice}}=0.8$ ; (b) embryo 2 with $\mathit{\text{Dice}}=0.93$ ; (c) embryo 3 with $\mathit{\text{Dice}}=0.92$ ; (d) embryo 4 with $\mathit{\text{Dice}}=0.71$ ; and (e) embryo 5 with $\mathit{\text{Dice}}=0.83$ . Bright values inside the ellipse represent the selected region.

Table 2

Results of Dice metric. The Dice metric is used to compare calculated and measured phase images in a selected region. The region corresponds to phase values greater than 15rad .