2.1.

Evaluation Criteria of Achieved EDFM Performance

The achieved performance of EDFM in the presence of SA is assessed using the following evaluation criteria: (a) achieved resolution, (b) localization accuracy of imaged structures, (c) residual image artifacts, (d) SNR drop, and (e) DOF extension. These criteria along with specific simulation examples enable evaluation of system performance. Specifically, achieved resolution is quantified using the Rayleigh resolution and a numerical test object consisting of pairs of closely spaced lines. EDFM performance is assessed experimentally using the last three criteria, which do not require knowledge of the true object. For example, in simulation, the achieved DOF is determined by comparing the final EDFM image to the projection of the true 3-D object, while in experiment, the projection of the 3-D deconvolved wide-field image is used instead. In the following, we provide more details on how these criteria are computed and/or assessed in each case.

2.2.

Image Restoration

Two image restoration algorithms known for their properties to ameliorate the impact of noise were used in this study: the penalized space-invariant expectation maximization (SIEM) algorithm²² based on a roughness penalty³⁰ and the regularized linear least squares (RLLS) method.²³ These methods are implemented in the COSMOS image estimation module³¹ and can be used for 2-D and 3-D image restoration. The RLLS is a noniterative method with processing time in the order of 1 iteration of the SIEM algorithm. The algorithms were used to process 2-D EDFM images from both the CPM-WFE and the GCPM-WFE systems using a simulated 2-D WFE-PSF selected from the 3-D PSF (computed as described in Sec. 2.3). For the iterative SIEM algorithm, results were examined at several iteration numbers. Results for simulation studies were obtained at 1000 iterations of the algorithm. Results for experimental studies were obtained at 5000 iterations for the test-sample image and live-lung-cell image, and 1000 iterations were completed for restoration of the brine shrimp image. For comparison purposes, synthetic and experimental images from the conventional CCA system were also processed with the SIEM algorithm and the corresponding CCA-PSF layer.

Noiseless simulated images were restored without regularization, whereas experimental data and noisy simulated data (computed as described in Secs. 2.5 and 2.6) were processed using a nonzero regularization parameter that controls the amount of regularization used in the RLLS method and the penalized SIEM method in order to ameliorate the effects of noise on the solution of the ill-posed inverse imaging problem. The effects of different values of the respective regularization parameters were investigated in simulation, and results are reported in Sec. 3.4. The range of regularization parameters used for restoration of experimental data (reported in Sec. 3.5) overlaps with the range of values studied in simulation.

The different simulated WFE-PSFs used for restoration in this study provide an approximation to the true PSF that characterizes the system in the presence of SA. Since PSF mismatch causes image artifacts in model-based restoration, direct assessment of these artifacts provides an evaluation of the EDFM performance.

2.3.

Wavefront Encoded PSF

WFE-PSFs based on two different phase masks and selected design parameters were used in this study to compute synthetic intermediate images and for image restoration. The mathematical functions that describe these phase masks as well as the model used to compute the 3-D WFE-PSFs are described in previous publications.^11,32,15 The CPM ($\alpha =-30$) pattern and the GCPM ($\alpha =-150$ and $\beta =-3\alpha$) selected for EDFM as well as the GCPM ($\alpha =-30$ and $\beta =-3\alpha$) are shown in Fig. 1. Examples of aberrated simulated WFE-PSFs computed with the above masks to validate experimental results are also shown in Fig. 1.

Fig. 1

Phase mask pattern designs for EDFM and resulting experimental and computed WFE-PSFs, with SA due to RI mismatch, $\mathrm{\Delta }n=0.05$: (a) traditional CPM design, (b) GCPM design ($\alpha =-150$) robust to SA, and (c) GCPM design robust to SA with a lower value of $\alpha =-30$. Phase mask designs are for 532 nm and 20-deg incident illumination. Experimental images acquired using a $63\times /1.4$ NA oil-immersion lens at 515 nm from 175-nm diameter fluorescent beads dried on the coverslip and embedded in glycerol. Slice views are shown through the maximum intensity point in the 3-D volume. All images are displayed on an individually normalized scale with the minimum intensity equal to zero and the maximum intensity equal to one and a linear color map; with the exception of (b) for which experimental PSFs are normalized and displayed with an 80% compressed linear color map (maximum displayed intensity equals 0.2) to show effect of the wavefront encoding.

For the comparison of computed and experimentally determined PSFs (Fig. 1), WFE-PSFs were computed on a $512\times 512\times 300$ grid with voxels of size $0.1\times 0.1\times 0.1\mu {\mathrm{m}}^3$. However, to match experimental conditions as closely as possible, they were computed assuming: (a) a point source located at 240 nm below the coverslip in glycerol (RI, $n_{\text{water}}=1.47$), (b) a $63\times /1.3$ NA oil-immersion objective lens (RI, $n_{\text{oil}}=1.515$), and (c) an emission wavelength ${\lambda }_{\text{emission}}=515\mathrm{nm}$. The phase distortion corresponding to a depth of $24\mu \mathrm{m}$ was added to the generalized pupil of all computed PSFs in order to account for a residual aberration in our system.³³

Examples of unaberrated computed PSFs used to produce synthetic intermediate images are shown in Fig. 2(b). To model the depth variability of the EDFM in the presence of SA in a simulated imaging system, four depth variant (DV) CCA-PSFs were computed, as described in a previous publication¹¹ assuming: (a) a point source located at a specific depth ($z_o=0$, 5, 10, or $15\mu \mathrm{m}$) in water (RI, $n_{\text{water}}=1.33$) below the coverslip, (b) a $60\times /1.2$ NA oil-immersion objective lens (RI, $n_{\text{oil}}=1.515$), and (c) an emission wavelength ${\lambda }_{\text{emission}}=633\mathrm{nm}$.

Fig. 2

Simulated intermediate GCPM- and CPM-EDFM images with SA of (a) object 1, compared with CCA images. (b) Nonaberrated CPM ($\alpha =40$) and GCPM ($\alpha =150$ and $\beta =-3\alpha$) PSFs. (c) $XY$-cut views from the 3-D simulated images are shown at a distance $d$ ($=z_i$) away from the central focal plane equal to: $-5\mu \mathrm{m}$ (left), $0\mu \mathrm{m}$ (middle), and $5\mu \mathrm{m}$ (right). The projection along the $Z$-axis of object 1 (described in Sec. 2.4) is shown in (a). Lens: $60\times /1.2$ NA oil-immersion. Wavelength: 633 nm. All images are displayed on a normalized scale with the minimum intensity equal to zero and the maximum intensity equal to one using a linear color map.

For reconstruction of experimentally acquired data, PSFs were computed with a model that takes into account the effects of the LC-SLM implementation of the phase mask by modifying the phase function that represents the phase mask.^33–34 Simulated PSFs for reconstruction of experimental data were computed on a $512\times 512\times 500$ grid with voxel size $0.1\times 0.1\times 0.1\mu {\mathrm{m}}^3$ assuming: (a) point source depth of 240 nm in embedding media with different RI (e.g., $n_{\text{cement}}=1.56$, $n_{\text{water}}=1.33$, and $n_{\text{cytoseal}}=1.495$), (b) a $63\times /1.3$ NA oil-immersion objective lens (RI, $n_{\text{oil}}=1.515$), and (c) an emission wavelength ${\lambda }_{\text{emission}}=515\mathrm{nm}$.

2.4.

Test Objects

The 3-D numerical test objects used in this study (shown in Figs. 2, 5, and 6) were chosen in order to investigate different properties of the WFE system: sensitivity to SA and noise, and the trade-off between SNR and resolution controlled by regularization. To model SA, an RI mismatch was used between the mounting medium for all of the test objects assumed to be 1.33 and the immersion medium of the oil lens (i.e., $\mathrm{RI}=1.515$). Object 1 (Fig. 2) consists of three small spheres, all $1\mu \mathrm{m}$ in diameter. The spheres are centered at depths 10, 15, and $20\mu \mathrm{m}$ on a $256\times 256\times 300$ grid with voxels of size $0.1\times 0.1\times 0.1\mu {\mathrm{m}}^3$. The origin of the $X_o{Y}_o$ plane is at the center of the grid, and the origin along the $z_o$-axis was placed at the 75th $X_o{Y}_o$ plane. The $(x_o,y_o)$ coordinates of the three spheres are: $(\mathrm{5,5}\mu \mathrm{m})$, $(\mathrm{0,0}\mu \mathrm{m})$, and $(-5,-5\mu \mathrm{m})$. Object 2 (Fig. 5) consists of three blocks with different shapes: a $4\times 4\times 4\mu {\mathrm{m}}^3$ cube centered at a depth of $2.5\mu \mathrm{m}$, a sphere ($4\mu \mathrm{m}$ in diameter) centered at a depth of $7.5\mu \mathrm{m}$, and a $4\times 4\times 6\mu {\mathrm{m}}^3$ rectangle centered at a depth of $12.5\mu \mathrm{m}$ below the coverslip. The grid size and $(x_o,y_o)$ coordinates of object 2 are the same as object 1. Object 3 (Fig. 6) contains six pairs of rectangle bars. Each bar is $10\mu \mathrm{m}$ along $y$ and $5\mu \mathrm{m}$ along $z$. The bar width along $x$ for each pair and the interval between two pairs are increased from 0.2 to $1.2\mu \mathrm{m}$ in increments of $0.2\mu \mathrm{m}$ (from left to right).

For analysis and validation of experimental EDFM, synthetic intermediate EDFM images (Fig. 9) of two additional objects were computed. Objects 4 and 5 (not shown) both consist of a pair of $6\mu \mathrm{m}$ in diameter spherical shells with a $1\text{-}\mu \mathrm{m}$ shell thickness embedded in a medium with $\mathrm{RI}=1.56$. The objects were computed on a $512\times 512\times 500$ grid with voxel size $0.1\times 0.1\times 0.1\mu {\mathrm{m}}^3$. Objects 4 and 5 were created to match experimental test-samples (described in Sec. 2.7) imaged with the WFE system. Object 4 was created for comparison with experimental CPM-EDFM results and has spherical shells with a displacement of $(\mathrm{\Delta }x,\mathrm{\Delta }y,\mathrm{\Delta }z)=(4.9,2.3,2.3\mu \mathrm{m})$ between them. The shells in object 5 (created for comparison with experimental GCPM-EDFM results) have a displacement of $(\mathrm{\Delta }x,\mathrm{\Delta }y,\mathrm{\Delta }z)=(4.1,2.3,2\mu \mathrm{m})$ between them.

2.5.

Intermediate WFE Image in the Presence of SA

Simulated intermediate images of these synthetic objects were computed using a strata-based model³⁵ that approximates wide-field microscopy in the presence of SA and a finite number of 3-D DV WFE-PSFs described in Sec. 2.3.

The synthetic intermediate images of objects 1 and 2 were computed using three strata or four DV WFE-PSFs due to point sources located at depths $z_o=0$, 5, 10, and $15\mu \mathrm{m}$, correspondingly, below the coverslip. Simulated intermediate images of objects 4 and 5 were computed using 11 and 15 strata, respectively (i.e., using 12 CPM-PSFs computed at depths $20\mu \mathrm{m}$ through $31\mu \mathrm{m}$ at $1\text{-}\mu \mathrm{m}$ intervals and 16 GCPM-PSFs at depths $0\mu \mathrm{m}$ through $45\mu \mathrm{m}$ at $3\text{-}\mu \mathrm{m}$ intervals, respectively).

2.6.

Noise and SNR Computation

To analyze EDFM performance under noisy conditions, noiseless simulated intermediate EDFM images were corrupted with Poisson and Gaussian noises. As is well known, Poisson noise is consistent with fluorescence imaging because it is signal dependent, i.e., its mean and variance depend on the number of photons captured by the camera based on the intensity of the fluorescence concentration in the sample. Using an appropriate scale factor, different amounts of Poisson noise were introduced in the simulated images using MATLAB’s™ (MathWorks, Massachusetts) imnoise built-in function. For images with Poisson noise, the peak SNR was calculated for a region in the image using

Gaussian noise was used to model electro-optical components in the microscope. For the Gaussian case, noise computed with imnoise for a specific mean and standard deviation was added to the image. The SNR in the region of interest (ROI) of the noisy image was computed as

2.7.

Data Acquisition

The WFE imaging path was implemented by incorporating an LC-SLM (Meadowlark Optics, Frederick, Colorado) at a side imaging port of a commercial microscope via a unit magnification, four-$f$ imaging system.²⁷ Experimental PSFs (for the three phase masks described in Sec. 2.3) and intermediate EDFM images (based on the GCPM $\alpha =-30$, $\beta =-3\alpha$ design, and the CPM $\alpha =-30$ design) of one test sample and two biological samples were acquired with a Zeiss Axio Imager Z1 microscope, a Zeiss AxioCam, a $63\times /1.4$ NA oil-immersion objective ($\mathrm{RI}=1.515$), and a 515- to 565-nm emission filter (Zeiss filter set 10). At this magnification, the lateral camera pixel size is equal to $0.1\times 0.1\mu {\mathrm{m}}^2$ in the object space.

Images from the experimental test sample are shown in Figs. 7 and 8. The images show pairs of spheres that are offset and stacked in depth such that the object extends $\sim 11\mu \mathrm{m}$ in the axial direction. The top of the first sphere is estimated to be approximately at the coverslip. The test sample consists of spheres ($6\mu \mathrm{m}$ in diameter dual labeled with fluorescent green and blue dyes, Invitrogen Life Sciences) embedded in UV cured optical cement (Norland Products, Inc., Optical Adhesive 63), which has a $\mathrm{RI}=1.56$. The green fluorescent dye does not permeate throughout the sphere. Instead, it labels a spherical shell with an inner diameter of $\sim 3.5\mu \mathrm{m}$. Within this fluorescent shell, green fluorescent emission is due only to a weak spectral overlap of the blue fluorescent dye emission and the emission filter.

Images from live and fixed biological samples are shown in Figs. 10 and 11, respectively. Figure 10 shows live human bronchial epithelial cells (HBE135-E6E7, epithelial HPV-16 E6/E7) that were cultured and imaged on a polydimethylsiloxane membrane coated with collagen I in 30-mm dishes. This type of membrane allows stretching of the confluent layer of cells and is used as a model for motion-induced stress in epithelial lung tissue.³⁶ Mitochondria were labeled with MitoTracker^® Green. Immediately before imaging, the membrane was transferred to a slide and capped with a coverglass (No. 1½) to accommodate an upright microscope configuration. The sample is weakly fluorescent such that illumination with a 1-s exposure time results in a maximum 12-bit gray level equal to 1888 and 1818 in the EDFM image based on a CPM and a GCPM, respectively. Before processing, the $\mathrm{SNR}\approx 13\mathrm{dB}$ in the intermediate GCPM image of the lung-cell layer while in the intermediate CPM image $\mathrm{SNR}\approx 14\mathrm{dB}$. Figure 11 shows a preparation of fixed brine shrimp of different sizes located at varying depths from the cover slip (Zeiss Microscopy), stained with eosin, and mounted in Cytoseal™ ($\mathrm{R}\mathrm{I}=1.495$; Thermo Fisher Scientific).

3.1.

Experimental EDFM WFE-PSF

Observation of experimental and computed PSFs is used to validate their experimental implementation. Figure 1 summarizes a qualitative comparison of experimental and computed WFE-PSFs based on theory. The CPM-PSF ($\alpha =-30$) verifies that in addition to its inherent axial curvature, the lateral shape of this PSF is degraded and varies along the axial direction due to SA [Fig. 1(a)]. As evident in Fig. 1(b), very weak qualitative agreement between experimental and simulated PSFs is achieved for the GCPM design with $\alpha =-150$ previously selected to account for SA. A physical limitation prevents the LC-SLM implementation of phase masks with designs for a high absolute value of the $\alpha$ parameter that are predicted to be more robust to SA.²⁷ Masks with many phase wrappings between 0 and $2\pi$ are not implemented precisely on LC-SLM due to the pixel size of the LC-SLM. However, implementation of the GCPM with $\alpha =-30$ shows agreement with simulation and a significant axial extent of the PSF compared with the CCA that is suitable for evaluation of EDFM performance in the presence of SA. Thus, the experimental studies in Sec. 3.5 utilize the GCPM $\alpha =-30$, $\beta =90$ design to evaluate its performance in EDFM when SA is present.

3.2.

Intermediate Synthetic Images with SA from EDFM Systems

Simulated EDFM intermediate images in the presence of SA computed using different phase masks are shown in Fig. 2. The extension of the WFE-PSFs along the $Z$-axis [Fig. 2(b)] causes intensity contributions from beads located in other planes, as seen in the $XY$-cut views of the intermediate image from the CPM- and GCPM-based systems [Fig. 2(c)]. This is in contrast to the CCA images, in which the intensity from the bead located at each plane dominates intensity contributions from beads located in other planes. The presence of SA is evident by the pronounced difference in the appearance of the top left and bottom right bead in the CPM intermediate images. It is noted that this difference is much diminished in the GCPM intermediate images.

3.3.

Restored Images from Noiseless Simulation of EDFM Systems in the Presence of SA

Figures 3 and 4 summarize results from SIEM algorithm restoration using a 2-D single unaberrated WFE-PSF that yields the final EDFM images of object 1 from a WFE system with a CPM ($\alpha =40$) and a GCPM ($\alpha =150$ and $\beta =-3\alpha$) from $XY$ intermediate images computed with SA (Fig. 2) and without SA. The case without SA was simulated as a control because it does not suffer from PSF mismatch due to SA, and thus it can be used to aid understanding of artifacts due to the structural differences of the PSF and not due to SA. A qualitative comparison of the restored images obtained at different focal planes (Fig. 3) to the projection of the object along the $Z$-axis [Fig. 2(a)] enables assessment of the achieved DOF in the CPM [Fig. 3(a)] and the GCPM [Fig. 3(b)] WFE systems. The theoretical DOF of the simulated CCA case [Fig. 3(c)] is $\sim 0.7\mu \mathrm{m}$,³⁷ and the observed DOF in the CPM and the GCPM cases is at least $5\mu \mathrm{m}$ based on these restored images.

Fig. 3

Restored GCPM- and CPM-EDFM images at three focal planes from object 1 computed from data without SA (left) and with SA (right) from a system with a: (a) CPM ($\alpha =40$) and (b) GCPM ($\alpha =150$ and $\beta =-3\alpha$). (c) CCA system images with SA for comparison. For the SA results, the intermediate EDFM images presented in Fig. 2 were used. The relative depth of each focal plane ($d=z_i$) is measured from the central focal plane and is equal to: $-5\mu \mathrm{m}$ (top), $0\mu \mathrm{m}$ (middle), and $5\mu \mathrm{m}$ (bottom). Each image is displayed by mapping its minimum and maximum intensity to a linear colorscale map.

Fig. 4

Quantitative comparison of object 1 restorations from images acquired from different systems. True object and reconstructed EDFM image intensity profiles show achieved accuracy in restored location and shape of structures. Profiles shown are from the $Z$-projection of the true object [Fig. 2(a)] and restored EDFM images along a diagonal line from top left to bottom right through the three beads. Profiles through the restored 2-D images in the presence of SA (Fig. 3) are compared at a distance $z_i$ away from the central focal plane equal to: (a) $-5$, (b) 0, and (c) $5\mu \mathrm{m}$.

The dotted horizontal lines in Fig. 3 mark the true locations of the beads in the underlying object and, thereby, show the lateral shift in the position of the beads in the restored CPM images. This lateral shift occurs due to the curvature of the 3-D CPM-PSF that is not accounted for by restoration with a single 2-D PSF. The effect of this model mismatch has been noted and discussed previously.^3,4,11 The PSF curvature effect can be seen by the distorted shape of the beads in the leftmost column of Fig. 3(a), where SA is not included. The effect of SA is evident by comparing the difference in the appearance of the top left and bottom right bead in the final CPM images with and without SA [Fig. 3(a)], in contrast to the GCPM images [Fig. 3(b)], which do not change with the presence of SA.

To quantitatively evaluate the restored results, intensity profiles from the images in the presence of SA (Fig. 3) are summarized in Fig. 4. Since CPM with $\alpha =90$ has also been applied in other EDFM-related studies,^12,38 the result from this case is also included in Fig. 4 for comparison. The observed differences in peak values and location of peaks in the intensity profiles quantify the residual restoration artifacts in the shape and position of the beads due to PSF mismatch. It is noted that these artifacts are reduced in the final GCPM image result [Fig. 3(b)], which was produced with the GCPM design selected to reduce the impact of SA on the engineered PSF. The maximum position error in bead location in the CPM image is $\sim 2\mu \mathrm{m}$ for $\alpha =40$ and $\sim 1\mu \mathrm{m}$ for $\alpha =90$. The scale of this inaccuracy is an order of magnitude higher than the $0.4\text{-}\mu \mathrm{m}$ lateral resolution limit of the optical imaging system. In contrast, the bead position error in the GCPM image is $<0.5\mu \mathrm{m}$. This comparison highlights that localization inaccuracy is a compounded effect of SA and PSF mismatch in EDFM using the CPM.

3.4.

Restored Images from Noisy Simulations of an EDFM System in the Presence of SA

The performance of GCPM-based EDFM with noisy images was evaluated, and the results are summarized in Fig. 5 and Table 1. Object 2 and the noise model used in this study are described in Secs. 2.4 and 2.6, respectively. The EDFM intermediate images at two different noise levels are shown in Figs. 5(a) and 5(b), respectively. The corresponding restored images using the regularized SIEM algorithm are shown in Figs. 5(c) and 5(d) while the images obtained with the RLLS method are shown in Figs. 5(e) and 5(f), respectively.

Fig. 5

Reconstructed GCPM-EDFM results from two algorithms at different noise levels. The $Z$-projection view of the 3-D object is shown at the leftmost panel. Simulated images with an SNR of 17 and 12 dB are shown in (a) and (b), respectively, while restorations from these two cases are shown in (c and e) and (d and f), correspondingly. The SIEM restored results shown are obtained with a regularization parameter equal to: $5\times {10}^{-4}$ in (c) and $1.5\times {10}^{-3}$ in (d). The RLLS results are shown in (e) and (f) with a regularization parameter equal to $7\times {10}^{-5}$ and $2\times {10}^{-4}$, respectively. Each image is displayed by mapping its minimum and maximum intensity to a linear colorscale map.

Table 1

SNR in noisy simulated images shown in Fig. 5 and restoration algorithm parameters.

The trade-off between noise and resolution due to the regularization process is analyzed explicitly with the restoration of object 3 (described in Sec. 2.4) summarized in Fig. 6 and Table 2. Figure 6(a) shows the central $XY$-section from the 3-D object. The intermediate image ($\mathrm{SNR}=17\mathrm{dB}$) from the EDFM system with the selected GCPM design ($\alpha =150$, $\beta =-3\alpha$) is shown in Fig. 6(b). SIEM restored images are shown in Figs. 6(c)–6(e). Figures 6(f)–6(h) show restored images using the RLLS method. Algorithms and regularization parameters used to compute Figs. 6(e) and 6(h) are identical to those used to compute Figs. 5(c) and 5(e), respectively, and thus the resolution (Table 2) and SNR (Table 1) in these pairs of images can be directly compared. This comparison shows that the achieved resolution and SNR are better with the SIEM algorithm over the RLLS method.

Fig. 6

Resolution study of a noisy GCPM-EDFM system with Rayleigh resolution limit equal to $0.32\mu \mathrm{m}$. (a) The central $XY$-section of a 3-D test object with six resolution bar pairs. From left to right, the interval between the bar pairs is 0.2, 0.4, 0.6, 0.8, 1.0, and $1.2\mu \mathrm{m}$. (b) The center $XY$-section of the noisy intermediate GCPM-EDFM image, which includes both Gaussian and Poisson noise with a total SNR of 17 dB. (c–e) EDFM restored images obtained with the SIEM algorithm with a roughness penalty and a regularization parameter equal to $5\times {10}^{-5}$, $1.5\times {10}^{-4}$, and $5\times {10}^{-4}$, respectively. (f–h) EDFM restored images obtained with the RLLS algorithm with regularization parameter equal to $7\times {10}^{-6}$, $2\times {10}^{-5}$, and $7\times {10}^{-5}$, respectively.

Table 2

Achieved resolution in noisy simulated images shown in Fig. 6 and restoration algorithm parameters.

3.5.

Restored Images of Experimental Data Acquired with an EDFM System

The performance of GCPM-based EDFM with experimental data was evaluated using a test sample (Figs. 7Fig. 8–9) with well-defined size and shape as well as with biological structures from two different types of samples (Figs. 10 and 11). The test sample and biological samples are described in Sec. 2.7. The $6\text{-}\mu \mathrm{m}$ bead-pair test sample allows comparison of experiment with simulation in order to validate the results of restored experimental EDFM images. Images of clustered mitochondria, in live epithelial lung cells, demonstrate EDFM of low emission from subcellular structure in a sample that is $\sim 2\text{-}\mu \mathrm{m}$ thick and is imaged adjacent to the coverslip. Images of fine protrusions from the body of a fixed brine shrimp demonstrate EDFM of high emission from a sample that is $\sim 6\text{-}\mu \mathrm{m}$ thick and acquired at successive depths away from the coverslip. The former demonstrates EDFM of a densely labeled subject, and the latter demonstrates EDFM of thin structures that arc through the 3-D volume. As is the case in the human lung epithelial cells and brine shrimp images, the bead pair is larger than the DOF such that some emitted light remains out of focus despite the EDF.

Fig. 7

WFE-EDFM produced by CPM and GCPM compared with axial intensity projections of deconvolved CCA images. (a) CPM-EDFM images restored with SIEM and RLLS algorithms. (b) Intensity projection over a $6\text{-}\mu \mathrm{m}$ range from the sum of 60 $XY$-section images at $0.1\text{-}\mu \mathrm{m}$ intervals from the 3-D restoration (COSM) of a CCA image of the same bead pair. (c) Intensity projection over a $29.8\text{-}\mu \mathrm{m}$ range from the sum of 298 $XY$-section images at $0.1\text{-}\mu \mathrm{m}$ intervals from the 3-D restored CCA image. (d) GCPM-EDFM images of the second bead pair. (e) and (f) Intensity projection over 3 and $50\mu \mathrm{m}$, respectively, from restored 3-D CCA image of corresponding bead pair. SIEM restoration was regularized with a $1\times {10}^{-3}$ roughness penalty weight and is shown after 5000 iterations. RLLS restoration regularization parameter was equal to $1\times {10}^{-3}$ for CPM and GCPM, and $1\times {10}^{-4}$ for CCA. Acquisition parameters: $63\times /1.4$ NA oil-immersion objective lens and 515-nm emission. Each image is displayed by mapping its minimum and maximum intensity to a linear grayscale map.

Fig. 8

DOF in different imaging systems: WFE-EDFM (2-D SIEM reconstruction), unprocessed CCA (conventional) microscopy, and CCA COSM (3-D SIEM restoration). $XY$ views of (a) experimental CPM-EDFM images compared with CCA and COSM images at $1\text{-}\mu \mathrm{m}$ intervals over a $6\text{-}\mu \mathrm{m}$ depth and (b) GCPM-EDFM images compared with CCA and COSM images at $0.5\mu \mathrm{m}$ intervals over a $3\text{-}\mu \mathrm{m}$ depth. CPM- and GCPM-EDFM images at reference depth $d_o$ are repeated from Figs. 7(a) and 7(d). Slices shown in the CCA COSM are taken from the 3-D restored CCA image [corresponding intensity projection shown in Figs. 7(b) and 7(e)]. Each row shows a concatenated image displayed by mapping its minimum and maximum intensity to a linear grayscale map.

Fig. 9

Simulated EDFM images of a synthetic $6\text{-}\mu \mathrm{m}$ bead pair validate the extension in the observed DOF achieved in experimental EDFM images (Fig. 8). $XY$ views of (a) simulated CPM-based EDFM images at $1\text{-}\mu \mathrm{m}$ intervals over a $6\text{-}\mu \mathrm{m}$ depth and (b) GCPM-based EDFM images at $0.5\text{-}\mu \mathrm{m}$ intervals over a $3\text{-}\mu \mathrm{m}$ depth. Images were restored using SIEM from simulated intermediated WFE images of the synthetic $6\text{-}\mu \mathrm{m}$ bead after 5000 iterations. GCPM simulated intermediate image has $\mathrm{SNR}=11.69\mathrm{dB}$ due to additive Gaussian noise. Lens: $63\times /1.4$ NA oil-immersion. Emission wavelength: 515 nm. Each row shows a concatenated image displayed by mapping its minimum and maximum intensity to a linear grayscale map.

Fig. 10

EDFM images of live mitochondria labeled with MitoTracker^® Green in human bronchial epithelial cell culture (16HBE) demonstrate improved trade-off between DOF and resolution as compared with CCA. (a) GCPM-EDFM and (b) CPM-EDFM images restored with the SIEM algorithm after 5000 iterations (top row) and the RLLS method (middle row). In SIEM restoration, the roughness penalty weight used was $1\times {10}^{-2}$ and $1\times {10}^{-3}$ for the GCPM and CPM cases, respectively. In RLLS restoration, the regularization parameter used was: $1\times {10}^{-3}$ for GCPM and $1\times {10}^{-4}$ for CPM. (c) Intensity projection over a $2\text{-}\mu \mathrm{m}$-depth range from the sum of 20 2-D images at $0.1\text{-}\mu \mathrm{m}$ intervals after restoration of a 3-D CCA image using a constrained iterative deconvolution method (in Zeiss ZEN software) with medium strength regularization after 100 iterations and a 0.5% quality threshold. (d) $XY$-section images from the processed 3-D CCA image over a $2\text{-}\mu \mathrm{m}$ range display the change in focus of the conventional system over the same depth range. Each image is displayed by mapping its minimum and maximum intensity to a linear colorscale map.

Fig. 11

GCPM-EDFM results at depths (measured from the cover slip) within a fixed preparation of eosin-stained brine shrimp mounted in Cytoseal™ acquired with a $63\times /1.4$ NA oil lens show a DOF similar to a $10\times /0.3$ NA dry lens. Resolution is less than that of $\mathrm{NA}=1.4$ but improved relative to $\mathrm{NA}=0.3$. (a–c) Single frame GCPM-EDFM wide-field images restored from intermediate images at three different depths after 1000 iterations of the SIEM algorithm. (d and e) Intensity projections of SIEM processed CCA COSM images over a $3\text{-}\mu \mathrm{m}$-depth range. The range is centered around an acquisition depth $\sim 1\mu \mathrm{m}$ above the corresponding EDFM image (left panel). (f) Intensity projection of SIEM processed CCA COSM images over a $6.5\text{-}\mu \mathrm{m}$-depth range from the sum of 300 images at $0.1\text{-}\mu \mathrm{m}$ intervals after restoration of CCA images with SIEM. The roughness penalty weight used for both the EDFM and CCA images is $5\times {10}^{-4}$. All processed images were acquired with a $63\times /1.4$ NA lens. (g) An unprocessed CCA image acquired with a $10\times /0.3$ NA lens shows a wide field of view of a similar brine shrimp mounted in the same slide preparation as that in images (a)–(f). The inset indicates a region shown enlarged in (h and i), to show the small projections that are the structures in the ROI in images (a–f) and indicate their relationship to the brine shrimp anatomy. Image (i) is cropped from (g) while image (h) was acquired with a $63\times /1.4$ NA lens. All images are shown with saturated intensity color-maps in order to highlight the smallest structures.

Experimental CPM- [Fig. 7(a)] and GCPM-based [Fig. 7(d)] images show different, although very similar, bead pairs from two different locations in the same sample preparation (the two beads shown in the GCPM images are slightly further separated laterally and axially than those in the CPM images; however, they are at a similar depth). The DOF in experimental CPM- and GCPM-based EDF systems is analyzed via comparison with processed CCA wide-field images of the $6\text{-}\mu \mathrm{m}$ bead pair using two approaches. First, CPM- and GCPM-EDFM images are compared with axial intensity projections of a 3-D CCA image (referred to as COSM in Fig. 7), all restored with the SIEM and the RLLS algorithms.

Experimentally, the EDFM image is not identical to the intensity projection of truly optically sectioned images over the imaging volume due to contribution of out-of-focus emission from planes outside the achieved EDF. Thus, for our analysis, projections over a truncated volume of the CCA image were investigated and compared with the experimental EDFM images. The comparison shows qualitative agreement between the normalized intensity distribution of the EDFM and CCA COSM image when the 3-D CCA COSM image is projected over a $6\text{-}\mu \mathrm{m}$ DOF [Fig. 7(b)] for CPM and a $3\text{-}\mu \mathrm{m}$ DOF [Fig. 7(e)] for GCPM.

Second, CPM- [Fig. 8(a)] and GCPM- [Fig. 8(b)] EDFM (SIEM restored) images at varying depths over a 6- and a $3\text{-}\mu \mathrm{m}$ range, respectively, are compared with unprocessed CCA and 3-D-processed CCA images (or CCA COSM) over the same range relative to the center focal plane of the test sample, referred to as depth $d_o$ (Fig. 8). Rather than remaining invariant over the depth range identified above, details that are in-focus at the reference depth begin to blur toward the edges of the depth range. This result is consistent with the expectation that the bead-pair object is larger than the EDF of the system, which does not extend equally in both directions from $d_o$.

The two sets of unprocessed 3-D CCA images (Fig. 8) show that the depth over which objects remain resolved and undistorted provides an estimate of the $0.4\text{-}\mu \mathrm{m}$ DOF for the CCA system, calculated as the sum of the wave and geometrical optical depths of fields.^37,39 Using this approach, the EDF is judged to be $\sim 2$ and $1\mu \mathrm{m}$ for CPM- and GCPM-EDFM, respectively. These ranges correspond to $5\times$ and $2.5\times$ DOF extensions. Simulated CPM- and GCPM-EDFM images (Fig. 9) support this observation over a similar range of depth, in which in-focus objects remain undistorted. These simulated images are of a synthetic $6\text{-}\mu \mathrm{m}$ bead pair (described in Sec. 2.4) with the same size, shape, and RI properties as the experimental test sample. Note that EDFM significantly increases the image acquisition rate over a volume (by decreasing the number of images needed to be investigated) but does not necessarily preclude the need for a 3-D stack in every application.

Analysis of the effect of restoration on the SNR of the GCPM-EDFM experimental images of the $6\text{-}\mu \mathrm{m}$ bead pair [Fig. 7(d)] indicates that the SNR drop after processing (see Table 3) is greater than the one observed in simulation (Table 1). These larger changes in SNR not predicted by simulation indicate that additional factors, such as larger PSF mismatch error and other unaccounted sources of mismatch between the image model and the experimental WFE image formation, may also contribute to an overall loss in SNR.

Table 3

SNR in experimental GCPM-EDFM images shown in Fig. 7(d) and restoration algorithm parameters.

A similar analysis of the DOF as that given above for the $6\text{-}\mu \mathrm{m}$ bead pair is applied to experimental CPM- and GCPM-EDFM results of imaging live cells in culture. Images of epithelial lung cells show mitochondria labeled with MitoTracker green throughout the cell layers (Fig. 10). $XY$-section images from a CCA volume processed with 3-D deconvolution [Fig. 10(d)] show that varying elements of the cell layers are in and out of focus at different depths. The CCA COSM projection [Fig. 10(c)] shows the composite of this information and has a similar intensity distribution as that in the EDFM images [Figs. 10(a) and 10(b)] particularly in the central and brightest regions. In these areas, structures that are resolved at different planes in the CCA COSM images [Fig. 10(d)] can be identified in a single plane in the EDFM image [Fig. 10(a)] as well as in the CCA COSM projection [Fig. 10(c)]. In areas of low signal, some structures that appear in the CCA COSM projection are not visible in the EDFM images. This is consistent with the superior performance of EDFM at higher relative SNR observed in simulation. The resultant DOF after processing appears to be between 1 and $2\mu \mathrm{m}$ for both CPM- and GCPM-EDFM, which is consistent with the results from the $6\text{-}\mu \mathrm{m}$ bead.

Additional biological results are presented from EDFM images of microstructures in a fixed brine shrimp sample that is relatively thick ($\sim 6\mu \mathrm{m}$), compared with the conventional system’s DOF (Fig. 11). GCPM-based EDFM images at successive depths away from the coverslip restored with SIEM [Figs. 11(a)–11(c)] are compared with two different axial intensity projections computed over $3\mu \mathrm{m}$ from a 3-D deconvolved CCA image [Fig. 11(d) and 11(e)], as well as with an axial intensity projection over the full axial extent of the brine shrimp structure [Fig. 11(f)]. These results show that EDFM can be successful in producing images of objects away from the coverslip in the presence of object-induced SA.

4.1.

EDF and the Effect of Phase Mask Strength on Sensitivity to SA

The assumption that the EDFM image formation model can be represented with a 2-D PSF is valid over a larger depth range than is the case for conventional (CCA) image formation. For larger values of the design parameter $\alpha$, which controls the phase mask strength, the 3-D PSF is observed to have a longer axial extent, indicating a larger DOF. Although, the DOF is extended (from $0.7\mu \mathrm{m}$ in the conventional system to at least $5\mu \mathrm{m}$ in the EDFM of Fig. 3), it is not infinite. Thus, reconstruction artifacts occur when objects are outside the DOF in the intermediate image, i.e., when the 2-D model is no longer valid.

In the investigated simulation studies, restored results from the GCPM-EDFM more accurately represent the true object than the CPM-EDFM system. This is because the symmetry of the GCPM creates a 3-D PSF that is rotationally symmetric about the axial direction. Results from simulation with and without SA (Figs. 3 and 4) show that restored GCPM-EDFM images are in better agreement with each other than restored CPM-EDFM images. This demonstrates that the performance of the EDFM system based on the selected GCPM phase mask design is less sensitive to SA than a traditional CPM-based system ($\alpha =40$). The GCPM-EDFM images provide the correct location for each bead in the true object, whereas conventional CPM-EDFM images show lateral shifts (an order of magnitude greater than the optical resolution due to the curved shape of the CPM-PSF and its sensitivity to SA) in the position of beads that are located in $Z$-planes that do not coincide with the processed $Z$-plane. In the CPM case, the observed shift in the restored bead location is less with increased value of $\alpha$ [Figs. 4(a) and 4(c)].

Experimentally, the LC-SLM implementation of WFE limits the value of $\alpha$ for GCPM to a value ($|\alpha |=30$) that is relatively low compared with that selected and shown to improve EDF in simulation ($|\alpha |=150$). As a result, both the simulated and experimental GCPM-EDFM images of the bead-pair test sample show a smaller DOF produced by the GCPM ($|\alpha |=30$) than the CPM ($|\alpha |=30$). The size of the bead pair relative to the EDF enables an evaluation of the limits of the DOF for the experimentally implemented systems and aids in the interpretation of EDFM in the nonideal case for which the entire object is not within the EDF. Although this case in EDFM is analogous to conventional CCA fluorescence microscopy in 2-D, it must be considered that the in-focus portion of the 2-D EDFM image is a projection over a relatively thick volume. Simulated and experimental results of the bead pair also show that the fluorescent spherical shell is represented with better fidelity in the GCPM results as compared with the CPM within the range of the EDF. Despite its smaller EDF, the experimentally implemented GCPM design ($|\alpha |=30$) produces images with fewer artifacts than the CPM design ($|\alpha |=30$).

4.2.

Trade-off Between Achieved Resolution and SNR

Simulated and experimental images demonstrate results of model-based regularized linear image estimation applied to EDFM. As expected, when using regularization methods, the larger the regularization parameter value becomes, the more the achieved resolution is reduced (Table 2). Results showing an achieved resolution of 0.2 to $0.4\mu \mathrm{m}$ when the Rayleigh resolution limit of the simulated system is $0.32\mu \mathrm{m}$ demonstrate that EDFM in the presence of SA can achieve the conventional resolution limit in a noisy image with a properly regularized algorithm.

At the specific noise levels in this study, the drop in average SNR after processing is significantly reduced compared with Wiener filtering used in previous methods and is comparable to other nonlinear filtering methods in EDFM based on CPM.^12,29,40 Specifically, our simulation study is limited to two common noise level cases 12 and 17 dB, whereas our experimental study encompasses a noise level range of 10.4 to 14 dB in the intermediate image before processing. The maximum SNR drop after restoration was found to be as large as 46% in experiment compared with 20% in simulation. Overall, we observe a greater signal-dependent loss of resolution in the final EDFM images as compared with that in the final CCA COSM images. This observation is consistent with the differences in frequency attenuation in the MTF of the EDFM and CCA systems (Fig. 12). The EDFM system attenuates frequencies more than the CCA system, and it is, therefore, more sensitive to noise. The frequency attenuation in the bandpass region results in a more challenging inverse imaging problem, which cannot be solved without regularization in the presence of noise or other aberrations that cause a model mismatch in the restoration process.⁴¹

Fig. 12

Frequency attenuation in EDFM. $f_x-f_y$ (top row) and $f_x-f_z$ (bottom row) cut view images of the 3-D MTF from: (a) conventional CCA system, (b) EDFM system with selected GCPM design ($\alpha =150$ and $\beta =-3\alpha$), and (c) EDFM system with selected CPM design ($\alpha =40$). The MTF of the GCPM with $\left|\alpha \right|=30$ (not shown) used in the experimental studies attenuates frequencies less rapidly than the MTF in (b).