Translator Disclaimer
Open Access
27 September 2022 Hybrid brightfield and darkfield transport of intensity approach for high-throughput quantitative phase microscopy
Author Affiliations +
Abstract

Transport of intensity equation (TIE) is a well-established non-interferometric phase retrieval approach that enables quantitative phase imaging (QPI) by simply measuring intensity images at multiple axially displaced planes. The advantage of a TIE-based QPI system is its compatibility with partially coherent illumination, which provides speckle-free imaging with resolution beyond the coherent diffraction limit. However, TIE is generally implemented with a brightfield (BF) configuration, and the maximum achievable imaging resolution is still limited to the incoherent diffraction limit (twice the coherent diffraction limit). It is desirable that TIE-related approaches can surpass this limit and achieve high-throughput [high-resolution and wide field of view (FOV)] QPI. We propose a hybrid BF and darkfield transport of intensity (HBDTI) approach for high-throughput quantitative phase microscopy. Two through-focus intensity stacks corresponding to BF and darkfield illuminations are acquired through a low-numerical-aperture (NA) objective lens. The high-resolution and large-FOV complex amplitude (both quantitative absorption and phase distributions) can then be synthesized based on an iterative phase retrieval algorithm taking the coherence model decomposition into account. The effectiveness of the proposed method is experimentally verified by the retrieval of the USAF resolution target and different types of biological cells. The experimental results demonstrate that the half-width imaging resolution can be improved from 1230 nm to 488 nm with 2.5 × expansion across a 4 × FOV of 7.19 mm2, corresponding to a 6.25 × increase in space-bandwidth product from ∼5 to ∼30.2 megapixels. In contrast to conventional TIE-based QPI methods where only BF illumination is used, the synthetic aperture process of HBDTI further incorporates darkfield illuminations to expand the accessible object frequency, thereby significantly extending the maximum available resolution from 2NA to ∼5NA with a ∼5 × promotion of the coherent diffraction limit. Given its capability for high-throughput QPI, the proposed HBDTI approach is expected to be adopted in biomedical fields, such as personalized genomics and cancer diagnostics.

1.

Introduction

Organelles are implicated in multiple cellular life activities, and their metabolism or dysfunction is closely associated with the development and metastasis of cancers.1 The explorations of subcellular structures and their abnormal states facilitate insight into multiple pathological mechanisms, which are expected to achieve the early diagnosis and effective therapy of diseases.2,3 Hence, noninvasive high-throughput microscopic imaging is of great significance for attaining the precise detections of subcellular features and high-content quantitative analysis of multiple events in a large population of cells.47

Due to the low absorption or weak scattering of almost pure phase objects, the unstained cells have difficulty generating sufficient intrinsic contrast.810 Though these samples can be visualized by specific dyes or fluorescent labels, the phototoxicity and photobleaching of exogenous agents could damage the cell structure and prevent the long-term observation of live cells.11,12 Recently, quantitative phase imaging (QPI) has gained a lot of attention because of its unique capability to quantify the phase delay of unlabeled biological specimens in a nondestructive way.13,14 As a well-established deterministic QPI approach, the transport of intensity equation (TIE) was originally derived in the monochromatic coherent illumination situation and linearized to the phase retrieval problem under the paraxial approximation.15,16 The phase retrieval methods based on TIE enable quantitative phase microscopy by simply measuring intensities at multiple axially displaced planes. TIE has emerged as a promising QPI tool owing to the advantages of noninterferometric imaging that it is free of phase-unwrapping, and Köhler illumination compatibility within a brightfield (BF) microscope.1719 However, the achievable space-bandwidth product (SBP) of TIE is fundamentally restricted by the coherent optical system, exacerbating the tradeoff between image resolution and field of view (FOV).7

Generally, to increase the imaging system throughput, a low numerical-aperture (NA) objective is used to acquire a wider FOV and a larger NA illumination is applied for extending the maximum achievable resolution beyond the coherent diffraction limit. However, the original TIE is limited to the circular BF illumination and the phase contrast progressively vanishes as the illumination NA (NAill) approaches the objective NA (NAobj).20,21 Partial coherence expands the maximum obtainable imaging resolution but aggravates the noise-to-resolution compromise since the decrease in illumination coherence tends to weaken the phase effect.22 Due to the deviation from the coherent linearized model, the TIE method has difficulty retrieving accurate results with incoherent diffraction-limited resolution under a circle-shaped BF illumination (NAill=NAobj).15,16 This resolution-FOV tradeoff deficiency limits the applications of TIE-based QPI methods in high-throughput biomedical fields with high-SBP needs, including cancer diagnostics and drug development.5,7

To alleviate the above problem, researchers suggested to explicitly interpreting partial coherence by generalizing the coherent contrast transfer function to the weak object transfer function (WOTF) of a partially coherent system.20,23 As is predicted by the WOTF analysis, the high-resolution and high-contrast QPI can be achieved via boosting the TIE phase transfer function response within the theoretical framework of a transmission cross-coefficient based on the illumination modulation.24,25 In the annular illumination-based TIE (AI-TIE),25,26 the image resolution can be extended to 2NA because of the robust transfer function response at high frequencies when the circular illumination aperture is replaced with an annular one (where the NAill is equal to the NAobj). Nevertheless, the SBP of AI-TIE is still restricted within the incoherent diffraction limit. To meet the needs of noninvasive high-throughput imaging, it is hoped that TIE-related methods can exceed this limit to acquire high-resolution and wide FOV quantitative phase microscopy.

To the best of our knowledge, TIE is always limited by BF illumination,15,16,26 and the maximum attainable imaging resolution is restrained to the incoherent diffraction limit when matched annular illumination is used.25 Fundamentally, the resolution-to-FOV tradeoff of TIE-based QPI methods originates from the imperfection of the simplified and linearized imaging model, which ignores partial coherence or introduces strict assumptions. In general, higher NAill implies a larger spectral high-frequency cutoff frequency, increasing the ability of the microscope to record high-frequency details of the specimen. Specifically, the partial coherence in darkfield (DF) imaging (NAill>NAobj) extends the maximum achievable imaging resolution beyond the incoherent diffraction limit. But the image formation model of DF imaging transforms into a nonlinear inverse problem under the nonparaxial condition, which cannot be solved by TIE because of the model mismatch. Consequently, high-throughput TIE-based QPI surpassing the incoherent diffraction-limited resolution has not been reported so far.

In this work, we propose a hybrid BF and DF transport of intensity approach, termed HBDTI, to obtain high-throughput QPI with a large SBP. HBDTI uses a low-NA objective to acquire two through-focus intensity stacks corresponding to BF and DF illuminations with a programmable light-emitting-diode (LED) array. The forward imaging model of HBDTI is established to depict the captured intensity stacks under the partially coherent illuminations for both BF and DF imaging precisely and elegantly. Based on an iterative phase retrieval algorithm with coherence model decomposition, the synthetic aperture process of HBDTI combines BF illumination with high-angle DF illumination to synthesize a high-resolution and large-FOV complex amplitude. The effectiveness of HBDTI is validated by measuring the USAF resolution target and biological samples both with absorption and phase or pure phase. It is demonstrated that HBDTI achieves a 7.19  mm2 wide FOV of 488 nm half-width resolution that is 2.5× higher than the incoherent diffraction limit, corresponding to 30.2  megapixels SBP. Experimental results show the capability of HBDTI for high-SBP QPI to attain accurate detections of subcellular structures from large groups of cells across multiscales.

2.

Methods

The essential idea of computational phase microscopy based on TIE is to build an accurate forward image formation model and then implement the phase retrieval algorithm via solving the corresponding inverse problem.19,27 However, for partially coherent illumination, the imaging system is nonlinear in either amplitude or intensity, making phase retrieval complicated.25 To simplify the image formation process, weakening the effect of partial coherence and implementing restrictive assumptions, such as the paraxial approximation and the weak defocusing, are often applied to linearize the TIE-related phase retrieval problems. In linearized imaging models, quantitative phase distributions can be directly reconstructed by Fourier space deconvolution in one step by simplifying the model complexity to decouple the phase solution from the model explicitly.

For diffraction-limited optical imaging systems, higher NAobj and NAill raise the available angles and expand the accessible object frequency, thereby having a more powerful capability to resolve fine specimen details. However, when NAill>NAobj, the image formation process, from linear BF imaging (paraxial) to nonlinear DF imaging (nonparaxial), cannot be solved by the TIE linearized model.28,29 To obtain the imaging resolution over the incoherent diffraction limit, it is necessary to break the strict approximations of the original TIE and establish a more realistic forward physical model. The iterative solutions, such as Fourier ptychographic microscopy (FPM), can bypass the analytical modeling of the complex inverse problems and achieve high-resolution complex amplitude recovery by performing forward imaging models only. Thus, we establish an HBDTI forward imaging model to describe the measured intensity for both BF and DF imaging in the partially coherent fields precisely in an elegant way. HBDTI doesn’t require the restrictive assumptions like TIE as it considers the partial coherence of the image formation and utilizes coherence model decomposition in an iterative process.

In the actual measurement of the intensity image, the dynamic range of BF and DF intensity images differs by several orders of magnitude, and the signal-to-noise ratio (SNR) between corresponding intensity images is quite different as well.30,31 Thus, the intensity images are separately captured under these two illuminations for better SNR and the capture process takes full advantage of the detector dynamic range, as illustrated in Fig. 1(b). It is worth mentioning that acquiring three sets (or more) of data according to the illumination NA can further effectively enhance the imaging SNR. But considering that the difference in the dynamic range of DF images with different NAs is much smaller than that in the BF case, the improved SNR performance of this strategy is relatively limited and increases the complexity of data acquisition. Hence, the proposed HBDTI captures only two sets of through-focus intensity stacks in BF and DF for the tradeoff of SNR and acquisition efficiency. The utilization of through-focus in TIE-based QPI is a traditional approach to introducing the imaginary component of the optical transfer function for improving phase effect, thereby the phase information can be transferred into the defocused intensity images.23,32 Based on the above considerations, we capture two through-focus intensity stacks under BF and DF illuminations as the inputs of the convergent iterative process for solving the nonlinear image formation of HBDTI.

Fig. 1

Schematic diagram of the optical system comparison corresponding to the traditional FPM method and the proposed HBDTI method. (a) FPM requires a variably illuminated in-focus intensity stack that is captured under point illuminations. (b) HBDTI captures two through-focus intensity stacks under the discrete circle (BF) and the complementary-shaped (DF) illuminations, respectively.

AP_4_5_056002_f001.png

To illustrate the advantages of HBDTI from the perspective of optical principle, we compared the optical system of the traditional FPM method and that of the proposed HBDTI method, as shown in Fig. 1. According to Fig. 1(a), FPM requires a variably illuminated in-focus intensity stack (200 frames) with both BF and DF intensity images,3336 which are captured in low-SNR under point illumination. In terms of the matched illumination condition,37 FPM recovers low-frequency components only when the illumination angles are matched to the cut-off angles allowed by the objective pupil function. In contrast, HBDTI captures two through-focus intensity stacks (100 frames) under BF and DF illuminations. Using the discrete circle and complementary-shaped patterns shown in Fig. 1(b) rather than point sources, HBDTI obtains high-SNR intensities to ensure high quality and robustness of phase retrieval. Moreover, HBDTI bypasses the strict requirement of the matched illumination condition via through-focus scanning, avoiding the missing of low-frequency components.38 By contrast, HBDTI can achieve high-throughput quantitative phase microscopy as FPM based on a relatively small amount of data without the need to consider the matched illumination condition. In particular, the advantages of HBDTI compared with different QPI techniques are shown in Sec. 2 in the Supplementary Material.

We implement the HBDTI recovery algorithm by measuring two sets of Nz through-focus intensity stacks IBFcap and IDFcap corresponding to the BF and DF illuminations [Fig. 2(a)], where z is the defocus distance and zstep is the step size of the intensity stack. As shown in step 2 in Fig. 2(b), the captured in-focus intensity is upsampled for the initialization of the object complex amplitude U. Then, a series of high-resolution defocused intensity stacks are obtained through numerical propagation under corresponding LED illumination angles [step 3 in Fig. 2(c)], and the calculated low-resolution intensities after the downsampling of pixel binning can be treated as the input of the following intensity constraints process. In step 4 [Fig. 2(d)], the intensity images under different illumination angles are summed up to form the two intensity stacks IBFcal and IDFcal under corresponding BF and DF illuminations by coherent mode decomposition.34,39,40 The measured intensity stacks are divided by the two stacks as the factor to update the complex amplitude corresponding to various defocus distances. Inspired by the strategy of FPM,33 we apply the schemes of synthetic aperture and multiplexing to the difference map [step 5 in Fig. 2(e)],4144 and the intensity iterative constraint process is compatible with high-angle DF illuminations, which significantly expands the accessible object frequency. The complex amplitude stacks at different planes are backpropagated to the in-focal object plane and then these propagated complex amplitude stacks are synthesized in the Fourier domain to update the object complex amplitude. Finally, the high-resolution object complex amplitude can be retrieved by repeating step 3 to step 5 until the intensity constraint is satisfied upon convergence. More details are provided in Sec. 1 in the Supplementary Material.

Fig. 2

Roadmap of the proposed HBDTI method. (a) BF and DF defocused intensity stacks are measured under BF and DF illumination provided by 9×9 LEDs, respectively, as input data for the HBDTI algorithm. (b) BF in-focus intensity and zero phase combine to form the initial guess for the high-resolution complex amplitude. (c) Coherent mode decomposition for the estimated high-resolution complex amplitude to obtain equally spaced intensity stacks via the angular spectrum method. (d) The intensity constraints of the propagated complex amplitudes. (e) Updated frequency spectrum and the retrieval high-resolution intensity based on difference map. (f) High-resolution frequency spectrum and corresponding intensity reconstructed by HBDTI after iterations until convergence.

AP_4_5_056002_f002.png

It is worth mentioning that the HBDTI method proposed in this paper is different from traditional FPM.33,35,36 In FPM, it requires a variable-illuminated intensity stack (200 frames) including DF features at the in-focus plane under point illumination (leading to low-SNR intensity).33,45 Incorporating ptychographic phase retrieval and coherent synthetic aperture, FPM allows high-SBP imaging beyond the incoherent diffraction limit. But in terms of the matched illumination condition,37 FPM recovers low-frequency components only if the illumination angles match the cutoff angle allowed by the objective pupil function. Differently, HBDTI captures two through-focus intensity stacks (100 frames) under BF and DF illumination with discrete circular and complementary-shaped patterns for high-SNR intensity. By merging synthetic aperture and multiplexed illumination,46 the achievable resolution of HBDTI can exceed the incoherent diffraction limit for high-SBP imaging. In contrast to FPM, as a propagation-based phase recovery method, the low-frequency phase components of HBDTI can be completely transferred by z-scanning. Accordingly, HBDTI bypasses the strict requirement of the matched illumination condition by through-focus scanning, avoiding the loss of low-frequency components.38 Therefore, HBDTI can achieve the same high-SBP reconstruction capability as FPM only based on simpler operation and a smaller amount of data without the need for the matched illumination condition.

3.

Results and Discussion

We perform the simulation under the 632 nm wavelength LED illumination with a 5.5  μm pixel size detector and the 4×, 0.1 NA objective to verify the feasibility of the proposed HBDTI approach. The simulated 9×9 LED array of 120 mm below the sample is shown in Fig. 3(b), with a 6.5 mm distance between adjacent units. The original low-resolution data were defined with 64×64  pixels resolution, and the recovered 192×192  pixels high-resolution intensity IHR was calculated in iteration number Niter=10. As illustrated in Fig. 3(c), the details in the simulated low-resolution intensity image were severely blurred due to pixel binning. The HBDTI retrieval result in Fig. 3(d) presents more features than that in Fig. 3(c), such as the folds of clothes and the sideburns of the cameraman.

Fig. 3

Quantitative simulation results by using HBDTI. (a1) Cameraman image with 192×192  pixels as the ground truth value of intensity. (b1) and (b2) BF and DF intensity stacks were generated based on the forward model of the corresponding illumination mode after 3× downsampling of (a1). (c1) BF in-focus intensity is the initial intensity guess for the high-resolution complex amplitude. (d1) The high-resolution intensity is reconstructed by HBDTI after 50 iterations. (a2), (c2), and (d2) The Fourier frequency spectrum corresponding to (a1), (c1), and (d1), respectively. (a3), (c3), and (d3) The enlarged regions corresponding to the boxes in (a1), (c1), and (d1), respectively.

AP_4_5_056002_f003.png

Moreover, the high-throughput microscopy capability of HBDTI was validated by measuring the USAF resolution target, blood smear, and unlabeled Henrietta Lacks (HeLa) cells (both quantitative absorption and phase distributions) displayed in Figs. 4Fig. 56. The experiments were implemented on a commercial microscope (IX83, Olympus, Japan) that was equipped with a 4×, 0.16 NA objective (PLN, Olympus, Japan) utilizing a motorized focus drive with a minimum step size of 10 nm, an industrial camera (The Imaging Source DMK33UX183, pixel resolution 5472×3648, pixel pitch 2.4  μm) and a programmable LED array. Each LED (RS-1515MBAM, Nationstar) approximately consumes 40 mW of power and provides spatially coherent quasimonochromatic illumination through three individual channels (central wavelength red 632 nm, green 504 nm, blue 460 nm, and 20  nm bandwidth). The LED array is driven by a self-developed circuit with a field programmable gate array (FPGA) unit (EP4CE10E22C8N, Intel FPGA, United States) to provide logical control. The central 9×9 LED array with a distance of 2 mm between adjacent units was placed 12.3 mm above the sample, offering the largest effective NAill0.6 (in the case of BF illumination, NAill0.16). In the experiments, in total, 100 intensity frames are acquired within a 3D intensity set (5472×3648×100) for 50 axial z-slices under BF and DF illuminations achieved by a 9×9 LED array within 5.5 s acquisition time (55 ms exposure time for each frame under 4× objective lens). To improve computational efficiency, we adopt a nonmechanical image stitching algorithm of image segment recombination to reduce computational cost. Thus, in a normal laptop without GPU acceleration, our proposed method takes 1  min at ten iterations for 100 intensity images in the experiments.

Fig. 4

High-throughput imaging results of USAF absorption target. (a1) Setup of HBDTI system based on a commercial microscope equipped with a programmable LED source at the front-end of illumination and a drive mechanism at the back-end of acquisition. (a2) The low-resolution BF in-focus intensity image of USAF was captured under a large FOV of 7.19  mm2. (b1) The enlarged region of the yellow box in (a) corresponds to a half-pitch resolution of 1.23  μm. (b2) HBDTI recovery intensity image corresponding to (b1), whose half-pitch imaging resolution is 0.488  μm. (c1), (c2) Quantitative distribution of the HBDTI retrieval results corresponding to the white line in (b1) and (b2), respectively.

AP_4_5_056002_f004.png

Fig. 5

Imaging results of the stained blood smear. (a) BF low-resolution in-focus color image of stained blood cells. (b1) and (b2) BF and DF low-resolution in-focus intensity images of area 1 in (a). (b3), (c1), and (d1) The intensity images recovered by HBDTI in areas 1, 2, and 3 via summing the high-resolution HBDTI intensity results acquired separately for each of the three channels (λ=632  nm, 504 nm, and 460 nm). (b4), (c2), and (d2) The HBDTI retrieval high-resolution phase images attained at 632 nm wavelength in areas 1, 2, and 3, respectively. (e1) and (e2) Quantitative distribution of the HBDTI retrieval results along the respective arrow in (b1) and (b3), respectively (Video 1, mp4, 2.70 MB [URL: https://doi.org/10.1117/1.AP.4.5.056002.s1]).

AP_4_5_056002_f005.png

Fig. 6

QPI results of unlabeled HeLa cells (as the phase object). (a) The retrieval full-FOV label-free HeLa cells phase result using the HBDTI method shows approximately 4000 HeLa cells on a 7.19  mm2 FOV. (b1) and (c1) The low-resolution BF in-focus intensity images of areas 1 and 2 in (a), respectively. (b2) and (c2) The low-resolution DF in-focus intensity images of (b1) and (c1), respectively. (b3) and (c3) The retrieval phase results of (b1) and (c1) using the FFT-based TIE method, respectively. (b4) and (c4) The retrieval phase results of (b1) and (c1) utilizing the proposed HBDTI method, respectively (Video 2, mp4, 8.35 MB [URL: https://doi.org/10.1117/1.AP.4.5.056002.s2]).

AP_4_5_056002_f006.png

To demonstrate the throughput of HBDTI, the 1951 USAF resolution target (Ready Optics Company, United States) was captured with a 4×, 0.16 NA objective under illumination λ=632  nm as illustrated in Fig. 4. According to Sec. 3 in the Supplementary Material, the two sets of Nz=50 z-axis intensity stacks were measured in zstep=5  μm under BF and DF illuminations. For the measured in-focus intensity shown in Fig. 4(a2), the highest distinguishable resolution target bars were element 5 in group 8 [Fig. 4(b1), 1.23μm half-pitch resolution]. Compared with the measured BF intensity and the quantitative profile in Figs. 4(b1) and 4(c1), all the unit details of group 8 were resolved in the HBDTI recovery result in higher contrast shown in Figs. 4(b2) and 4(c2). As depicted in Fig. 4(b2), the highest resolvable target bars of HBDTI retrieval result were element 1 in group 10, with a half-pitch resolution of 0.488  μm. The maximum resolution of the experimental result presented the success of HBDTI to retrieve high-frequency features consistent with the theoretical half-pitch resolution of 0.452  μm, λ/(NAill+NAobj).47 It was validated that HBDTI achieved high-resolution imaging of 2.5× higher than the incoherent diffraction limit in the 0.16 NA objective, and the corresponding SBP was up to 30.2 megapixels with 6.25× improvement under the 4× FOV of 7.19  mm2.

HBDTI quantitatively recovers both phase and amplitude easily using two sets of defocused intensity stacks, which was confirmed by measuring the blood smear (Carolina Biological Supply Company, Burlington, North Carolina, United States) based on the 4×, 0.16 NA objective (see Fig. 5 and Video 1). The low-resolution BF in-focus intensity shown in Fig. 5(a) was combined by the intensity images captured under quasimonochromatic illumination at λ=632  nm (red), 504 nm (green), and 460 nm (blue), respectively. Applying the same experimental and computational parameters as the first experiment, the blurred edges and missing details of the blood cells in Fig. 5(b1) were retrieved using the proposed HBDTI method [Fig. 5(b3)]. Compared with the quantitative distribution in the measured intensity [Fig. 5(e1)], the profile of the blood cell was resolved clearly in the HBDTI recovery result both in intensity and phase [Figs. 5(e2) and 5(e3)]. In addition, as shown in Figs. 5(c1)5(d2), the sharp red blood cells boundaries and the clear white blood cells’ internal particles were distinguished under a large FOV containing 80,000 cells [Fig. 5(a)],48 proving the capability of HBDTI for precise detections of subcellular structures across the submicron scale to millimeter scale.

Our technique visualizes and provides high-resolution phase delay quantification of the unstained HeLa cells as the phase object, which was proved under the 4×, 0.16 NA objective with illumination λ=632  nm (see Fig. 6 and Video 2). HeLa cells were seeded (at an initial density of 300  cells/cm2) in a 35 mm glass-bottom Petri dish (No.0 Uncoated Coverslip, 10 mm Glass Diameter, MatTek Corporation, P35G010C) using DMEM high glucose with pyruvate (4.5 g 1:1 glucose, with GlutaMAX supplement, Gibco, Thermo Fisher Scientific or Roti-CELL DMEM, Roth) supplemented with 10% fetal bovine serum and 1× penicillin-streptomycin (both Gibco, Thermo Fisher Scientific). Cells were incubated at 37°C in a humidified atmosphere of 5% carbon dioxide for 8 h to allow attachment. After that, cells were fixed in phosphate-buffered saline (PBS) buffer (nm=1.34) on a microscope slide to preserve the cell morphology. According to Sec. 3 in the Supplementary Material, the two sets of Nz=50 z-axis intensity stacks were also measured in zstep=5  μm under BF and DF illuminations. We adopted a nonmechanical image stitching algorithm of image segment recombination to reduce computational costs.33 As illustrated in Fig. 6(a), about 4000 HeLa cells with subcellular details were observed in the HBDTI recovery phase under a large FOV of 7.19  mm2. For better visual effects, we normalized the dynamic display range of the BF intensity and DF intensity, respectively. As shown in Figs. 6(b1) and 6(b2) and Figs. 6(c1) and 6(c2), it was confirmed that the intensity images acquired by HBDTI based on the taking full advantage of the detector dynamic range have a high SNR. As shown in Figs. 6(b3) and 6(c3), because of the low spatial coherence of circle-shaped BF illumination, TIE retrieved blurred cell boundaries and subcellular details with low contrast. Meanwhile, the missing details in the TIE retrieval results were accurately recovered by utilizing the proposed HBDTI approach in a wide FOV [Figs. 6(b4) and 6(c4)], achieving noninvasive high-throughput imaging for the high-content analysis of subcellular structure detections.

4.

Conclusion

We have proposed a hybrid BF and DF transport of intensity approach for high-throughput QPI called HBDTI to solve the dilemma between the resolution and FOV. HBDTI extends the achievable imaging resolution beyond the incoherent diffraction limit via solving the nonlinear phase recovery problem under DF illumination, which is difficult for the linear formula of TIE. Utilizing two through-focus intensity stacks under BF and DF illuminations as input, HBDTI solves the nonlinear DF image formation model based on the coherent mode decomposition, which provides high-resolution recovery exceeding 2.5× the incoherent diffraction limit in a large FOV. The successful performance of HBDTI has been demonstrated by the recovery of the USAF absorption target and biological specimens both with absorption and phase or pure phase. The experimental results have verified that the proposed HBDTI can achieve a 7.19mm2 wide FOV with a half-width resolution of 488 nm, achieving an improvement of 6.25× in SBP from 5  megapixels to 30.2  megapixels. In the traditional TIE-based QPI method, whose illumination is limited in BF, its maximum attainable imaging resolution is at most 2NA. By combining BF with DF illuminations, the proposed HBDTI method has the capability to achieve the maximum resolution of about 5NA with a 5× increase in the coherent diffraction limit. HBDTI shows the high-throughput ability to record large FOV images without degradation of the spatial resolution and has the potential in delineating subcellular structures in large-scale cell studies applicable for relatively thin objects. Further efforts, such as the differential collection of BF and DF images to relieve the data requirement and reduce the capturing time, are needed to promote the high-speed implementation of HBDTI in large-group live cell analysis.

Acknowledgments

This work was supported by the National Natural Science Foundation of China (61905115, 62105151, 62175109, and U21B2033), Leading Technology of Jiangsu Basic Research Plan (BK20192003), Youth Foundation of Jiangsu Province (BK20190445, BK20210338), Fundamental Research Funds for the Central Universities (30920032101), and Open Research Fund of Jiangsu Key Laboratory of Spectral Imaging and Intelligent Sense (JSGP202105). L. L, J. L, Y. S, J. S, J. Z, and E. Y. L. declare no competing financial interests. C. Z and Q. C are named inventors on several related patent applications. C. Z. and Q. C. also have competing financial interests in Nanjing Jiangnan Novel Optics Co., Ltd. and Suzhou Flyingman Precision Instrument Co., Ltd. China, which, however, did not support this work.

Data, Materials, and Code Availability

Source data are available from the corresponding author on reasonable request. The customized MATLAB codes for HBDTI (Supplementary MATLAB source code and datasets are provided at the code page of SCILab website, https://www.scilaboratory.com/code.html). See also Sec. 7 in the Supplementary Material for supporting content.

References

1. 

Y. Shen et al., “Recent progress of surface-enhanced Raman spectroscopy for subcellular compartment analysis,” Theranostics, 11 (10), 4872 –4893 https://doi.org/10.7150/thno.56409 (2021). Google Scholar

2. 

M. T. Lin and M. F. Beal, “Mitochondrial dysfunction and oxidative stress in neurodegenerative diseases,” Nature, 443 (7113), 787 –795 https://doi.org/10.1038/nature05292 (2006). Google Scholar

3. 

L. A. Austin, B. Kang and M. A. El-Sayed, “A new nanotechnology technique for determining drug efficacy using targeted plasmonically enhanced single cell imaging spectroscopy,” J. Am. Chem. Soc., 135 (12), 4688 –4691 https://doi.org/10.1021/ja4011145 JACSAT 0002-7863 (2013). Google Scholar

4. 

E. Glory and R. F. Murphy, “Automated subcellular location determination and high-throughput microscopy,” Dev. Cell, 12 (1), 7 –16 https://doi.org/10.1016/j.devcel.2006.12.007 1534-5807 (2007). Google Scholar

5. 

V. Starkuviene and R. Pepperkok, “The potential of high-content high-throughput microscopy in drug discovery,” Br. J. Pharmacol., 152 (1), 62 –71 https://doi.org/10.1038/sj.bjp.0707346 (2007). Google Scholar

6. 

N. Meng et al., “Large-scale multi-class image-based cell classification with deep learning,” IEEE J. Biomed. Health Inform., 23 (5), 2091 –2098 https://doi.org/10.1109/JBHI.2018.2878878 (2018). Google Scholar

7. 

J. Park et al., “Review of bio-optical imaging systems with a high space-bandwidth product,” Adv. Photonics, 3 (4), 044001 https://doi.org/10.1117/1.AP.3.4.044001 AOPAC7 1943-8206 (2021). Google Scholar

8. 

G. Popescu, Quantitative Phase Imaging of Cells and Tissues, McGraw-Hill Education( (2011). Google Scholar

9. 

K. Lee et al., “Quantitative phase imaging techniques for the study of cell pathophysiology: from principles to applications,” Sensors, 13 (4), 4170 –4191 https://doi.org/10.3390/s130404170 SNSRES 0746-9462 (2013). Google Scholar

10. 

J. Mertz, Introduction to Optical Microscopy, Cambridge University Press( (2019). Google Scholar

11. 

R. H. Webb, “Confocal optical microscopy,” Rep. Prog. Phys., 59 (3), 427 https://doi.org/10.1088/0034-4885/59/3/003 RPPHAG 0034-4885 (1996). Google Scholar

12. 

D. Stephens and V. Allan, “Light microscopy techniques for live cell imaging,” Science, 300 82 –86 https://doi.org/10.1126/science.1082160 SCIEAS 0036-8075 (2003). Google Scholar

13. 

Y. Park, C. Depeursinge and G. Popescu, “Quantitative phase imaging in biomedicine,” Nat. Photonics, 12 (10), 578 –589 https://doi.org/10.1038/s41566-018-0253-x NPAHBY 1749-4885 (2018). Google Scholar

14. 

Y. Fan et al., “Smart computational light microscopes (SCLMs) of smart computational imaging laboratory (SCILab),” PhotoniX, 2 (1), 19 https://doi.org/10.1186/s43074-021-00040-2 (2021). Google Scholar

15. 

M. R. Teague, “Deterministic phase retrieval: a Green’s function solution,” J. Opt. Soc. Am., 73 (11), 1434 –1441 https://doi.org/10.1364/JOSA.73.001434 JOSAAH 0030-3941 (1983). Google Scholar

16. 

M. R. Teague, “Image formation in terms of the transport equation,” J. Opt. Soc. Am. A, 2 (11), 2019 –2026 https://doi.org/10.1364/JOSAA.2.002019 JOAOD6 0740-3232 (1985). Google Scholar

17. 

A. Barty et al., “Quantitative optical phase microscopy,” Opt. Lett., 23 (11), 817 –819 https://doi.org/10.1364/OL.23.000817 OPLEDP 0146-9592 (1998). Google Scholar

18. 

T. Chakraborty and J. C. Petruccelli, “Source diversity for transport of intensity phase imaging,” Opt. Express, 25 (8), 9122 –9137 https://doi.org/10.1364/OE.25.009122 OPEXFF 1094-4087 (2017). Google Scholar

19. 

C. Zuo et al., “Transport of intensity equation: a tutorial,” Opt. Laser. Eng., 135 106187 https://doi.org/10.1016/j.optlaseng.2020.106187 OLENDN 0143-8166 (2020). Google Scholar

20. 

E. Barone-Nugent, A. Barty and K. Nugent, “Quantitative phase-amplitude microscopy I: optical microscopy,” J. Microsc., 206 (3), 194 –203 https://doi.org/10.1046/j.1365-2818.2002.01027.x JMICAR 0022-2720 (2002). Google Scholar

21. 

L. Lu et al., “Accurate quantitative phase imaging by the transport of intensity equation: a mixed-transfer-function approach,” Opt. Lett., 46 (7), 1740 –1743 https://doi.org/10.1364/OL.422095 OPLEDP 0146-9592 (2021). Google Scholar

22. 

M. H. Jenkins, J. M. Long and T. K. Gaylord, “Multifilter phase imaging with partially coherent light,” Appl. Opt., 53 (16), D29 –D39 https://doi.org/10.1364/AO.53.000D29 APOPAI 0003-6935 (2014). Google Scholar

23. 

C. J. Sheppard, “Defocused transfer function for a partially coherent microscope and application to phase retrieval,” J. Opt. Soc. Am. A, 21 (5), 828 –831 https://doi.org/10.1364/JOSAA.21.000828 JOAOD6 0740-3232 (2004). Google Scholar

24. 

C. J. Sheppard, “Three-dimensional phase imaging with the intensity transport equation,” Appl. Opt., 41 (28), 5951 –5955 https://doi.org/10.1364/AO.41.005951 APOPAI 0003-6935 (2002). Google Scholar

25. 

C. Zuo et al., “High-resolution transport-of-intensity quantitative phase microscopy with annular illumination,” Sci. Rep., 7 7654 https://doi.org/10.1038/s41598-017-06837-1 SRCEC3 2045-2322 (2017). Google Scholar

26. 

J. Li et al., “Optimal illumination pattern for transport-of-intensity quantitative phase microscopy,” Opt. Express, 26 (21), 27599 –27614 https://doi.org/10.1364/OE.26.027599 OPEXFF 1094-4087 (2018). Google Scholar

27. 

J. Li et al., “High-speed in vitro intensity diffraction tomography,” Adv. Photonics, 4 (6), 066004 https://doi.org/10.1117/1.AP.1.6.066004 AOPAC7 1943-8206 (2019). Google Scholar

28. 

T. Löffler et al., “Terahertz dark-field imaging of biomedical tissue,” Opt. Express, 9 (12), 616 –621 https://doi.org/10.1364/OE.9.000616 OPEXFF 1094-4087 (2001). Google Scholar

29. 

S. Berujon, H. Wang and K. Sawhney, “X-ray multimodal imaging using a random-phase object,” Phys. Rev. A, 86 (6), 063813 https://doi.org/10.1103/PhysRevA.86.063813 (2012). Google Scholar

30. 

G. Zheng, C. Kolner and C. Yang, “Microscopy refocusing and dark-field imaging by using a simple led array,” Opt. Lett., 36 (20), 3987 –3989 https://doi.org/10.1364/OL.36.003987 OPLEDP 0146-9592 (2011). Google Scholar

31. 

L.-H. Yeh et al., “Experimental robustness of fourier ptychography phase retrieval algorithms,” Opt. Express, 23 (26), 33214 –33240 https://doi.org/10.1364/OE.23.033214 OPEXFF 1094-4087 (2015). Google Scholar

32. 

C. J. Sheppard and T. Wilson, “Fourier imaging of phase information in scanning and conventional optical microscopes,” Philos. Trans. R. Soc. Lond., 295 (1415), 513 –536 https://doi.org/10.1098/rsta.1980.0148 (1980). Google Scholar

33. 

G. Zheng, R. Horstmeyer and C. Yang, “Wide-field, high-resolution fourier ptychographic microscopy,” Nat. Photonics, 7 (9), 739 –745 https://doi.org/10.1038/nphoton.2013.187 NPAHBY 1749-4885 (2013). Google Scholar

34. 

L. Tian et al., “Multiplexed coded illumination for Fourier ptychography with an led array microscope,” Biomed. Opt. Express, 5 (7), 2376 –2389 https://doi.org/10.1364/BOE.5.002376 BOEICL 2156-7085 (2014). Google Scholar

35. 

J. Sun et al., “Resolution-enhanced Fourier ptychographic microscopy based on high-numerical-aperture illuminations,” Sci. Rep., 7 1187 https://doi.org/10.1038/s41598-017-01346-7 SRCEC3 2045-2322 (2017). Google Scholar

36. 

X. Chang, L. Bian and J. Zhang, “Large-scale phase retrieval,” eLight, 4 (1), 4 https://doi.org/10.1186/s43593-021-00004-w (2021). Google Scholar

37. 

Y. Baek and Y. Park, “Intensity-based holographic imaging via space-domain Kramers–Kronig relations,” Nat. Photonics, 15 (5), 354 –360 https://doi.org/10.1038/s41566-021-00760-8 NPAHBY 1749-4885 (2021). Google Scholar

38. 

J. Li et al., “Transport of intensity diffraction tomography with non-interferometric synthetic aperture for three-dimensional label-free microscopy,” Light: Sci. Appl., 11 (1), 154 https://doi.org/10.1038/s41377-022-00815-7 (2022). Google Scholar

39. 

S. B. Mehta and R. Oldenbourg, “Image simulation for biological microscopy: microlith,” Biomed. Opt. Express, 5 (6), 1822 –1838 https://doi.org/10.1364/BOE.5.001822 BOEICL 2156-7085 (2014). Google Scholar

40. 

S. Dong et al., “Spectral multiplexing and coherent-state decomposition in fourier ptychographic imaging,” Biomed. Opt. Express, 5 (6), 1757 –1767 https://doi.org/10.1364/BOE.5.001757 BOEICL 2156-7085 (2014). Google Scholar

41. 

V. Mico et al., “Synthetic aperture superresolution with multiple off-axis holograms,” J. Opt. Soc. Am. A, 23 (12), 3162 –3170 https://doi.org/10.1364/JOSAA.23.003162 JOAOD6 0740-3232 (2006). Google Scholar

42. 

C. Zheng et al., “High spatial and temporal resolution synthetic aperture phase microscopy,” Adv. Photonics, 2 (6), 065002 https://doi.org/10.1117/1.AP.2.6.065002 AOPAC7 1943-8206 (2020). Google Scholar

43. 

D. Dan et al., “Super-resolution and optical sectioning integrated structured illumination microscopy,” J. Phys. D, 54 (7), 074004 https://doi.org/10.1088/1361-6463/abc4a8 JPAPBE 0022-3727 (2020). Google Scholar

44. 

V. Elser, “Phase retrieval by iterated projections,” J. Opt. Soc. Am. A, 20 (1), 40 –55 https://doi.org/10.1364/JOSAA.20.000040 JOAOD6 0740-3232 (2003). Google Scholar

45. 

J. Sun et al., “Single-shot quantitative phase microscopy based on color-multiplexed fourier ptychography,” Opt. Lett., 43 (14), 3365 –3368 https://doi.org/10.1364/OL.43.003365 OPLEDP 0146-9592 (2018). Google Scholar

46. 

V. Mico et al., “Superresolved imaging in digital holography by superposition of tilted wavefronts,” Appl. Opt., 45 (5), 822 –828 https://doi.org/10.1364/AO.45.000822 APOPAI 0003-6935 (2006). Google Scholar

47. 

E. Narimanov, “Resolution limit of label-free far-field microscopy,” Adv. Photonics, 4 (5), 056003 https://doi.org/10.1117/1.AP.1.5.056003 AOPAC7 1943-8206 (2019). Google Scholar

48. 

M. M. Alam and M. T. Islam, “Machine learning approach of automatic identification and counting of blood cells,” Healthcare Technol. Lett., 6 (4), 103 –108 https://doi.org/10.1049/htl.2018.5098 (2019). Google Scholar

49. 

S. Chowdhury et al., “Structured illumination multimodal 3D-resolved quantitative phase and fluorescence sub-diffraction microscopy,” Biomed. Opt. Express, 8 (5), 2496 –2518 https://doi.org/10.1364/BOE.8.002496 BOEICL 2156-7085 (2017). Google Scholar

50. 

Y. Cotte et al., “Marker-free phase nanoscopy,” Nat. Photonics, 7 (2), 113 –117 https://doi.org/10.1038/nphoton.2012.329 NPAHBY 1749-4885 (2013). Google Scholar

51. 

C. Yuan et al., “Resolution improvement in digital holography by angular and polarization multiplexing,” Appl. Opt., 50 (7), B6 –B11 https://doi.org/10.1364/AO.50.0000B6 APOPAI 0003-6935 (2011). Google Scholar

52. 

P. Gao, G. Pedrini and W. Osten, “Structured illumination for resolution enhancement and autofocusing in digital holographic microscopy,” Opt. Lett., 38 (8), 1328 –1330 https://doi.org/10.1364/OL.38.001328 OPLEDP 0146-9592 (2013). Google Scholar

53. 

C. Zuo et al., “Wide-field high-resolution 3d microscopy with Fourier ptychographic diffraction tomography,” Opt. Laser. Eng., 128 106003 https://doi.org/10.1016/j.optlaseng.2020.106003 OLENDN 0143-8166 (2020). Google Scholar

54. 

G. Zheng et al., “Concept, implementations and applications of fourier ptychography,” Nat. Rev. Phys., 3 (3), 207 –223 https://doi.org/10.1038/s42254-021-00280-y (2021). Google Scholar

55. 

L. Tian and L. Waller, “Quantitative differential phase contrast imaging in an led array microscope,” Opt. Express, 23 (9), 11394 –11403 https://doi.org/10.1364/OE.23.011394 OPEXFF 1094-4087 (2015). Google Scholar

56. 

J. Sun, C. Zuo and Q. Chen, “Iterative optimum frequency combination method for high efficiency phase imaging of absorptive objects based on phase transfer function,” Opt. Express, 23 28031 –28049 https://doi.org/10.1364/OE.23.028031 OPEXFF 1094-4087 (2015). Google Scholar

Biography

Linpeng Lu received her BE degree from Nanjing University of Science and Technology. She is a fourth-year PhD student at Nanjing University of Science and Technology. Her research interests include computational microscopy, phase retrieval, and quantitative phase imaging. She is a member of SPIE and Optica.

Jiaji Li is a postdoc at the School of Electronic and Optical Engineering, Nanjing University of Science and Technology. He received his PhD from Nanjing University of Science and Technology in 2020. He was a visiting PhD student at the Department of Electrical and Computer Engineering, Boston University, from 2018 to 2019. He is currently a member of the Smart Computational Imaging Laboratory. His research interests include phase imaging and diffraction tomographic imaging. He is a member of SPIE and Optica.

Yefeng Shu is pursuing his PhD at Nanjing University of Science and Technology. His research interests include super-resolution microscopy, phase retrieval, and computational imaging. He is a member of SPIE and Optica.

Jiasong Sun received his BS degree from Soochow University in 2012 and his PhD from Nanjing University of Science and Technology in 2019. He is now an associate professor at the Department of Electronic and Optical Engineering. His research interest focuses on computational imaging, phase retrieval, super-resolution microscopy, diffraction tomography, and optical imaging. He is a member of SPIE and Optica.

Jie Zhou received his BE degree and is pursuing his ME degree at Nanjing University of Science and Technology. His research interests include deep learning-based phase retrieval and super-resolution microscopy. He is a member of SPIE and Optica.

Edmund Y. Lam received his BS degree, his MS degree, and his PhD, all in electrical engineering from Stanford University, in 1995, 1996, and 2000, respectively. He was the 49th PhD graduate of Prof. Joseph W. Goodman. He is now a professor in electrical and electronic engineering, the Director of the Computer Engineering Program, and the Founding Director of the Imaging Systems Laboratory at the University of Hong Kong. A recipient of the IBM Faculty Award, he is also a fellow of Optica, SPIE, IEEE, IS&T, as well as the Hong Kong Institution of Engineers.

Qian Chen received his BS, MS, and PhD degrees from Nanjing University of Science and Technology. He is currently a professor and vice-principal at Nanjing University of Science and Technology. He has been selected as Changjiang Scholar Distinguished Professor. With broad research interests in photoelectric imaging and information processing, he has authored more than 200 journal papers. His research team develops novel technologies and systems for non-interferometric quantitative phase imaging and high-speed 3D sensing and imaging, with particular applications in national defense, industry, and bio-medicine. He is a member of SPIE and Optica.

Chao Zuo received his BE and PhD degrees from Nanjing University of Science and Technology (NJUST) in 2009 and 2014, respectively. He was a research assistant at the Centre for Optics and Lasers Engineering, Nanyang Technological University, from 2012 to 2013. He is now a professor at the Department of Electronic and Optical Engineering and principal investigator of the Smart Computational Imaging Laboratory, NJUST. He has broad research interests in computational imaging and high-speed 3D sensing and has authored over 160 peer-reviewed journal publications. He has been selected for the Natural Science Foundation of China for Excellent Young Scholars and the Outstanding Youth Foundation of Jiangsu Province, China. He is a senior member of SPIE and Optica.

CC BY: © The Authors. Published by SPIE and CLP under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Linpeng Lu, Jiaji Li, Yefeng Shu, Jiasong Sun, Jie Zhou, Edmund Y. Lam, Qian Chen, and Chao Zuo "Hybrid brightfield and darkfield transport of intensity approach for high-throughput quantitative phase microscopy," Advanced Photonics 4(5), 056002 (27 September 2022). https://doi.org/10.1117/1.AP.4.5.056002
Received: 15 June 2022; Accepted: 15 August 2022; Published: 27 September 2022
JOURNAL ARTICLE
13 PAGES


SHARE
Lens.org Logo
CITATIONS
Cited by 1 scholarly publication.
Advertisement
Advertisement
KEYWORDS
Image resolution

Diffraction

Microscopy

Phase retrieval

Light emitting diodes

Objectives

Image processing

Back to Top