1.

Introduction

Fourier ptychography (FP) is a recently developed imaging technique that iteratively stitches together a number of variably illuminated low-resolution (LR) images in Fourier space to recover a large field of view (FOV), high-resolution (HR) image of a complex object.¹ Essentially, FP is a coherent imaging technique that utilizes oblique plane wave illumination to shift high spatial frequency content into the passband of the imaging system, similar to the concept of synthetic aperture technique.^2–6 FP can also be considered a phase retrieval technique that recovers the lost phase information from the irradiance measurements.^7–10 Recently, the FP technique has been demonstrated for quantitative phase imaging,¹¹ refocusing,¹² recovering unknown pupil aberrations,¹³ and three-dimensional intensity and phase imaging.^14,15 The acquisition speed and data requirements can be significantly improved without sacrificing image quality by utilizing a multiplexed illumination strategy,^16,17 optimizing the sampling pattern of the light emitting diode (LED) source,¹⁸ or utilizing content adaptive illumination.¹⁹ The FP reconstruction algorithm without using an alternating projection approach has been demonstrated.^20,21 However, in most previous demonstrations, FP was implemented in a transmission mode, which imposes a serious limitation on its applications.

To meet the practical needs for tissue imaging and surface inspection, we have been investigating reflective FP.²² However, in our previous reported system, the cut-off frequency is still limited to twice the objective’s NA divided by the wavelength $({2\mathrm{NA}}_{\mathrm{obj}}/\lambda )$. We can add LEDs around the objective to increase the illumination angle, therefore increasing the cut-off frequency using FP, but this type of system is bulky and difficult to implement. Most importantly, there is potentially a large gap in illumination angle between the light from the LEDs imaged onto the aperture stop and the LEDs arranged around the objective. This gap in illumination angles may not appropriately sample the object’s Fourier spectrum for the FP reconstruction algorithm.

The key challenge in reflection mode is effectively illuminating the object using large angles since the aperture stop of the objective determines the NA of the imaging channel, while also limiting the illumination NA. We propose a solution by placing the imaging aperture stop outside the illumination path, which allows the illumination NA to be much greater than the imaging NA of the objective. Thus, the cut-off frequency achieved using FP in the proposed optical concept is greater than $2{\mathrm{NA}}_{\mathrm{obj}}/\lambda$, which is the theoretical limit for the cut-off frequency with an epi-illumination configuration. The proposed optical system will find many applications in endoscopy, biomedical imaging, surface metrology, and industrial inspection.

2.

Theory

Consider an object with a complex spatial profile $o(x,y)$, which is coherently illuminated by an oblique plane wave at wavelength $\lambda$. The irradiance $I(x,y)$ is given by

In a reflective imaging system, a convenient way to produce the oblique illuminating plane waves at the sample plane is to place an image of a point source at the back focal plane of the objective lens. Since the point source is positioned at the back focal plane of the objective, the output from the objective is an oblique collimated wave. A small LED imaged onto the back focal plane of the objective can be used to approximate a point source. This method approximates the plane waves necessary for FP. Using this set-up in air, the illumination ${\mathrm{NA}}_{\mathrm{ill}}$ is

Fig. 1

Illumination conditions for (a1) normal illumination, (a2) brightfield illumination, and (a3) darkfield illumination. Images of USAF target under (b1) normal illumination, (b2) brightfield illumination, and (b3) darkfield illumination. (c) Fourier spectrum of object with regions that correspond to (b1)–(b3) identified.

The FP recovery procedure utilizes a phase retrieval algorithm to estimate the complex object that creates the irradiance image captured using an oblique plane wave illumination. A more detailed description of the FP recovery procedure can be found in Refs. 1 and 17. In brief, the FP algorithm alternates between the spatial and Fourier domain for the specified region in Fourier space imposing the irradiance measurement as a constraint for that subregion in the spatial domain. The Fourier spectrum of the object is iteratively updated for $N$ raw irradiance images, each illuminated with a unique oblique plane wave to further expand the spectrum of the object in the Fourier domain. The whole iteration scheme is repeated for $M$ cycles until the complex reconstructed solution for the amplitude and phase of the object converges. The synthesized NA of the system after the FP recovery procedure is equal to the sum of the ${\mathrm{NA}}_{\mathrm{obj}}$ of the objective and the maximum ${\mathrm{NA}}_{\mathrm{ill},\max }$ of the illuminating plane waves. Therefore, the synthesized numerical aperture ${\mathrm{NA}}_{\mathrm{syn}}={\mathrm{NA}}_{\mathrm{obj}}+{\mathrm{NA}}_{\mathrm{ill},\max }$.

An epi-illumination microscope is a convenient system for reflective FP, however, this system has a major limitation. If an epi-illumination microscope is used for FP, the LEDs are imaged onto the back focal plane of the objective, which is the location of the aperture stop in an epi-illumination microscope. Since the physical aperture stop of the objective blocks the images of the LEDs outside the aperture radius $R_{\mathrm{EP}}$, the maximum illumination NA for the plane waves is the same as ${\mathrm{NA}}_{\mathrm{obj}}$ of the objective. Thus, the maximum synthesized NA using FP with an epi-illumination microscope is $2{\mathrm{NA}}_{\mathrm{obj}}$. An epi-illumination microscope can extend the cut-off frequency of the reconstructed FP image to the incoherent limit of $2{\mathrm{NA}}_{\mathrm{obj}}/\lambda$. While there is no advantage in terms of resolution over incoherent imaging in epi-illumination mode, the reconstructed FP images are advantageous, since the transfer function is coherent-like and the contrast is higher at high spatial frequencies.²²

To achieve a cut-off frequency greater than $2{\mathrm{NA}}_{\mathrm{obj}}/\lambda$ using reflective FP, the illumination ${\mathrm{NA}}_{\mathrm{ill}}$ must be greater than ${\mathrm{NA}}_{\mathrm{obj}}$. More LEDs can be arranged around the objective to obtain large angle illumination, but the system is bulky and, there is potentially a large gap in illumination angles between the light from the LEDs imaged onto the aperture stop and the LEDs arranged around the objective. The proposed optical concept for reflective FP is shown in Fig. 2, the imaging stop is placed outside the illumination path through a relay lens between the objective lens and the tube lens. The illumination path remains the same, and the illuminating light source is reflected off a beam-splitter onto the back focal plane of the objective providing the necessary plane wave illumination for FP. The imaging NA is determined by the external stop size, the illumination NA will not limited by the stop size, but by the lens size.

Fig. 2

Optical system for reflective Fourier ptychography. The imaging stop is positioned outside the illumination path using a relay lens between the objective lens and tube lens.

With the proposed system, a simple large NA and large FOV objective can be developed for FP microscopy to obtain a HR image over a large FOV. This objective only requires diffraction-limited performance within the low imaging ${\mathrm{NA}}_{\mathrm{obj}}$, which is determined by the aperture placed in the conjugated plane of the aperture stop of the objective. The size of the objective lens can be increased significantly to provide large illumination ${\mathrm{NA}}_{\mathrm{ill}}$, since good aberration correction is not needed for the illumination path. Using this system, the synthesized ${\mathrm{NA}}_{\mathrm{syn}}$ for reflective FP can exceed $2{\mathrm{NA}}_{\mathrm{obj}}$.

3.

Experiment

For the experimental demonstration, all samples are imaged using a simple doublet lens, which is an achromatic 50-mm focal length lens (Thorlabs AC254-050-A-ML), and a scientific CMOS sensor (PCO.edge, $6.5\text{-}\mu \mathrm{m}$ pixel pitch). This achromatic doublet is not a microscope objective, its effective imaging NA is not 0.25 as calculated from the first-order optics. It is used in this experiment to demonstrate the proposed concept because it has a small effective imaging NA 0.03 over large field of view, but with a large lens diameter of 25 mm. Because it is a single doublet, we can fully use its aperture for illumination.

Commercial objectives are not used in this experiment because all commercial objectives have internal aperture stops, we cannot illuminate the object with an angle larger than the NA. In the future, custom objectives with a large diameter, but a smaller imaging NA can be developed for specific applications. The image relay system has a magnification of $-1$. A single 660-nm fiber-coupled LED with a 25-nm bandwidth and $105\text{-}\mu \mathrm{m}$ core diameter is used as the light source. The LED is translated using an $x-y$ translation stage to move the image of the LED at the back focal plane of the objective. The LED is moved in a square array at the back focal plane of the objective to generate images of the object using 169 unique oblique illuminating plane waves. Due to the small size of the LED, the illumination is spatially coherent at the sample plane. An adjustable iris in the conjugated plane of the aperture stop of the objective is used to control the diameter of the stop. Due to the aberration of Thorlabs doublet, the remote aperture limits the imaging NA to 0.025, slightly smaller than the effective 0.03 NA.

To maintain sufficient spatial coherence in the reconstructed area, the FP image is reconstructed in patches of 150 pixels. For the 169 images, ${\mathrm{NA}}_{\mathrm{syn}}$ is 0.113 in the $\xi$ and $\eta$ directions. First, a USAF target is imaged. The image of the USAF target using normal illumination with ${\mathrm{NA}}_{\mathrm{obj}}=0.025$ is shown in Figs. 3(a) and 3(b), where Fig. 3(b) magnifies groups 6 to9. With this NA, element 3 in group 5 ($40.3\text{cycles}/\mathrm{mm}$) is just resolvable. Using the proposed optical system and the FP recovery procedure with 169 unique plane wave illuminations with the proposed set-up, the synthesized NA is increased from ${\mathrm{NA}}_{\mathrm{obj}}=0.025$ to ${\mathrm{NA}}_{\mathrm{syn}}=0.113$. Figure 3(c) shows the irradiance image of groups 4 to 9 and 3(d) magnifies groups 6 to 9 after the FP algorithm is used. Note all elements in group 5 are now resolvable. Furthermore, element 4 in group 7 ($181\text{cycles}/\mathrm{mm}$) is now just resolvable, as shown in Fig. 3(d). Using FP, a spatial frequency that is 4.5 times higher can be resolved with the proposed optical system. The FP recovery also estimates phase of the object, which is shown in Fig. 3(e) and magnified in Fig. 3(f).

Fig. 3

Raw irradiance image of USAF target with ${\mathrm{NA}}_{\mathrm{obj}}=0.025$ of (a) groups 4 to 9 and (b) groups 6 to 9. Irradiance image of USAF target after using FP recovery procedure with ${\mathrm{NA}}_{\mathrm{syn}}=0.113$ of (c) groups 4 to 9 and (d) groups 6 to 9. Phase estimate after using FP recovery procedure of (e) groups 4 to 9 and (f) groups 6 to 9.

The images reconstructed using FP are now compared with images of the USAF target when the objective has a similar NA. The NA of the objective is modified by varying the diameter of the stop, which is an adjustable iris. As shown in Fig. 4, the images acquired using an objective with an $\mathrm{NA}=0.04$ and the reconstructed image using FP have the same performance. The reconstructed images using FP utilize only nine brightfield images to reconstruct an image with ${\mathrm{NA}}_{\mathrm{syn}}=0.04$. Figure 4(a) shows groups 4 to 9 and Fig. 4(b) magnifies groups 6 to 9 of the image of the USAF target reconstructed using the FP recovery procedure with the nine brightfield images. Figure 4(c) shows groups 4 to 9 and Fig. 4(d) magnifies groups 6 to 9 of the image of the USAF target acquired when diameter of the stop is increased until the ${\mathrm{NA}}_{\mathrm{obj}}$ of the objective is 0.04. As shown in Figs. 4(a) and 4(c), all elements in group 5 are resolvable. As shown in Figs. 4(b) and 4(d), the last resolvable element is group 6 element 1 ($64.0\text{cycles}/\mathrm{mm}$). In terms of resolution, the image reconstructed using FP with nine brightfield images performs the same as an objective with ${\mathrm{NA}}_{\mathrm{obj}}=0.04$.

Fig. 4

USAF irradiance image of (a) groups 4 to 9 and (b) groups 6 to 9 reconstructed using FP recovery procedure with nine brightfield images for a synthesized ${\mathrm{NA}}_{\mathrm{syn}}=0.04$. USAF irradiance image of (c) groups 4 to 9 and (d) groups 6 to 9 acquired with objective with ${\mathrm{NA}}_{\mathrm{obj}}=0.04$.

Now the irradiance image of the USAF target with ${\mathrm{NA}}_{\mathrm{syn}}=0.113$ reconstructed using all 169 images is compared with an image of the USAF target when the diameter of the stop is increased such that the objective ${\mathrm{NA}}_{\mathrm{obj}}=0.118$. As shown in Fig. 5(a) and 5(b), group 7 element 4 ($181\mathrm{cycles}/\mathrm{mm}$) is just resolvable, which is near the theoretical limit of $171\text{cycles}/\mathrm{mm}$. Since the USAF target is comprised three bar targets instead of sinusoidal targets, the system is capable of resolving elements greater than the theoretical cut-off frequency at reduced contrast. Figure 5(c) shows groups 4 to 9 and 5(d) magnifies groups 6 to 9 of the USAF target when the stop is opened such that ${\mathrm{NA}}_{\mathrm{obj}}=0.118$. A Zemax model of the objective lens at this ${\mathrm{NA}}_{\mathrm{obj}}$ is shown in Fig. 5(e). Note as the stop is increased to this ${\mathrm{NA}}_{\mathrm{obj}}$, the objective is no longer diffraction-limited. Figure 5(f) shows the spot diagram from Zemax for this objective lens with ${\mathrm{NA}}_{\mathrm{obj}}=0.118$ when the system is optimized for minimum RMS wavefront. Note the spot diagram is larger than the Airy disk diameter, which is shown as a black circle. Thus, the aberrations in the objective when ${\mathrm{NA}}_{\mathrm{obj}}=0.118$ degrades the performance, which is evident in Fig. 5(d), since the last resolvable element is now only group 7 element 1 ($128\text{cycles}/\mathrm{mm}$).

Fig. 5

USAF irradiance image reconstructed using FP recovery procedure with 169 images with synthesized ${\mathrm{NA}}_{\mathrm{syn}}=0.113$ of (a) groups 4 to 9 and (b) groups 6-9. USAF irradiance image acquired with objective with ${\mathrm{NA}}_{\mathrm{obj}}=0.118$ of (c) groups 4 to 9, and (d) groups 6 to 9. (e) Zemax model of objective lens when ${\mathrm{NA}}_{\mathrm{obj}}=0.118$. (f) Spot diagram from Zemax when RMS wavefront is minimized at ${\mathrm{NA}}_{\mathrm{obj}}=0.118$.

Due to the aberrations, this objective lens is not suitable for diffraction-limited imaging with a ${\mathrm{NA}}_{\mathrm{obj}}=0.118$, as shown in Fig. 5(d). By using the effective ${\mathrm{NA}}_{\mathrm{obj}}$ of 0.025 and using the FP recovery procedure with 169 images with unique oblique plane wave illumination, the reconstructed image has an equivalent performance to a diffraction-limited objective with ${\mathrm{NA}}_{\mathrm{syn}}=0.113$, as shown in Fig. 5(b). Therefore, using FP with the proposed reflective imaging system results in better performance than this objective can achieve, since the performance of the objective lens is degraded by aberrations.

To demonstrate the application of the proposed system for tissue imaging, a slice of tissue was cut from the chicken thigh and was sandwiched between two glass microscope slides to create a flat surface. Figure 6(a) shows a large area of the chicken tissue after using the FP recovery procedure using 169 raw images to obtain a ${\mathrm{NA}}_{\mathrm{syn}}$ of 0.113. To demonstrate the improvement in the image, Fig. 6(b1) shows the raw image taken with ${\mathrm{NA}}_{\mathrm{obj}}=0.025$ of the subregion of the image in the indicated region in the middle of 6(a), Fig. 6(b2) shows the same subregion when the diameter of the stop is increased such that ${\mathrm{NA}}_{\mathrm{obj}}$ is 0.118, and Figure 6(b3) shows the same subregion reconstructed using FP. As shown in Fig. 6(b1), the details of the large structure are not resolvable and the smaller features particularly along the tilted white line along the bottom of the image are not resolvable. Figure 6(b2) is able to resolve the details of the large structure and show there is a small hole in the middle of it. Furthermore, it is now obvious that the large structure is surrounded by smaller circular structures that were not resolvable in Fig. 6(b1). Finally, Fig. 6(b3) resolves the finer features of the large structure as well as the smaller structures surrounding the large structure with higher contrast than Fig. 6(b2) since the image in Fig. 6(b2) is degraded by aberrations. Figure 6(c)–6(g) magnify other regions of Fig. 6(a). Note the small circular structures pervade all of these images. Therefore, the reflective FP optical system is capable of imaging reflective, planar objects with a resolution greater than the incoherent limit.

Fig. 6

(a) Large area of tissue from chicken thigh reconstructed with ${\mathrm{NA}}_{\mathrm{syn}}=0.113$ using FP recovery procedure. Image of indicated subregion using (b1) objective with ${\mathrm{NA}}_{\mathrm{obj}}=0.025$, (b2) objective with ${\mathrm{NA}}_{\mathrm{obj}}=0.118$, and (b3) recovered using FP with ${\mathrm{NA}}_{\mathrm{syn}}=0.113$. (c)–(g) Magnified regions indicated in (a).

4.

Conclusion

By placing the imaging stop outside the illumination path in a reflective microscope, the NA of the illumination can be much greater than the NA of the imaging system. Using this optical concept and the FP recovery procedure, a synthesized ${\mathrm{NA}}_{\mathrm{syn}}$ of the optical system can be greater than $2{\mathrm{NA}}_{\mathrm{obj}}$, which is the theoretical limit using an epi-illumination microscope. By imaging a USAF target, it was experimentally shown that the resolution of the objective was increased from ${\mathrm{NA}}_{\mathrm{obj}}=0.025$ to ${\mathrm{NA}}_{\mathrm{syn}}=0.113$, which is an enhancement by a factor of 4.5. Furthermore, a slice of tissue from the thigh of a chicken sandwiched between two microscope slides was also imaged to demonstrate the system performance.

The key advantage of the proposed concept is that it can achieve HR imaging over the large FOV with a simple objective. The objective only requires diffraction-limited performance within the imaging NA, while easily increasing the lens diameter for large angle illumination without controlling the aberrations. With significantly less requirements on aberration correction (namely, spherical aberration, coma, field curvature, astigmatism, and distortion), the objective lens can be much simpler with much larger working distance and FOV.

The proposed method is not limited to a resolution improvement of 4.5. In practice, the system performance is limited by the diameter of the objective lens. Assuming the diameter of the objective lens is large enough to transmit the illuminating plane wave without vignetting, the maximum achievable shift in Fourier space is ${\mathrm{NA}}_{\mathrm{ill},\max }=1.0$ in air. Therefore, the maximum achievable synthesized NA in air using this method is ($1+{\mathrm{NA}}_{\mathrm{obj}}$), where ${\mathrm{NA}}_{\mathrm{obj}}$ is the imaging NA of the objective lens.

In the future, we can also use the multiplexed illumination scheme^16,17 or the nonuniform Fourier-space sampling scheme ¹⁸ to further reduce the number of raw images for achieving the same imaging performance. We envision the proposed optical system can find applications in endoscopy, biomedical imaging, surface metrology, and industrial inspection.