Dither removing Fourier ptychographic microscope based on a two-axis rotation stage

Abstract. Significance: Large space-bandwidth product is highly desirable in many biomedical imaging. Fourier ptychographic microscopy (FPM) is a computational imaging technique that can significantly increase the space-bandwidth product of a standard microscope. The illuminator of a Fourier ptychographic microscope is not flexible at present, and it is inconvenient to meet different imaging needs. Aim: An illuminator based on a two-axis motorized rotation stage was presented to provide a more flexible illuminating way with the goal of meeting different imaging needs. Approach: The illuminator adopts a concentric illuminating method to provide coherent illumination in any direction on the sample plane. The sampling pattern can be freely designed and changed according to the parameters of the imaging system. A dither removing algorithm was proposed to remove the potential dither influence introduced in the image acquisition process. Results: The illuminator could be conveniently integrated into different imaging systems. The feasibility and flexibility were demonstrated by applying it to imaging systems with numerical aperture of 0.045 and 0.01. The resolution gain is about 4- and 13-fold, respectively. The effectiveness of the dither removing algorithm was validated in both simulation and experiment. Conclusions: A more flexible illuminator for FPM was presented to meet different imaging needs. A dither removing algorithm was proposed to remove dither influence.


Introduction
Fourier ptychographic microscopy (FPM) is a recent computational imaging technique that is capable of increasing the resolution of a standard microscope without sacrificing the field of view (FOV). [1][2][3][4] By applying a phase retrieval algorithm, FPM stitches a series of low-resolution images to produce a high-resolution image. 5,6 In addition, the phase of the reconstructed high-resolution image can be recovered without knowing the phases of the low-resolution images. 4 FPM has made great progress in many aspects, such as apparatus structure, 7-11 experimental method, [12][13][14][15][16][17][18][19][20][21] parameter correction, 3,[22][23][24] and reconstruction algorithm. [25][26][27][28] So far, most FPM platforms are using a light-emitting diode (LED) array as an illuminator. An LED array can provide stable and fast illumination without mechanical movement. However, an LED array has its own drawbacks as an illuminator in FPM. First, the intensity of an LED is relatively weak, especially for large-angle illumination. Second, the distribution density of LEDs is fixed, which is inflexible for adjusting the sampling density in spectrum. Third, the largest illumination angle is limited, which is inflexible for synthesizing a very large NA. Fourth, the positions of all LEDs are fixed; therefore, the sampling patterns with an LED array are limited. These drawbacks of an LED array lead to the emergence of some different illuminators in FPM. Chung et al. 9 have developed an illuminator using laser source guided by a galvo-scanner and a mirror array and achieved fast acquisition, but it is inflexible for the mirror array to change the sampling pattern. Guo et al. 29 have developed a flexible illuminator using liquid crystal display, the illuminator is convenient to change the sampling pattern but is inconvenient for large incident angles since the maximum incident angle is limited by the condenser. There are other different illuminators, such as quasi-dome LED array, 30 hemispherical condenser, 10,31 and illuminator with digital micromirror device. 8,32 These illuminators are still inflexible in some aspects; therefore, they could not be conveniently utilized to investigate the FPM technique under different conditions.
To provide a more convenient way to investigate FPM technique in depth, we present a more flexible illuminator for FPM. The illuminator uses a laser as its illumination source mounted on a two-axis motorized rotation stage. We adopt a concentric illuminating method, that is, the laser always illuminates the intersection of the two axes. The illuminator is capable of providing coherent illumination in any direction with an angular resolution of 0.0003125 deg and a maximum incident angle of 90 deg. Therefore, it is easy to design and change the sampling pattern in the FPM with a very high degree of freedom. The illuminator can be easily integrated into different FPM platforms with different parameters. It is quite convenient to use this illuminator to investigate various deep-seated problems related to FPM.
To reconstruct high-resolution image, FPM needs to capture a series of low-resolution images. Keeping the acquisition process stable is essential for reconstructing high-quality image. Our illuminator is driven in mechanical way, which inevitably introduces little dither during the FPM acquisition process. One feasible approach is to wait until the platform is stable before each image acquisition. However, this approach would greatly increase the acquisition time. To obtain the next image quickly without being disturbed by dither, we propose a dither removing algorithm for FPM. The dither removing algorithm is able to reconstruct a high-resolution image successfully from a series of low-resolution images polluted by dither. With the dither removing algorithm, the stability requirement can be relaxed and the acquisition time can be shortened.
The remainder of this paper is structured as follows. In Sec. 2, we describe the schematic of the setup that we used in the experiments and the principle of the dither removing algorithm. In Sec. 3, we first verify the effectiveness of our proposed dither removing algorithm by simulation. Then we perform three experiments: The first experiment demonstrates the feasibility of our illuminator and the effectiveness of our proposed dither removing algorithm. The second experiment shows the flexibility and wide applicability of the illuminator. The last experiment shows that the illuminator is applicable for real biological specimens. In Sec. 4, we present the conclusions of our works.

Experimental Setup
We integrate our illuminator into an FPM platform, which is described as follows. The illumination source is a laser whose wavelength is 635 nm. The laser is mounted on a two-axis motorized rotation stage. We adopt a concentric illuminating method, that is, the laser always illuminates the intersection of the two axes. We denote the angle associated with the vertical axis as θ and the angle associated with the horizontal axis as φ. θ and φ can cover the whole spatial angle with an accuracy of 0.0003125 deg. We adjust the laser beam to coincide with the vertical axis at the beginning. Benefitting from the flexibility of the rotation stage, the sampling pattern can be freely designed and changed according to the system parameters, making the illuminator widely applicable. The specimen is placed at the intersection of the two axes. The imaging lens is a 1× telecentric lens (WWH10-110CT-G, Coolens), whose numerical aperture (NA) is adjustable in the range of 0.005 to 0.045. A complementary metal-oxide-semiconductor camera is used with a pixel size of 3.45 μm (GS3-U3-123S6M-C, FLIR). The structure of the imaging platform is shown in Fig. 1.

Dither Removing Algorithm
In a general microscope imaging platform, the pixel size on the object plane can be calculated by d∕mag, which is about a few hundred nanometers to a few microns, where d is the pixel size of the camera and mag is the magnification of the system. It is inevitable to introduce little dither especially when starting and stopping mechanical movement. In FPM, the presence of dither will degrade the quality of the reconstructed image. It becomes especially essential to find the unknown individual shifted value of each low-resolution image correctly. Bian et al. 23 have tackled the sample motion problem using an annealing method. However, the dither in our application is relatively small, so here we use a more lightweight method to solve the dither problem. The main idea of our algorithm is straightforward. For each illumination direction, after updating the spectrum in conventional FPM recovery procedure, 1 we use this spectrum to correct the dither value of next image. We simulate the imaging process of the corresponding illumination direction. We assume that the captured image has the identical distribution as the simulated image except for a lateral shift. We apply the phase correlation algorithm to find the shifted value of the captured image 33,34 and use the found shifted value to shift the captured image back. Then the square root of the dither removed image is used to substitute the amplitude of the simulated image. The algorithm is described in detail below.
As suggested by the FPM recovery algorithm, 1 we initialize an estimation of the high-resolution spatial spectrumÕðf Þ first, where f represents the spatial frequency coordinate. Since the estimated spectrum could be different from the true spectrum, if necessary, we could perform a few iterations of the conventional FPM recovery procedure (experiments show that two iterations are sufficient). Then we calculate the simulated low-resolution image I s i ðrÞ under illumination of the i'th direction, 22 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 1 1 6 ; 1 9 7 where r represents the spatial coordinate, F −1 represents the inverse Fourier transform, f i is the illumination wavenumber of the i'th image, andPðf Þ represents the pupil function. Let I i ðrÞ denote the recorded image under illumination of the i'th direction. Under our assumption, we have E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 1 1 6 ; 1 1 5 where α i is a constant factor and r i is the unknown i'th shifted value introduced by dither. If we can find the exact value of r i , then we can remove the influence of dither by simply replacing where δðrÞ is the Dirac delta function. We determine r i by E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 1 1 6 ; 5 8 3 In practice, Eq. (2) usually does not hold, nor is D i ðrÞ a shifted Dirac function. However, as long as I s i ðrÞ is similar with I i ðrÞ, D i ðrÞ is a unimodal function, and Eq. (5) still works. We will not prove it analytically, as it will be demonstrated in simulation and experiments later. Then I i ðrÞ is replaced with I i ðr þ r i Þ to participate in the FPM recovery procedure. The flowchart of the dither removing algorithm is summarized in Fig. 2.

Dither Calibration
The dither removing algorithm we proposed does not consider the motion blur problem. To theoretically analyze the feasibility of the dither removing algorithm without considering the motion blur, we calibrated the dither of our experimental setup. Then we gave a sufficient condition that does not need to consider the motion blur.
The method that we calibrate the dither is described as follows. First, we start to capture images of the sample in a subregion of 352 × 360 pixels in static state at a framerate of 200 fps. Immediately we rotate the stage by an angle. We keep capturing images for about 30 s. When the image acquisition process is finished, we use the first image as the standard and calculate the shifted values for the remaining images using phase correlation method. 33,34 We calibrated the dither for three types of movements: (1) ðθ; φÞ from (1 deg, 0) to (0, 0); (2) ðθ; φÞ from (0, 1 deg) to (0, 0); and (3). ðθ; φÞ from (90 deg, 1 deg) to (90 deg, 0). The angular speed of the rotation stage is set to be 8 deg ∕s. The calibration results are plotted in Fig. 3. Figure 3(a1) shows the dither of the first type of movements in x direction over time. Figure 3(a2) shows the frequency spectrum of Fig. 3(a1). Figures 3(a3) and 3(a4) have similar meanings to Figs. 3(a1) and 3(a2) but in y direction. Figures 3(b1)-3(b4) and 3(c1)-3(c4) are similar to Figs. 3(a1)-3(a4) but belong to the second and third types of movements. Comparing the results of the three types of movements, we can conclude that the dither is more severe when rotating the stage along the vertical axis than along the horizontal axis. Comparing the results in x and y directions, we can conclude that the dither is more severe in y direction than in x direction. This conclusion can be also obtained by comparing the results of the last two types of movements since the dither is a little more severe when θ is 90 deg than 0. Results in Fig. 3  show that the dominant frequency when rotating the stage along the vertical axis is about 20 Hz and the dominant frequency when rotating the stage along the horizontal axis is about 10 Hz.
It is reasonable to use simple harmonic vibration model to represent the dither in a short time duration (T to T þ ΔT). We represent the dither D as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 6 ; 1 1 6 ; 6 8 7 DðtÞ ¼ N T sinð2πftÞ; T ≤ t ≤ T þ ΔT; (6) where N T is the amplitude in pixel around time T and f is the dominant frequency. The maximum velocity can be expressed as 2πfN T . When we capture an image, we think the motion blur can be neglected if the image travels less than 1 pixel. If the exposure time is Δt, we give a sufficient condition for not considering the motion blur by Eq. (7), E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 7 ; 1 1 6 ; 6 0 9 2πfN T Δt < 1: For rotating the stage along the horizontal axis, most N T is no more than 1 pixel, f is about 10 Hz, we can deduce that if Δt ≤ 16 ms, we can neglect the motion blur. For rotating the stage along the vertical axis, the case will be worse. f is about 20 Hz. If N T is 20, Δt should be less than 0.4 ms, and if N T is 5, Δt should be less than 1.6 ms.
In practice, we rotate the stage along the vertical axis to a specified θ, and then rotate the stage along the horizontal axis to all φs we needed. So the number of rotations along vertical axis is much less than that along horizontal axis. When changing θ, we wait for 3 s before capturing images to reduce the N T to about 5. The exposure time of bright-field images is set to be 0.6 ms. We increase the exposure time as φ goes up until the exposure time reaches 19.2 ms. Then we increase the gain instead if φ continues to go up. (But we cannot increase the gain too much since it will bring in noise.) In this way, we can ensure that most of the images are free of motion blur, and images that could be possibly polluted with motion blur correspond to the largest angles. This strategy works well in our experiments.

Simulation
We first demonstrate the effectiveness of the dither removing algorithm in simulation. We generate a dither-polluted simulation dataset by artificially adding a random shifted value to each simulated image. Concretely, in the process of simulating image acquisition, we artificially add the influence of dither for the i'th image by shifting the image with ðΔx i ; Δy i Þ pixels, where Δx i and Δy i are independent discrete random variables and have the identical probability distribution: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 8 ; 1 1 6 ; 2 8 2 where p > 0 is a preset positive integer that indicates the maximum shifted pixels due to dither. We simulate the situation where p is 1, 3, and 5. We compare the reconstruction results of the dither removing algorithm and the traditional FPM algorithm. The reconstruction results of simulation are summarized in Fig. 4. Figure 4(a1) shows the true image intensity of the simulated object, Fig. 4(a2) shows the raw image under normal illumination, and Fig. 4(a3) shows the reconstruction intensity of the traditional FPM algorithm applied to the dither-free dataset. Group (b) and (c) in Fig. 4 are the results reconstructed from the dataset polluted by dither. Figures 4(b1)-4(b3) show the intensities recovered by the traditional FPM algorithm whose maximum shifted pixels are 1, 3, and 5 while Figs. 4(c1)-4(c3) show the intensities recovered by the dither removing algorithm whose maximum shifted pixels are 1, 3, and 5. Results of group (b) in Fig. 4 show that dither can severely degrade the reconstruction quality. Results of group (c) in Fig. 4 show that the dither removing algorithm can effectively remove the influence of dither and successfully recover the image.
It is worth noting that the reconstructed image is allowed to have an uncertain global shifted value. Suppose HðrÞ is the reconstructed image through the dither removing algorithm and fr i ji ¼ 1;2; : : : ; ng is the shifted value set of the image set, n is the total number of raw images. It is easy to understand that Hðr þ r 0 Þ could also be a reconstructed image with the shifted value set fr i þ r 0 ji ¼ 1;2; : : : ; ng, where the Euclidean norm of r 0 is much smaller than the range of the image. In practice, when the high-resolution image HðrÞ is reconstructed, we shift the image by r, i.e., Hðr − rÞ, where r ¼ P n i¼1 r i ∕n. And the corresponding shifted value set would be fr i − rji ¼ 1; 2; : : : ; ng, which is zero mean. So Hðr − rÞ would not shift too far from the true image since the dither value is also zero mean.
We also examine the recovered shifted values with the case p is 5. The result is plotted in Fig. 4(d). The blue circles represent the true shifted values while the red asterisks represent the recovered shifted values. The black crosses represent the recovered errors. The image with index 1 is corresponding to the spectrum center. The position in spectrum goes outward in a spiral way with an increasing image index. Figure 4(d) shows that in this case, all images retrieved the correct shifted values. Figure 4(d) demonstrates that the recovered shifted value by dither removing algorithm is quite reliable.
The shifted values artificially added to the image sequence by Eq. (8) are all integers. In practice, the shifted values cannot be exactly integers. To examine the recovery quality in non-integer cases, we simulate the non-integer dither by interpolation. First, we upsample all the captured images by 10 times. Then we add the influence of dither as before, where p is 50. Finally, we downsample all images to their original size. In thus constructed images, the precision of shifted values is 0.1 pixel and the maximum shifted value is 5. Figures 4(e1) and 4(e2) show the reconstructed intensities without and with the dither removing algorithm. Results demonstrate that the dither removing algorithm is also effective in non-integer cases.

Experiments
To verify the feasibility of the illuminator and the effectiveness of the dither removing algorithm in realistic imaging, we conduct an experiment on the imaging platform using the dither removing algorithm. In this experiment, we adjust the NA of the lens to the maximum value of 0.045. We use a non-uniform sampling pattern for better sampling efficiency and image quality. 13 We set Δφ ¼ Δφ 1 when φ ≤ φ threshold and Δφ ¼ Δφ 2 when φ threshold ≤ φ ≤ φ max (the value of Δφ should meet the overlapping condition). 35 We adopt a simple strategy to design the sampling pattern:φ 0 ¼ 0, for φ n ¼ φ n−1 þ Δφ ≤ φ max , we set θ to be evenly distributed on the circumference. When n is 0, we only sample one angle where θ is 0. When n is 1, we sample four angles where θ is 0 deg, 90 deg, 180 deg, and 270 deg, respectively. Otherwise, we temporarily set the number of θ to be the same as in the case of n − 1. If the adjacent subareas of the spectrum do not overlap with each other, we double the number of θ. Here, we set Δφ 1 ¼ 1 deg, The total number of sampling angles is 45. The maximum exposure time is 5.4 ms with gain of 0.
The sample in this experiment is a USAF resolution target. We compare the reconstruction results of the dither removing algorithm and the traditional FPM algorithm. The experimental results are summarized in Fig. 5.  Fig. 5(d). The system NA is 0.045 (the red circle) and the theoretical synthetic NA is 0.1842 (the big dashed black circle), and the effective resolution gain is about 4-fold. This experiment verifies the feasibility of our illuminator and the effectiveness of the dither removing algorithm. The application of non-uniform sampling pattern also shows the incident angle selection flexibility of our illuminator.
Then we apply the imaging platform to an experiment with very low system NA to show the flexibility and wide applicability of the illuminator. In this experiment, we adopt the same strategy with the first experiment to design the sampling pattern. We set NA to be 0.01, The total number of sampling angles is 457. The maximum exposure time is 19.2 ms with gain of 12.11. The experimental result is shown in Fig. 6. Figure 6(a) shows the raw image, Fig. 6(b) shows the reconstructed intensity, and Fig. 6(c) shows the zoomed-in subarea of the red dashed square in Fig. 6(b). Figure 6(d) shows the reconstructed spectrum. The sampling pattern is portrayed in Fig. 6(e). The system NA is 0.01 (the red circle) and the theoretical synthetic NA is 0.1353 (the big dashed black circle), and the effective resolution gain is about 13-fold. Figure 6 shows that our imaging platform is also applicable to a very low-NA objective lens. It demonstrated that our illuminator is quite flexible and widely applicable.  To demonstrate that the illuminator is also feasible for a real biological specimen, we apply the imaging platform to well-differentiated squamous cell carcinoma. We use the same parameters and sampling pattern as the first experiment. Figure 7 shows the experimental results. Figure 7(a) shows the FOV image. Figures 7(b1)-7(d1) show the zoomed-in subareas of different squares in Fig. 7(a). Figures 7(b2)-7(d2) and 7(b3)-7(d3) show the reconstructed intensities and phases of (b1)-(d1), respectively. Results demonstrated that our platform is also applicable for a real biological specimen.

Conclusion and Discussion
In conclusion, we have presented a quite flexible illuminator for FPM based on a two-axis motorized rotation stage. It is capable of providing coherent illumination in any direction; therefore, the sampling pattern in FPM can be freely designed and changed. This illuminator provides a quite convenient way for investigating FPM in a deeper level and a wider range. To prevent tiny dither introduced by mechanical movements from degrading the reconstruction quality, we proposed a dither removing algorithm for FPM. The dither removing algorithm is able to retrieve a high-resolution image successfully from a sequence of low-resolution images polluted by dither. Simulation and experiment have verified the feasibility of our illuminator as well as the effectiveness of our dither removing algorithm. We demonstrated the high flexibility of our illuminator by applying it in FPM experiments with a 0.045 NA lens as well as a 0.01 NA lens and achieved a resolution gain of 4-and 13-fold, respectively.
A large FOV is very appealing in pathological section. However, a large FOV usually means a low NA, which requires illumination with a much denser angular distribution. Our illuminator can be integrated into a low-NA FPM platform conveniently. Our illuminator is quite suitable for investigating extremely high-resolution gain FPM, such as from 0.04 to 0.8 or more.
We use a laser as the illumination source, which is sensitive to any optical imperfection. It will bring fluctuating background signals and speckles in the captured images, which could be seen in Fig. 7(a). These fluctuations could then contribute negatively to the reconstructed phase. For a thin sample (with small phase), one can use the differential phase contrast deconvolution in the FP algorithm to further improve the quality of reconstructed phase. 9 We will use a more appropriate illumination source to further improve the reconstructed complex image in the future.
In many other cases, the motion blur problem cannot be neglected. We will investigate the motion blur problem and improve the dither removing algorithm in the future.
Due to the mechanical scanning, the slow acquisition speed is a disadvantage of our illuminator. So it is not suitable for the cases that have high requirement for time.
We believe that our illuminator and the dither removing algorithm will help the application and theory investigation for FPM.

Disclosures
The authors have no relevant financial interests in this paper and no potential conflicts of interest.