Optimization of structured illumination microscopy with designing and rotating a grid pattern using a spatial light modulator

Abstract. Our structured illumination microscopy (SIM) is based on a spatial light modulator (SLM) instead of an illumination mask, which does not need to be attached to a linear stage. This SIM can easily design the period of the one-dimensional grid related to the optical sectioning strength and can rapidly acquire three-dimensional data. The optimization of SIM with an SLM is proposed. Previous studies primarily varied magnification with a high numerical aperture objective to optimize the axial response. It is feasible to obtain the maximum optical sectioning strength by designing a grid pattern that has an appropriately high spatial frequency and to uniformly cover the entire frequency spectrum of the sample by rotating a grid pattern. We have successfully optimized SIM with such a grid and covered the frequency spectrum by rotating a grid pattern in multiple orientations.


Introduction
In the field of electronics and advanced materials, there is a growing demand for information on microstructures or surface morphology because of recent advances in the technology of fine particles. Recently, the structured illumination microscopy (SIM) has been developed for the threedimensional (3-D) measurements of samples with optical sectioning, which decreases blurring and increases the axial resolution. 1 The SIM performance can be further improved by increasing the axial response 2,3 and covering the entire frequency spectrum of a sample. 4 In previous studies, the axial response was increased by varying the magnification with a high numerical aperture objective between a grid plane and a sample plane, or the spatial frequency was changed in a grid-patterned mask. 2,3 This approach has a disadvantage in that it consumes cost and time because the optical system configuration must be modified or a high spatial grid-patterned mask suitable for the imaging system must be manufactured.
In this paper, we suggest an optimization of SIM by designing an appropriately high spatial frequency of the grid pattern in a spatial light modulator (SLM) to increase the axial response and rotating the grid pattern to uniformly cover the entire frequency spectrum of the sample without modifying the optical system. To investigate the correlation of the spatial frequency with the optical sectioning strength, we measured the axial response by changing the spatial frequency of the grid pattern in an SLM. 3 From the result, the grid pattern with an appropriately high spatial frequency was selected among various spatial frequencies of the grid patterns for the maximum optical sectioning strength. In addition, we rotated the grid pattern in multiple orientations to effectively extend the optical transfer function (OTF) in an SLM, thereby uniformly covering the entire frequency spectrum of the sample. Moreover, an SLM is expected to be broadly used for the application of SIM because it can easily design the spatial period, and the 3-D data acquisition speed is fast unlike a grid-patterned mask. [5][6][7][8][9] The remainder of this paper is organized as follows. Section 2 theoretically analyzes the effective OTF shifted by the period of the grid pattern with a single spatial frequency. In addition, we discuss the extended frequency spectrum of the sample after assembling the OTF with multiple orientations. Section 3 describes the experimental setup and the results. Section 4 concludes the paper.

Structured Illumination Microscope System
A Kohler illumination-based optical system was configured to operate SIM. The warm white LED was projected onto the SLM containing the grid pattern. Light reflected from the SLM was reflected by a beam splitter and transmitted through the band-pass filter to improve the grid pattern contrast. 3,11 The filtered light was reflected into the objective by a beam splitter to form the grid pattern image of the SLM onto the sample. Optical sectioning was performed by the phase shift of a sinusoidal wave in the grid pattern, and the phase shift was electrically implemented in the SIM based on the SLM. The intensity of the samples is modulated by the square-law detection algorithm: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 1 ; 3 2 6 ; 1 2 2 from the three phase-shifted images during optical sectioning. 2,12 While the samples loaded on the linear stage were axially scanned with the phase shift, the axial intensity was obtained by optical sectioning performed by the square law detection. The focal axial position was determined by the peak axial intensity in each pixel of the camera. The 3-D height map can be obtained by calculating the focal axial position in the Cartesian coordinate system with respect to all the pixels of the camera; this provides the height information of the sample. The performance of the SIM can be improved by increasing the axial response. Hence, as the full width at half maximum (FWHM) of the axial response is narrower, the SIM provides more precise height information.
For SLM-based SIM, the SIM can be optimized by the maximum optical sectioning strength at a normalized spatial frequency (ṽ) close to unity in Eq. (2): 2,3 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 6 3 ; 5 8 7 DðuÞ∼ where u andṽ represent the normalized defocus and spatial frequency of the pattern, respectively, described by Neil et al. 2 It is important to take the appropriate spatial frequency because higher spatial frequency results in reducing the contrast of the grid pattern, 10 thereby weakening the optical sectioning strength. Therefore, the grid pattern with an optimal period with the normalized spatial frequency of unity must be formed, which improves the axial response.

Analysis of the Frequency Spectrum
For an incoherent illumination system, the detected image intensity is the result of the convolution of the light intensity (I s ) from the sample and the point spread function (PSF) (h) of the imaging system: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 3 ; 6 3 ; 3 6 4 DðrÞ ¼ ðI s ⊗ hÞðrÞ; where r represents the position vector on the camera. The light intensity (I s ) is related to the incident light intensity (I in ) and the sample density (S) through I in ¼ I s S. For simplicity, the frequency spectrum detected on the camera is written as follows: where Eq. (4) gives the detected image (DðkÞ) with the frequency spectrum of the sample measured by a conventional microscope. Only the frequency spectrum of the sample passing through the cutoff frequency (k c ) is measured and the image information of the sample from the remaining part of the spectrum is lost. Thus, on the conventional microscope, both sharp and faint images are seen simultaneously. However, the high-frequency components of the sample pass through the cutoff frequency and the low-frequency components are eliminated in the SIM so that only the focused part can be observed. The image information of the sample in the SIM is analyzed as follows.
When the grid pattern is projected uniformly onto the sample, as illustrated in Fig. 1

(a):
E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 5 ; 6 3 ; 9 2I where I 0 and m represent the intensity of a conventional wide-field image and the grid pattern contrast and ϕ is an arbitrary spatial phase. It is then expressed with the frequency spectrum, as illustrated in Fig. 1

(b):
E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 6 ; 3 2 6 ; 7 0 8Ĩ where δðkÞ denotes the Dirac delta function that is formed in Fourier transform of plane wave illumination (I in ¼ 1). The frequency spectrum of the image captured by the camera is given by E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 7 ; 3 2 6 ; 6 1 9D The first term is the detected image of the frequency spectrum cutoff by low-pass filtering and has no high-frequency spectra, and the second and third terms are the detected images of the high-frequency spectra, which represent small structures in the sample. These high-frequency spectra are effectively shifted by a single spatial frequency in the grid pattern and are passed through the cutoff frequency of the imaging system, and they overlap the low spatial frequency spectra near the DC frequency, permitting small structures in the sample to be observed, as illustrated in Fig. 1(c). These overlapped spectra can be extracted by optical sectioning, as shown in Fig. 1(d). After optical sectioning, the cutoff frequency of the reconstructed frequency spectrum is extended by the spatial frequency (k 0 ). Therefore, the cutoff frequency is higher than that of a conventional optical microscope, and the high-frequency spectra of the sample can be observed. In particular, optimal sectioning occurs when the spatial frequency (k 0 ) is half of the cutoff frequency (k c ) of the OTF. 2,13 To cover the entire frequency spectrum of the sample, the OTF must be effectively extended in multiple orientations by rotating the grid pattern. A description of the raw image, with a frequency spectrum of the sample by the OTF extended to the entire area, can be written as follows: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 8 ; 3 2 6 ; 3 0 8D The rotation shows different areas in the frequency spectrum of the sample that can provide more accurate height information of the sample and reduce the background noise. However, high-frequency spectra overlap with the lowfrequency spectra, which degrades the raw image quality before optical sectioning. Therefore, these components must be extracted and reconstructed for observing small structures in the sample. To separate them, optical sectioning is performed by the square law detection from three phase-shifted images, which correspond to the relative spatial phases of ϕ 0;1;2;3 ¼ 0; ϕ 0;1;2;3 ¼ 2π∕3; ϕ 0;1;2;3 ¼ −2π∕3 After optical sections in multiple orientations, the reconstructed frequency spectrum of the sample extended to other frequency spectrum regions, as depicted in Fig. 1(e). Four optical sections corresponding to four different orientations of the grid pattern are required at any one longitudinal position to measure the entire frequency spectrum of the sample. After optical sectioning, the components in which the high-frequency spectrum overlaps the low-frequency spectrum are extracted, and the low-frequency components are eliminated. As a result, we can obtain the sectioned image with the entire frequency spectrum of the sample. It is expected to provide more accurate height information and reduce background noise.

Experimental Setup
An optical imaging system was constructed to operate the SIM, as shown in Fig. 2. The warm white LED (Thorlabs, MWWHLP1) was used as the light source; the grid pattern loaded on the SLM was projected onto the sample through the objective lens (Mitutoyo, 20X Plan Apo Infinity Corrected Long WD Objective); and the final image with the grid pattern was formed on the camera (The Imaging Source, DMK 33GX174e). The image of the sample was clearly formed on the camera when the grid pattern was matched on the sample. The phase shift was implemented by electrically shifting a sinusoidal wave pattern by one third of the spatial period of the pattern in the SLM, which has a spatial period of 0.83 μm and a pixel pitch of 0.003 μm along with the objective lens used in the experiment. Optical sectioning was performed by square-law detection using three phase-shifted images with zero spatial phase. Han et al.: Optimization of structured illumination microscopy with designing and rotating a grid pattern using a spatial light modulator As the sample was axially scanned, we took optically sectioned images for all axial positions and extracted the height information of the sample at the focused axial position. Finally, a 3-D height map was reconstructed based on these sectioned images.

Designing the Grid Pattern in the SLM
As the spacing of the grid pattern becomes narrower, the normalized spatial frequency becomes close to unity. Hence, the optical sectioning strength becomes stronger than that of the low spatial frequency of the grid pattern designed for the SLM. In the experiment, we designed grid patterns with different spatial frequencies (v) that were 100 line pairs/mm, 64 line pairs/mm, and 50 line pairs/mm, and the normalized spatial frequencies were 1.16, 0.75, and 0.58, respectively, by calculatingṽ ¼ βλv∕NA, where β and λ denote the magnification of the imaging system and the wavelength, respectively, v is the spatial frequency of the grid pattern and NA is the numerical aperture. Raw images in grid patterns with different spatial frequencies were formed, as shown in Fig. 3. Of course, all the methods presented in this paper are based on the invariant PSF. In a noninvariant PSF, the axial extent of the PSF increases as the focal position becomes deeper along the Z-axis, which causes aberration. 14,15 However, the objective lens of our system was well calibrated, and aberrations did not occur because each relay lens was well placed at the focal distance. Figure 3 shows that there was no distortion or blurring at the edge of the grid pattern, which indicates that aberration did not occur. The axial responses between each system were measured under the condition that the system was stable. Axial responses were measured for the grid patterns with different spatial frequencies, as shown in Fig. 4, to investigate the correlation of the spatial frequency with the optical sectioning strength. We introduce the FWHM to facilitate the comparison of these axis responses. The FWHM refers to actual defocus at an axis response of 0.5. In general, normalized spatial frequency and normalized defocus are inversely proportional, so the higher the normalized spatial frequency, the lower the normalized defocus. As the normalized defocus u is related to the actual defocus z through u ¼ 8ðπ∕λÞz sin 2 ðα∕2Þ, we can estimate the actual defocus by obtaining the normalized defocus according to the normalized spatial frequency, and comparing actual defocuses can evaluate each axis response. 2 Therefore, it can be confirmed from the aforementioned equation that the actual defocus changes by the normalized spatial frequency, which  affects the axial response. The FWHMs of axial responses at each normalized spatial frequency were measured to facilitate comparison of axial responses. In practice, we measured 15.8, 18.8, and 20 μm for the FWHMs of axial responses, which correspond to grid patterns with spatial periods of 10, 15.63, and 20 μm, respectively. The maximum optical sectioning strength was shown at the grid pattern with the spatial period of 10 μm in which the normalized spatial frequency is close to unity. As the FWHM narrowed, there was an improvement in the axial response, and the FWHM was minimized when the normalized spatial frequency was approaching unity, as shown in Fig. 4. Consequently, the FWHM continuously became narrower as the normalized spatial frequency approached unity, leading to improved axial response. For the grid pattern with the narrowest FWHM of the axial response, SIM provides high precision when extracting the height information of the sample to be observed.

Reconstructing the Sectioning Image
We performed SIM experiments with a coin as the sample to verify the effects of different axial responses on SIM performance. Figure 5 shows the sectioning image, which is reconstructed by a square-law detection algorithm from the phase shift images. First, we discuss the sectioned image in the outof-focus region. In this region, the sectioned image, which is obtained by Eq. (1), should have zero intensity for all image pixels, provided that the object is ideally out-of-focus. However, as shown in Figs. 5(b) and 5(c), the sectioned images of the object are constructed even in the out-of-focus area. This means that the image is measured as if there are samples even in the out-of-focus image, causing an error in the 3-D height map information. In addition, the shorter period of the grid pattern of the SLM leads to the smaller intensity of the sectioned image relative to the larger period of the grid pattern in out-of-focus. This is because the depthof-field becomes smaller as the period becomes smaller. It is also consistent with the narrower FWHM discussed in Sec. 3.2. This means that the imaging system has a high dynamic range in a short spatial period of the grid pattern and can observe the sample well in the focal plane. Consequently, it is expected that the smaller period of the grid pattern provides more accurate height map information. Second, we discuss the sectioned image in the focused area. In this area, as shown in Figs. 5(d)-5(f), the shorter period results in significantly clearer surface information in the sectioned image. As the period of the grid was sufficiently shortened, the sectioning strength was improved, which is described in Sec. 2. Consequently, it is expected that the sufficient small period of the grid pattern provides more accurate height map information (Fig. 6).
We also rotated the grid pattern in multiple orientations and merged each sectioned image to obtain the sectioned image of the fully extended OTF in all areas. This showed that the OTF was shifted in the k x or k y direction when optical sectioning was performed in only one orientation, whereas the OTF was effectively expanded in the k x and k y directions when optical sectioning was performed in multiple orientations. It was demonstrated that the fully extended frequency spectra of the sample were observed in the raw image with the rotated grid pattern. From the results, the axial resolution was increased in the grid pattern with the normalized spatial frequency of unity compared with other spatial frequencies (Fig. 7).

Reconstructing the Height Map
The 3-D height maps were reconstructed for the plane mirror between each system. The axial scanning was performed with 1-μm step and total scan range was 100 μm. A better optical sectioning occurred in the grid pattern with the spatial period of 10 μm, where the normalized spatial frequency is close to unity. It was confirmed that the height map reconstructed by the grid pattern with the spatial period of 10 μm had more accurate height information compared with height maps reconstructed by grid patterns with spatial periods of 15.625 and 20 μm.
To examine the roughness of the plane mirror, standard deviations were calculated for the reconstructed 3-D height maps. The standard deviation values were 0.25, 0.37, and 0.39 μm, which correspond to grid patterns with spatial periods of 10, 15.625, and 20 μm, respectively. This showed that the height map reconstructed by the grid pattern with the normalized spatial frequency of unity yielded a smaller standard deviation compared with height maps reconstructed by grid patterns with different spatial frequencies. These results suggest that SIM can provide a high precision measurement for grid patterns with normalized spatial frequency of unity (Fig. 8).
We also measured the height of the coin for a region of interest (ROI) of size 100 × 100 μm 2 in the 3-D height map. To obtain the objective data, we set the value measured with the confocal microscope (PRECITEC, CHRocodile C) as the reference value. Each optical sectioning was evaluated by comparing the reference value to the value measured by grid patterns with different spatial frequencies. The axial scanning was performed with 0.7-μm step and a scan range of 700 μm. The height was computed by subtracting the average height of the two faces of the ROI. The height of the coin in the ROI was measured as 29.44, 30.96, and 30.8 μm, which correspond to grid patterns with spatial periods of 10, 15.625, and 20 μm, respectively. The height of the same ROI was also measured as 30 μm using the confocal microscope. The SIM using the grid pattern with an appropriately Fig. 6 Frequency spectra of the grid in the (a) k x direction and the (b) k y direction by effectively extended OTF.  high spatial frequency provided high accuracy, considering that the error in the measurement of the confocal microscope is 5%.
For optical sectioning performed in multiple orientations, the noise was dramatically reduced in the height map, as shown in Fig. 9. The grid pattern was rotated by 45 deg. At least three rotations (four grid orientations) allowed for the measurement of the entire frequency spectrum of the sample. The frequency spectrum of the sample was measured by an effectively extended OTF. The more the area occupied by the OTF, the more uniformly the frequency spectrum of the sample can be measured. Therefore, if the rotation angle of the grid pattern is reduced and rotated several times, more frequency spectrum areas of the sample can be uniformly measured. Of course, measuring multiple rotations increases the integration time of optical sectioning images, which is disadvantageous for the measurement speed. The number of grid orientations is equal to 180 divided by the angular step size. Considering the measurement speed, we adopted an angular step size of 45 deg. In other words, the grid pattern was rotated three times, and the number of grid orientations was 4. This provided adequate measurement speed and uniform frequency spectrum measurement. It is shown in Fig. 9(b) that noise was reduced in the 3-D height map. Therefore, we successfully optimized the SIM by designing a proper grid pattern and performing optical sectioning in multiple orientations in the SLM. The accuracy obtained from the measurement is comparable to that of a confocal microscope.

Conclusion
We demonstrated that the optimization of SIM was achieved by designing an appropriately high spatial frequency and rotating a grid pattern using an SLM. To examine the correlation of the spatial frequency with the optical sectioning strength, the axial response was measured between each system. It was confirmed that the optimal sectioning occurred at the grid pattern in the SLM having a normalized spatial frequency close to unity. Furthermore, we could uniformly measure the high-frequency spectra of the sample with the effectively extended OTF by rotating the grid pattern. The frequency spectrum area to be measured in the sample was determined by the angular step size and the grid orientations, and the optimum angular step size was determined considering the measurement speed. Consequently, it was confirmed that noise in the 3-D height map was remarkably reduced. In our optical system, we successfully optimized SIM using an SLM without the modification of the optical system and changing a grid-patterned mask.