2.1.

Phase Contrast Reconstructions

We consider a planar approximation of an object located in a transverse plane $z=0$. A normalized monochromatic scalar field propagating in free space along the $z$ axis and incident on that object creates a complex optical field $U_z(\mathbf{r})$ that can be described in the in-focus plane by Eq. (1), where $A_0$ is the amplitude and ${\phi }_0$ is the phase distribution in this plane:

Let $\mathbf{r}$ be the coordinates in the transverse plane; $z$ be the coordinate in the propagation direction, with a reference $z=0$ for the object plane; and $R$ be the distance in space away from the object.

If we consider the diffraction that arises from illuminating the object with a semicoherent plane light field, we can write the wave’s complex amplitude in any transverse plane $z\ge 0$ in terms of the wave amplitude in the prior plane $z=0$:

Sensors are only sensitive to the irradiance, i.e., the square of the modulus of the complex field, as stated by Eq. (4). Therefore, the phase information is lost in the acquisition process:

The complex field in the object plane $U_0(\mathbf{r})$ can be numerically reconstructed from an acquisition of irradiance in a single out-of-focus plane $I_z(\mathbf{r})$. This in-line diffraction pattern is often referred to as a hologram.

Numerous reconstruction methods are described in the literature for the reconstruction of phase maps from holograms. Here, we used a state-of-the art iterative reconstruction algorithm from a single image, which is described in Ref. 31. To find the phase in the sensor plane ${\phi }_z$, the $L_1$-norm ${\Vert g\mathrm{rad}(h_{-z}*\sqrt{I_z}{e}^{i{\phi }_z})\Vert }_1$ is minimized using gradient descent. Such a scheme forces a fit to the acquired data $I_z$. Once a value is found for the phase in the sensor plane ${\phi }_z$, the complex field in the object plane $A_0·e^{\mathrm{i}{\phi }_0}$ can be retrieved by computing $h_{-z}*\sqrt{I_z(\mathbf{r})}{e}^{i{\phi }_z}$. The phase map is reconstructed from a single in-line hologram. Absolute values can only be obtained at the cost of a priori information or constraints on the object associated with a calibration relative to a quantitative reference method. Therefore, the phase contrast is not considered quantitative in our configuration. Other approaches, in particular off-axis configurations combined with phase unwrapping algorithms,³² multiwavelength illumination,³³ or multiheight acquisitions,³⁴ allow the recovery of quantitative phase maps. However, they require acquiring multiple images, whereas our holographic approach is a single shot.

2.2.

Instrument Development

The combination of phase and fluorescence requires the images to be registered. Either hardware or software registrations may be considered. Numerical registration between two modalities has been intensely studied but remains a challenge. Therefore, it is advantageous for instrumentation to enable the acquisition of already-registered images. In this paper, we implemented a common-path configuration with a single sensor. For this reason, multimodal images did not require post-treatment registration. With the mesoscope system, as illustrated in Fig. 1, two images could be sequentially acquired: one out-of-focus transmission image that is processed to obtain phase and amplitude maps and one in-focus fluorescence image. In addition, the system could be used for absorption-contrast brightfield imaging.

Fig. 1

Schematic representation of the mesoscope bimodal imaging system.

Performing statistical imaging of micrometric biological objects requires a large FOV and a micrometric resolution. In microscopy, a compromise must be found between the two. Large FOVs result from low-magnification objectives, whereas high resolution is associated with high-magnification objectives. Both factors are linked to the numerical aperture. We aimed to develop a $1\times$ magnification system to enable larger FOV imaging, and thus optimize the number of objects that could be imaged in a single shot. To achieve this goal, we sought a $1\times$ objective with a high numerical aperture and a wide FOV. Air microscopy objectives typically have a limited numerical aperture at low magnifications and a FOV of around 25 mm in diameter. We chose a cost-effective SLR camera lens from the macrophotography series of Canon Inc. (EF 100 mm $f/2.8$ Macro USM, Canon Inc.). SLR camera lenses are designed to be used without tube lenses.

This objective has a front lens of 58 mm in diameter, enabling it to achieve a numerical aperture of $0.113\pm 0.003$. To experimentally measure this value, we collimated the light from a green light-emitting diode (LED) (XLamp^® XM-L™ Color LED, Cree Inc.) with a large aspheric condenser lens (ACL756U-A, Thorlabs Inc.). This was mounted on a goniometer stage and the resulting 75-mm-diameter beam could be tilted away from a central position, which is defined as the normal incidence on the objective. The precision of the stage was 0.1 deg. The numerical aperture was defined in regard to the cut-off angles ${\theta }_{\max }$ and ${\theta }_{\min }$ of the resulting curve: $\mathrm{NA}=\sin (\frac{{\theta }_{\max }-{\theta }_{\min }}{2})$, as illustrated in Fig. 2. The uncertainty on this value is the error induced on the cut-off angles by a 0.1-deg error on the inclination angle. This numerical aperture corresponds to a theoretical depth of field³⁰ of $20.4\mu \mathrm{m}$. The thickness of the samples we studied was typically below this value, making the planar approximation a decent model for these objects.

Fig. 2

Experimental evaluation of the effective numerical aperture of the system. The crosses show the experimental data points. The continuous line is a linear approximation of the experimental data. The black box points out the numerical aperture.

The compromise between resolution and FOV can be measured by the space–bandwidth product (SBP), i.e., the number of pixels required to capture the full area at full resolution. $\mathrm{SBP}=\frac{\mathrm{FOV}}{p{s}^2}$, where FOV is the circular FOV at the image plane disregarding the sensor used and $ps$ is the pixel size required to achieve the Nyquist sampling, i.e., half of the resolution defined by the Airy disk radius.²⁹ This is a reasonable method to quantify the amount of information transmitted by the optical system.³⁵

Figure 3 illustrates the limited SBP of standard Zeiss microscopy objectives and the improvements that can be made using larger optics²³ or multiple angle illumination.³⁶ Although the SBP of SLR camera lenses differ from those of dedicated microscopy objectives by two orders of magnitude, only a few research groups have reported the use of such lenses for microscopy.

Fig. 3

Comparison of the SBP plotted against magnification. For our system, we show both the available SBP and the SBP that is effectively used. The other systems are Zeiss microscopy objectives (Carl Zeiss, Germany), Fourier Ptychography microscope,³⁶ Zygo objectives (Zygo Corp.),³⁷ and custom-developed Mesolens.²³

One difficulty lies in acquiring the total available SBP in a single shot. Photography objectives are developed for full format $24\text{-}\times 36\text{-}{\mathrm{mm}}^2$ sensors. The available SBP is spread over a disk of 43 mm in diameter in the image plane. Smaller size sensors apply a crop factor. As a general trend, in the past decade, the development of sensor technology has been guided by high-resolution mobile-phone imaging, i.e., pixel size tends to shrink and chip size remains rather small. Therefore, in our system, a trade-off had to be made between chip size and pixel size to optimize the effective SBP.

According to the Shannon–Nyquist criterion, sampling the point spread function (PSF) in a $1\times$ system requires pixels half the size of the PSF. Theoretically, a numerical aperture of 0.113 provides a PSF with a radius of $r=\frac{0.61\lambda }{\mathrm{NA}}\sim 2.81\mu \mathrm{m}$ at 520 nm. Therefore, we selected a monochromatic complementary metal-oxide-semiconductor (CMOS) 11 mega pixels $6.4\times 4.6{\mathrm{mm}}^2$ sensor with a $1.67\text{-}\mu \mathrm{m}$ pixel pitch (UI-1492LE-M, IDS GmbH, Germany) resulting in a $29.4\text{-}{\mathrm{mm}}^2$ FOV. This pixel size optimized the resolution, but a larger chip size would have improved the effective SBP, as pointed out in Fig. 3. However, to our knowledge, such sensors are not commercially available.

To perform phase imaging, we chose an in-line configuration for digital holography. A diffraction pattern is formed when the object is illuminated with a partially coherent light. It is recommended in Ref. 38 to consider both spatial and temporal coherences. A low degree of coherence reduces the speckle artifacts and unwanted fringes that result from reflections on multiple interfaces. A high degree of coherence, such as that of lasers, creates holograms with more spectral content. To mitigate both effects, we compromised by using a LED (XLamp^® XM-L™ Color LED, Cree Inc.) coupled into a $200\text{-}\mu \mathrm{m}$-core diameter multimode optical fiber (FG200UEA, Thorlabs Inc.).

To minimize the aberrations, we implemented a convergent illumination, as illustrated in Fig. 1. Collimated or divergent illuminations resulted in enhanced geometric aberrations because of the high incidence angles on the border of the objective and prevented accurate phase reconstruction. A singlet lens (LAT075, Thorlabs Inc.), referred to as $L_3$ in Fig. 1, was used to make the illumination convergent on the objective’s front lens. The distance from the optical fiber output to the $L_3$ lens was 143 mm, and that from $L_3$ to the objective ML was 191 mm. As is commonly done in holographic imaging,¹⁴ we considered the field curvature negligible in the object plane and made the plane wave approximation. This was required to apply the Fresnel formalism described previously.

The position of the object plane was imposed by the $2f–2f$ configuration that allowed achieving a 1:1 magnification ratio. The distance from the object plane to the objective front lens was 145 mm. Numerous methods exist to obtain out-of-focus images, typically introduction of a dephasing medium³⁹ or manual or mechanical translation of either the object, the objective, or the sensor. We implemented $S_2$, a manual micrometric $z$ axis translation stage (SM1Z, Thorlabs Inc.) with a 25-mm range to move the sensor. This procedure was preferred both because it avoided disturbing liquid samples and because it was mechanically easier than moving the objective. The optimal defocus distance is dependent on the object. We arbitrarily chose it in a range from 50 to $1000\mu \mathrm{m}$ to optimize the SNR of the diffraction fringes. As described, the in-line configuration allowed acquiring a diffraction pattern from which a phase-contrast image could be reconstructed.

To introduce fluorescence modality to this system, an excitation source and a high-pass emission filter must be added. The excitation source can be a spatially filtered laser diode (LD), as illustrated in Fig. 1. Alternatively, a monochromatic LED may be used. In addition, it was straightforward to implement multiple excitation sources, thus enabling sequential multiwavelength fluorescence. To avoid having excitation light directly incident on the sensor, the excitation module was implemented with an $\sim 45\text{-}\mathrm{deg}$ angle from the optical axis. This allowed for the lessening of the constraints on the emission filter; unlike standard epi-illumination microscopy, no additional dichroic filter was used. The distance between the sensor and the objective back lens was 3 cm, which provided enough space to insert the filter in the optical path. Because the brightfield illumination wavelength was chosen higher than the filter’s cutoff wavelength, holographic imaging could be performed without removing the fluorescence emission filter.

3.1.

Transmission

For brightfield imaging, resolution can be defined as the width of the bars in the last resolved group of the amplitude 1951 United States Air Force (USAF) resolution test chart. The uncertainty on the measurement was considered to be half the size difference between the resolved bar and the one from the next group on the target. In transmission, a group was considered resolved if the contrast of its bars, both horizontal and vertical, exceeded 10%. Contrast was defined from the maximum and minimum gray values in the considered line profile as $C=\frac{\max -\min }{\max +\min }$.

Under blue LED illumination (central wavelength 469 nm), we measured a resolution of $2.76\pm 0.15\mu \mathrm{m}$ in the center [Figs. 4(a)–4(b)] and $3.91\pm 0.23\mu \mathrm{m}$ on the edges of the FOV. The resolution in the center of the FOV was $3.10\pm 0.17\mu \mathrm{m}$ under green LED illumination (central wavelength 526 nm), i.e., resolution was degraded by the use of higher wavelengths.

Fig. 4

Determination of the resolution in brightfield when the amplitude 1951 USAF resolution test chart was placed in the center of the FOV and illuminated by a blue LED: (a) acquisition and (b) horizontal and vertical line profiles drawn in (a).

A phase 1951 USAF resolution test chart was used to determine the resolution of the phase reconstruction. A hologram [Fig. 5(a)] of this test chart was acquired by the system under blue-light illumination; in particular, it was blurred by the PSF and sampled by the pixels. The hologram was then reconstructed into a phase map [Fig. 5(b)]. We used the normalized values of the line profiles along a group of the target to determine the resolution [Fig. 5(c)]. The resolution and optimal acquisition distance are highly dependent on the object. We found that at a $910\text{-}\mu \mathrm{m}$ defocus distance, the reconstructed phase image had a $4.38\text{-}\pm 0.23\text{-}\mu \mathrm{m}$ lateral resolution. The resolution depends on several factors: the resolution of the acquired hologram, the reconstruction algorithm, and the object properties.

Fig. 5

Determination of the resolution of the phase reconstruction when the phase 1951 USAF resolution test chart was placed in the center of the FOV and illuminated by a blue LED: (a) acquired hologram, (b) phase reconstruction, and (c) normalized horizontal and vertical line profiles drawn in (b).

As the phase reconstruction was qualitative, the image could also be evaluated based on its SNR. The phase resolution test chart has been designed to have lines with a 183-nm optical path difference. For an 8-bit sensor, the standard deviation of a reconstructed zone with no signal was 1.6 levels of gray. The SNR of the reconstruction was measured to be 11.9 for the smallest resolved line. This validates that all resolved lines were clearly contrasted.

3.2.

Fluorescence Resolution Measurements

Fluorescence resolution was obtained by measuring the PSF of the system. A solution of $1\text{-}\mu \mathrm{m}$ green fluorescent protein (GFP) fluorescent beads was placed on a microscope glass slide, excited at 488 nm, and detected with a high-pass 488-nm emission filter (BLP01-488, Semrock Inc.). Line profiles of isolated beads were drawn, as illustrated in Fig. 6. For a given spatial position, the resolution was defined as the full width at half maximum (FWHM) of a normalized Gaussian curve fit on the experimental data points of the PSF. The variability of uncertainty of this value was obtained by considering five PSFs.

Fig. 6

PSF for a $1\mu \mathrm{m}$ fluorescent bead located in the center of the FOV. Crosses indicate experimental data points and the dashed line corresponds to the Gaussian fit. The black box indicates the FWHM.

Average resolutions of $3.72\pm 0.25\mu \mathrm{m}$ and $3.98\pm 0.10\mu \mathrm{m}$ were obtained in the center of the FOV and on the edges, respectively. The theoretical limit of the resolution, which results from diffraction,²⁹ is $r=\frac{0.61\lambda }{\mathrm{NA}}\sim 2.74\mu \mathrm{m}$ at 507 nm (i.e., at the central emission wavelength of GFP). This indicated that the system’s resolution was primarily limited by the optics.

The level of noise in a fluorescence image is crucial because it confuses the signal of interest. The occurrence of noise has thus been assessed and a postprocessing method has been chosen to reduce artifacts in the images. When exposure time was long, typically $>500\mathrm{ms}$, a median two-by-two filter was applied. To correct for the inhomogeneous excitation (Gaussian laser excitation profile), the image was normalized by a reference excitation profile. To enhance the image quality, the fluorescence image was then deconvolved by the experimental PSF using a Lucy–Richardson algorithm.

3.3.

Sensitivity Measurements

For applications in biology, it is critical to evaluate the imaging system’s sensitivity to fluorescence. The sensitivity of our system was positioned relative to flow cytometry (CyAn™ ADP, Beckman Coulter Inc.); the channel was fluorescein isothiocyanate (FITC) PMT 650 and the gain was set to 1. For calibration, we used a solution of 6.0- to $6.4\text{-}\mu \mathrm{m}$-diameter beads with six different fluorescence levels (Rainbow Calibration Particles No. RCP-60-5, Spherotech Inc.). These beads are typically used for fluorescence quality control of flow cytometers.

The flow cytometer was used with a 488-nm laser line coupled to a $530/40\mathrm{nm}$ bandpass filter. The laser output power of 20 mW was spread over an area $<0.015{\mathrm{mm}}^2$. The cytometry graph in Fig. 7(c) shows the number of detected beads as a function of the detected fluorescence intensity for the FITC fluorophore. The six groups of fluorescent beads were detected.

Fig. 7

Detection of fluorescent Sphero™ beads: (a) details of a raw acquisition by our system showing beads of three different fluorescence levels with an integration time of 1000 ms and a gain of 1, (b) histogram of the levels of fluorescence detected by our system with an integration time of 1000 ms and an analog gain of 8.52, and (c) histogram of the levels of fluorescence detected by the flow cytometer on FITC PMT 650 channel with a gain of 1.

Performances of our system were assessed in comparison to flow cytometry. This was done by integrating a 488-nm laser combined with the appropriate emission filter (FF01-531/40, Semrock Inc.) into our system. The laser had an output power of 25-mW spread over an area $>30{\mathrm{mm}}^2$. An integration time of 1 s and an analog gain of 8.52 were used for calibration. The raw acquisition of the beads and the histogram of the mean gray value of the detected beads are shown in Figs. 7(a) and 7(b), respectively. The mean gray level of each bead is a direct measurement of its fluorescence intensity. The three brightest bead groups could be detected.

With this calibration, we obtained quantitative information on the sensitivity of the mesoscope with a specific fluorescence excitation source. We can be confident that high quantum yield fluorescence markers, in particular nuclear and membrane markers, will be detected by our system. Furthermore, we showed that we were able to discriminate bead populations based on their fluorescence levels.

4.1.

Statistical and Rare Event Imaging

Assessing the ability of our system to perform bimodal statistical imaging of small objects is of particular interest. Recent studies have demonstrated the potential of bimodal imaging techniques for the diagnosis of meningitis from counting blood cells in cerebrospinal fluid⁴⁰ or from counting red blood cells (RBCs), white blood cells (WBCs), or platelets in whole blood.⁴¹ To demonstrate the high-throughput performances of the system, we performed WBC counting in whole blood samples. In healthy blood, the ratio of WBC to RBC is roughly 0.1%.

Nucleated WBCs were specifically labeled with Thiazole Orange dye (390063, Sigma-Aldrich Inc.). This nucleic acid marker is known to have a high quantum yield.⁴² A fluorescent labeling solution was obtained by dissolving Thiazole Orange in methanol at a concentration of $1\mathrm{mg}/\mathrm{mL}$ and it was then diluted to $0.2\mu \mathrm{g}/\mathrm{mL}$ in phosphate buffered saline (PBS) (D8537, Sigma-Aldrich Inc.). About $1\mu \mathrm{L}$ of whole blood was incubated for 30 min in 1 mL of this solution. Three slides (CV 1100-2cv, CellVision Inc.), each having two chambers, were filled with $25\mu \mathrm{L}$ of the solution.

A hologram and the corresponding fluorescence image were acquired with our system. We considered two FOVs per chamber, and hence we had 12 different FOVs for this experiment. A total number of $\mathrm{24,000}\pm 2000$ cells (WBCs and RBCs) were detected in each FOV, owing to their phase contrast, as shown in Figs. 8(a)–8(b). This number of cells allowed us to infer statistically significant results from the acquired data. The detection and classification were achieved using automatic custom-developed algorithms. In the phase images, platelets and nonblood cell objects such as dust could be discriminated based on their size. Labeled WBCs could be counted on the fluorescence image, as shown in Fig. 8(c). To sum up, two images were required to differentiate the WBCs from the RBCs: one in phase and one in fluorescence. The sum of the integration times was below 2 s.

Fig. 8

Multimodal acquisition of whole blood labeled with Thiazole Orange. Full-field images and details of (a) a hologram, (b) the reconstructed phase map, and (c) the fluorescence acquisition.

The ratio $\frac{\mathrm{WBC}}{\mathrm{WBC}+\mathrm{RBC}}$ obtained was $0.159\%\pm 0.026\%$. The error range was calculated from the variance of the results obtained for the 12 FOVs. To check the validity of our approach, we compared this result with the routine complete blood count examination performed using flow cytometry (Sysmex Corp., Japan) and shown in Table 1.

Table 1

Results of the complete blood count examination.

The ratio of interest obtained was: $\frac{\mathrm{WBC}}{\mathrm{WBC}+\mathrm{RBC}}=0.148\%$. This is within the error range of our measurement. This result suggests that the developed system is capable of performing a WBC count.

4.2.

Multimodal Cell Imaging

We additionally explored the ability of our system to image cell cultures with phase and fluorescence contrasts. In particular, we studied fixed U-2 OS cells (human osteosarcoma).

U-2 OS cells were plated and incubated overnight on 18-mm coverslips coated with poly-L-lysin in a 12-well plate in McCoy medium (Gibco) supplemented with 10% Fetal Bovine Serum. They were fixed with 4% paraformaldehyde in PBS for 10 min, permeabilized with 0.5% triton in PBS for 10 min, and rinsed three times with PBS. They were then incubated overnight with 165-nM AlexaFluor 488 phalloidin (Invitrogen Molecular Probes) at 4°C in a humidified chamber in the dark. They were rinsed three times with PBS, then stained with $5\mu \mathrm{g}/\mathrm{mL}$ Hoechst 33342 (Invitrogen Molecular Probes) for 6 min. They were rinsed two times for 10 min with PBS and then quickly with ultrapure water to prevent the formation of salt crystals and were then mounted on a slide with Fluoromount-G medium (Sigma-Aldrich). They were stored at 4°C overnight before imaging.

Owing to the significant phase contrast of the cell nucleus, cells could be detected on the phase image. In particular, if membranes of two contiguous cells were superposed, thresholding the phase signal was enough to detect individual nuclei. Consequently, cells could be counted, as illustrated in Fig. 9(c). This information was qualitative and did not require having a quantitative phase contrast. In the example shown, 1219 cells were detected in the FOV.

Fig. 9

(a) Wide-field segmented fluorescence image of U-2 OS cells. The fluorophore Alexa Fluor 488 phalloidin labels the actin filaments. (b) Details of (a) showing the adhesion surface segmentation for several cells. (c) Same FOV as (b), combined segmentation of the adhesion surface (in green) and the nuclei (in yellow). The nuclei were segmented from the phase reconstruction.

The actin filaments of U-2 OS cells were labeled with Alexa Fluor 488 phalloidin. As illustrated in Figs. 9(a)–9(b), the fluorescence of the cell membrane allowed for segmentation of the surface on which it was adhered to the glass slide. The phase image was not self-consistent because of the low phase contrast of the cell extensions. The use of phase rather than a second fluorescent marker for the detection of cell nuclei simplifies the sample preparation and reduces the cost or leaves the possibility to label other structures of interest. For the accurate segmentation of cells, the proposed bimodal tool is more thorough than the study of a single imaging modality. The segmentation on the fluorescent and phase images was performed using ImageJ.⁴³ Owing to the common path setup, the fluorescence and phase segmentations were accurately registered without the need for numerical operations, as illustrated in Fig. 9(c).

To demonstrate the possibility of performing multiwavelength fluorescence imaging, we imaged U-2 OS cells labeled with Hoechst 33342 and Alexa Fluor 488 phalloidin, which stains the DNA (thus, mostly the nucleus) and the actin filaments, respectively. The green fluorescence was acquired under 520-nm excitation (L520P50, Thorlabs Inc.) coupled to a 561-nm high-pass fluorescence filter (BLP02-561R-25, Semrock Inc.) and the blue fluorescence was acquired under 405-nm excitation (L405P20, Thorlabs Inc.) coupled to a 461-nm high-pass fluorescence filter (03FCG461, Melles Griot Inc.). The images shown in Figs. 10(a)–10(c) are a color merge of the two sequentially acquired images. As the same FOVs were imaged, no numerical registration was required. This image shows the optical performances of the system: subcellular resolution on a large FOV.

Fig. 10

Wide-field bicolor fluorescence data of U-2 OS cells shown as a color merge of sequentially acquired images. The blue fluorescence reveals the DNA in the nucleus stained with Hoechst 33342 and the actin filaments are green because of Alexa Fluor 488 phalloidin. (d)–(f) Corresponding phase images obtained from holographic reconstruction of a single-defocus image at $1387\mu \mathrm{m}$. The red boxes in (a) and (d) correspond to the zoomed images in (b) and (e), respectively. The yellow boxes in (b) and (e) correspond to the zoomed images in (c) and (f), respectively.

These results suggest that it is possible to access both the morphology of individual cells and the cell population statistics. This might be of interest for the study of the cell adhesion process, i.e., the mechanical interaction between a cell and an extracellular matrix, artificial or not. It is of interest in tissue engineering and biomaterial design to study cell growth and motion.⁴⁴