1 May 2009 Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle
Author Affiliations +
J. of Biomedical Optics, 14(3), 030502 (2009). doi:10.1117/1.3130266
We present an endomicroscope apparatus that exhibits out-of-focus background rejection based on wide-field illumination through a flexible imaging fiber bundle. Our technique, called HiLo microscopy, involves acquiring two images, one with grid-pattern illumination and another with standard uniform illumination. An evaluation of the image contrast with grid-pattern illumination provides an optically sectioned image with low resolution. This is complemented with high-resolution information from the uniform illumination image, leading to a full-resolution image that is optically sectioned. HiLo endomicroscope movies are presented of fluorescently labeled rat colonic mucosa.
Santos, Chu, Lim, Bozinovic, Ford, Hourtoule, Bartoo, Singh, and Mertz: Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle

The development of a simple, robust high-resolution fluorescence endomicroscope is driven by preclinical and clinical needs.1 Standard wide-field techniques are hampered by their inability to to reject out-of-focus background, generally leading to low signal contrast. Strategies to reduce out-of-focus background have been based on confocal detection2, 3, 4, 5, 6 or two-photon excitation,7, 8 both requiring some sort of scanning mechanism. Alternatively, out-of-focus background can be rejected in a nonscanning wide-field endoscope by use of structured illumination microscopy (SIM),9 which we have implemented with a flexible fiber bundle.10 While SIM is effective at optical sectioning, we have found that it is highly susceptible to sample motion, the difficulty being that high-resolution image information in SIM is distributed over a series of at least three raw images, meaning that any misregistration between the raw images leads to artifacts in the final processed SIM image. Recently, we have developed a novel hybrid-illumination technique to address this problem.11 In this technique, two raw images are required, only one containing high-resolution information and the other containing low-resolution information. Hence, the name of our new imaging technique: HiLo microscopy.

The two raw images required for HiLo microscopy are based respectively on uniform and nonuniform (or structured) illumination. In our initial implementation of HiLo microscopy, the nonuniform illumination was obtained with laser speckle.11 However, HiLo microscopy is more general than this and can be implemented with any type of nonuniform illumination. In particular, we demonstrate here the implementation of HiLo endomicroscopy with nonuniform illumination in the form of a grid pattern, of the same type as used in SIM. Indeed, our HiLo endomicroscope setup is identical to our previous SIM endomicroscope setup (see Fig. 1 ), except that for HiLo, the spatial light modulator now toggles between two illumination patterns, grid and uniform, whereas for SIM, it sequentially produced three grid patterns of incrementing phase.10

Fig. 1

(a) HiLo endomicroscope setup: an expanded laser beam (Cobolt Calypso, λ=491nm ) is directed onto a spatial light modulator (Holoeye LC-R 768) to create grid and uniform illumination patterns, which are then projected into an imaging fiber bundle ( 600-μm useful diameter; 30,000 fibers; 1.9-μm core diameters; 3.3-μm core separations) equiped with a distal micro-objective (Mauna Kea Technologies: NA=0.8 water, working distance=60μm , field of view=240μm ). The resultant fluorescence is isolated with a dichroic and emission filter (Chroma) and recorded with a CCD camera (QImaging Retiga). Raw images of fluorescently labeled lens-paper fibers with (b) grid and (c) uniform illumination.


The principle of HiLo microscopy was described in Ref. 11. In brief, a final optically sectioned HiLo image is constructed from the fusion of complementary in-focus high- and low-frequency image components. High-frequency components in the uniform illumination image are inherently in focus and are extracted with a high-pass filter. In-focus low-frequency components, on the other hand, must be extracted in a more complicated manner, since the simple application of a low-pass filter to the uniform illumination image does not reject out-of-focus background. To reject low-frequency out-of-focus background, we evaluate the local contrast in the fluorescence image obtained with nonuniform illumination. This local contrast is higher for in-focus image components than for out-of-focus components and hence is axially resolved. A multiplication of the local image contrast with the original uniform illumination image then provides an optically sectioned image, although at low resolution. A fusion of this in-focus low-resolution image with the complementary high-resolution image (inherently in focus) then leads to a full-resolution image that is axially resolved over all spatial frequencies within the microscope passband.

A key step in HiLo microscopy is the extraction of local image contrast from the nonuniform illumination image. In our previous implementation where the nonuniform illumination consisted of laser speckle, this local image contrast was evaluated by calculating the standard deviation of the nonuniform illumination image intensity over local, coarse-grained resolution areas. In our current implementation with a grid pattern, we will adopt a slightly different, although essentially equivalent, approach based on single-sideband demodulation. To understand this approach, let us phenomenologically decompose our uniform illumination image Iu(ρ) into in-focus and out-of-focus components. That is, we write


where ρ={x,y} are spatial coordinates in the image plane. Our final goal is to isolate Iin(ρ) .

The nonuniform image can be decomposed similarly into


where the (imaged) grid illumination is modeled as a sinusoidal pattern of spatial frequency κg in the x direction, with arbitrary phase φ and (imaged) modulation contrast M . Note that only the in-focus image component appears to be modulated, whereas the out-of-focus component does not, precisely because the latter is out of focus.

In(ρ) and Iu(ρ) are the two raw images required for HiLo microscopy. The ratio R(ρ)=In(ρ)Iu(ρ) of these two images leads to


where C(ρ)=Iin(ρ)[Iin(ρ)+Iout(ρ)] is the local image contrast that we have set out to derive. The Fourier transform of R(ρ) —namely, R(κ) —is illustrated in Fig. 2, where we observe that C(κ) [the Fourier transform of C(ρ) ] and its complex conjugate reside in sidebands centered at ±κg . The more highly contrasted the image, the taller and narrower these sidebands become. While several techniques may be used to extract C(κ) from R(κ) , we adopt the standard technique of single-sideband demodulation. In the frequency domain, this corresponds to isolating only a single sideband R+(κ) by applying a one-sided high-pass filter to R(κ) (dashed line in Fig. 2), followed by an inverse Fourier transform to retrieve R+(ρ) . Provided the sidebands are well separated from one another (i.e., they do not overlap), then the local image contrast is given by C(ρ)=[R+(ρ)R+*(ρ)]12M . From C(ρ) , derived in this manner, and the uniform illumination image Iu(ρ) , we can then infer Iin(ρ)=C(ρ)Iu(ρ) . That is, we can extract Iin(ρ) from Iu(ρ) , thereby obtaining an optically sectioned image containing only in-focus contributions.

Fig. 2

Schematic of R(κ) . The right sideband R+(κ) is isolated from the left sideband and from the zero-frequency delta function by applying a one-sided, high-pass filter to R(κ) (dashed line). Note: The exact cutoff response of the filter function can be chosen somewhat arbitrarily.


There are two difficulties with the preceding procedure. First, the sidebands R(κ) and R+(κ) may be so wide as to overlap. This problem can be alleviated by choosing a one-sided, high-pass filter cutoff profile that helps suppress the overlap region. The second problem is that, in general, M is not known a priori. We thus define a new parameter Isu(ρ)=[R+(ρ)R+*(ρ)]12Iu(ρ) that is independent of M . Moreover, we purposefully restrict Isu(ρ) to spatial frequencies smaller than κg by applying a low-pass filter to Isu(ρ) , with user-defined cutoff frequency κcκg , obtaining Ilp(ρ)=LP[Isu(ρ)] . In practice, LP[Isu(ρ)] is performed by convolving Isu(ρ) with a square window of size 2πκg (or integral multiple thereof, to minimize the possibility of aliasing). In addition to confining Isu(ρ) to a well-defined bandwidth, such filtering helps suppress potential artifacts arising, for example, from a nonperfectly sinusoidal illumination pattern.

Finally, the low-resolution image Ilp(ρ) is combined with complementary high-resolution information Ihp(ρ) obtained by applying a high-pass filter directly to the uniform illumination image, such that Ihp(ρ)=HP[Iu(ρ)]=Iu(ρ)LP[Iu(ρ)] . The final processed HiLo image is given by


where η is a scaling factor introduced to ensure a seamless transition of the frequency content of Ihilo(ρ) across the cutoff frequency κc . [In effect, the introduction of η compensates for the fact that M is unknown—see Ref. 11 for an explanation of how η is derived from Ilp(ρ) and Ihp(ρ) , based on their complementary nature.] We emphasize that Ihilo(ρ) is axially resolved over all spatial frequencies, both high and low, within the endomicroscope passband and therefore constitutes an optically sectioned image. Moreover, it provides a faithful rendition of the sample since, to high accuracy, Ihilo(ρ) is directly proportional to the in-focus fluorophore concentration in the sample.

Figure 3 provides comparisons of standard wide-field and HiLo endomicroscope images through a fiber bundle. It should be noted that a longer grid period leads to a stronger imaged modulation depth M —however, at the expense of weaker HiLo optical sectioning capacity. A grid period of 30μm was found to provide a reasonable compromise between grid pattern contrast and HiLo optical sectioning capacity. A rough measure of this sectioning capacity can be inferred from a measurement of the detected signal strength from a uniform fluorescent half-space whose interface is scanned through the focal plane. A comparison of the signal strengths acquired with wide-field and HiLo endomicroscopy is illustrated in Fig. 3d, where we observe that the HiLo signal strength decays much more precipitously than the wide-field signal strength. The corresponding HiLo axial resolution for a laterally uniform plane is inferred from the derivative of the HiLo signal strength, and is found, in this case, to be about 30-μm FWHM. Note that for a laterally uniform sample, Ihp(ρ) vanishes, and the HiLo image is comprised solely of Ilp(ρ) .

Fig. 3

Intermediate (a) Ilp(ρ) and (b) Ihp(ρ) images obtained from raw images in Fig. 1, and (c) final HiLo image. [Note: Panel (b) contains both positive and negative values.) (d) Demonstration of HiLo sectioning capacity using axially scanned fluorescent half-space (Chroma plastic slide—see inset.)


Finally, a comparison of wide-field and HiLo endomicroscopic imaging of colon tissue in motion is illustrated in the multimedia movies ( Videos 1 and Videos 2 ). Note the absence of a residual grid pattern or other motion-related artifacts. Our net HiLo imaging acquisition rate was about 2Hz (i.e., 250ms per raw image). This rate was software limited and will be improved in future versions of our apparatus. Our imaging resolution is about 2.6μm , limited by the Nyquist frequency associated with the (magnified) fiber core quasiperiodicity (see Ref. 10).

Video 1

Wide-field movie of an exteriorized rat colonic mucosa labeled with Acridine Orange dye (MPG, 3.2MPG ).


Video 2

HiLo movie of an exteriorized rat colonic mucosa labeled with Acridine Orange dye (acquired with same grid period as in Fig. 3) (MPG, 3.2MG ).


In conclusion, we have successfully demonstrated the capacity of HiLo microscopy to provide optically sectioned imaging of fluorescently labeled colon tissue through a flexible optical fiber bundle. We anticipate that this will open new possibilities in high-resolution fluorescence endomicroscopy.


This work was partially funded by the NIH (R21 EB007338).



A. Hoffman, M. Goetz, M. Vieth, P. R. Galle, M. F. Neurath, and R. Kiesslich, “Confocal laser endomicroscopy: technical status and current indications,” Endoscopy0013-726X 38, 1275–1283 (2006).10.1055/s-2006-944813Google Scholar


A. R. Rouse and A. F. Gmitro, “Multispectral imaging with a confocal microendoscope,” Opt. Lett.0146-9592 25, 1708–1710 (2000).10.1364/OL.25.001708Google Scholar


L. D. Swindle, S. G. Thomas, M. Freeman, and P. M. Delaney, “View of normal human skin in vivo as observed using fluorescent fiber-optics confocal microscopic imaging,” J. Invest. Dermatol.0022-202X 121, 706–712 (2003).10.1046/j.1523-1747.2003.12477.xGoogle Scholar


F. Jean, G. Bourg-Heckly, and B. Viellerobe, “Fibered confocal spectroscopy and multicolor imaging system for in vivo fluorescence analysis,” Opt. Express1094-4087 15, 4008–4017 (2007).10.1364/OE.15.004008Google Scholar


H.-J. Shin, M. C. Pierce, D. Lee, H. Ra, O. Solgaard, and R. Richards-Kortum, “Fiber-optic confocal microscope using MEMS scanner and miniature objective lens,” Opt. Express1094-4087 15, 9113–9122 (2007).10.1364/OE.15.009113Google Scholar


J. T. C. Liu, M. J. Mandella, H. Ra, L. K. Wong, O. Solgaard, G. S. Kino, W. Piyawattanametha, C. H. Contag, and T. D. Wang, “Miniature near-infrared dual-axes confocal microscope utilizing two-dimensional microelectromechanical systems scanner,” Opt. Lett.0146-9592 32, 256–258 (2007).10.1364/OL.32.000256Google Scholar


B. Flusberg, E. Cocker, W. Piyawattanametha, J. Jung, E. Cheung, and M. Schnitzer, “Fiber-optic fluorescence imaging,” Nat. Methods1548-7091 2, 941–950 (2005).10.1038/nmeth820Google Scholar


P. Kim, M. Puoris’haag, D. Coté, C. P. Lin, and S. H. Yun, “In vivo confocal and multiphoton microendoscopy,” J. Biomed. Opt.1083-3668 13, 010501 (2008).10.1117/1.2839043Google Scholar


D. Karadaglic, R. Juškaitis, and T. Wilson, “Confocal endoscopy via Structured Illumination,” Scanning0161-0457 24, 301–304 (2002).Google Scholar


N. Bozinovic, C. Ventalon, T. Ford, and J. Mertz, “Fluorescence endomicroscopy with structured illumination,” Opt. Express1094-4087 16, 8016–8025 (2008).10.1364/OE.16.008016Google Scholar


D. Lim, K. K. Chu, and J. Mertz, “Wide-field fluorescence sectioning with hybrid speckle and uniform-illumination microscopy,” Opt. Lett.0146-9592 33, 1819–1821 (2008).10.1364/OL.33.001819Google Scholar

Silvia Santos, Kengyeh K. Chu, Daryl Lim, Nenad Bozinovic, Timothy N. Ford, Claire Hourtoule, Aaron C. Bartoo, Satish K. Singh, Jerome Mertz, "Optically sectioned fluorescence endomicroscopy with hybrid-illumination imaging through a flexible fiber bundle," Journal of Biomedical Optics 14(3), 030502 (1 May 2009). http://dx.doi.org/10.1117/1.3130266

Image fusion




Linear filtering

Image resolution


Back to Top