Current techniques for multimodal microscopy are limited to working with separate microscopes and then combining images through digital processing techniques.1 One challenge with this method is ensuring spatial registration of the images while maintaining physiologically correct structure and function. A microscope with coaxial illumination and detection in multiple modalities on one common platform may simultaneously capture images without disturbing the specimen and thereby enable accurate and repeatable spatial registration between modes.
We designed a multimodal microscope that consists of the following imaging modalities on one common platform: optical quadrature microscopy (OQM), differential interference contrast (DIC) microscopy, epi-fluorescence microscopy, confocal reflectance microscopy (CRM), confocal fluorescence microscopy (CFM), and two-photon microscopy (2PM). The OQM, DIC, and epi-fluorescence modes are staring modes because the full field is illuminated and then imaged with an array detector. The CRM, CFM, and 2PM modes are scanning modes, in which the illumination is scanned in a raster pattern across the specimen and each illumination spot acts as a point source that is captured by a point detector. The point detector is synchronized with the scanning mechanism to place the detected intensity within the correct pixel in the image.
The first application of the multimodal microscope was to study embryonic health and viability for potential applications within in vitro fertilization (IVF). Clinicians presently use DIC, or the similar Hoffman optics, to noninvasively analyze the morphological characteristics of embryos to determine which are viable and have the greatest chance of leading to a successful pregnancy. In human embryos, there are two stages in which the embryo will be transferred back to the mother, the eight-cell stage on day 3 or the blastocyst stage on day 5. One important criterion for viability is the number of cells at specific time points during development, but this criterion can be used only up to the eight-cell stage.2, 3 After the eight-cell stage, the cells, termed blastomeres, begin to overlap and prevent the clinician from obtaining an accurate count. Thus, the cell number cannot be determined during the morula stage on day 4. After day 4, the embryo undergoes compaction, and the individual cells cannot be identified with current DIC imaging methods. With a multimodal microscope, we have shown that accurate, nontoxic cell counts can be obtained in mouse embryos past the eight-cell stage using the fusion of OQM and DIC.4 Mouse embryos provide an excellent animal model for human research because early mouse embryo development is very similar to human embryo development, and they have been classified as a model organism by the National Institute of Health. The other scanning modes have the ability to section through the specimen optically, thereby creating three-dimensional (3-D) images of ultra structure. CFM and 2PM may detect the distribution of nuclei and mitochondria with exogenous fluorescent stains; however, exogenous staining of embryos is invasive and, therefore, useful mainly for laboratory research. Similarly, CRM imaging has not been shown to be nontoxic to developing embryos, and thus, CRM is currently used in conjunction with the fluorescent modes for verification of our methods.
In this paper, the design of the multimodal microscope is discussed. The instrumentation and major components of the microscope are described, especially the optical interfaces between the different modes. Preliminary images of embryos are shown to demonstrate the multimodal imaging capability.
The multimodal microscope occupies a 4 by 8-foot vibration-isolated optical table. Major components of the microscope include three lasers, a polygonal-galvanometric confocal scanner, a Nikon inverted TE2000U microscope, and five detectors that are optically interfaced to each other.
Figure 1 shows the design of the multimodal microscope, which is built around and integrated into a commercially available inverted microscope (Nikon TE2000U, Nikon, Japan). This microscope houses a halogen lamp for white light illumination in brightfield and DIC and a mercury lamp for illumination in epi-fluorescence.
A novel feature of the TE2000U microscope is the space and access provided within the infinity space that allows the laser sources to be aligned with the axial path of the microscope. The lasers integrated into the infinity space include: a line-tunable krypton-argon laser (35KAP431-208, Melles Griot, Carlsbad, California) that provides wavelengths of , a line-tunable argon-ion laser (35LAP431-208, Melles Griot) that provides wavelengths of , and an ultrafast (femtosecond) titanium-sapphire laser (Spectra-Physics, Palo Alto, California) that provides wavelengths of . Thus, these lasers provide a total wavelength range of , except for a small gap between , for all scanning modes. In addition, a linearly polarized helium-neon laser is coupled into the microscope by a bandstop dichroic positioned directly above the condenser lens for OQM.
The objective lenses available for imaging include: Nikon Plan Apo NA, Nikon Plan Apo NA, Nikon Plan Apo NA, Nikon ELWD Plan Fluor NA, Nikon Fluor NA water immersion, Nikon Plan Apo TIRF NA oil immersion, and Nikon Plan Apo TIRF NA oil immersion.
Optical Quadrature Microscope
The configuration5 for OQM shown in Fig. 2 is based on a Mach-Zehnder interferometer with a linearly polarized helium-neon laser. The laser is coupled to a single-mode fiber optic 50/50 beamsplitter NPBS1 that separates the beam into two paths, the signal arm (sig) and the reference arm (ref). The output of the fiber from the signal arm is collimated and enters the optical path of the microscope by reflecting from a dichroic that is positioned above the condenser lens. The dichroic is a narrow beamstop centered at to reflect the laser light for OQM and transmit most of the broadband white light for DIC. The light from the signal arm is then phase-delayed by the specimen, collected by the objective lens, and relayed through the beamsplitters to the image plane of the CCD cameras. The output of the fiber from the reference arm is collimated, circularly polarized with a quarter-wave plate (QWP) oriented at to the incident polarization, and relayed through the beamsplitters to the image plane of the CCD cameras to match the wavefront of the signal arm. The light from the signal arm mixes with the circularly polarized light from the reference arm at a nonpolarizing beamsplitter NPBS2, and both outputs have their quadrature components separated by a pair of polarizing beamsplitters PBS1 and PBS2. The four outputs of the polarizing beamsplitters are acquired with four synchronized CCD cameras (Model KP2MN, Hitachi, Japan) and a framegrabber (Matrox Genesis LC, Matrox, Canada) that has the ability to buffer four simultaneous video channels.
Quadrature detection is based on a radio frequency electronic detection technique for measuring the phase of a sinusoidal signal. A reference signal is split, and one component is phase shifted by with respect to the second component. The specimen signal is then mixed separately with both components of the reference. The specimen signal mixed with the non-phase-shifted reference signal is referred to as the in-phase channel (I channel), and the specimen signal mixed with the phase-shifted signal is referred to as the quadrature channel (Q channel). By interpreting the I and Q signals as real and imaginary values of a complex number, it is possible to find the amplitude and phase of the unknown signal. In optical quadrature, the phase shift is accomplished with the QWP in the reference arm, and the I and Q channels are separated with a polarizing beamsplitter.
Analytically, light entering an interferometer is split into the reference and signal paths, represented by the electric fields:is circularly polarized, and the signal beam is linearly polarized at with respect to the coordinate system. The signal beam transmits through the optically transparent sample, and an amplitude and phase shift are applied to the signal beam: signal and a -shifted signal, which are acquired by two cameras and : and , can be captured experimentally by blocking the signal and acquiring an image of the pure reference and then blocking the reference and acquiring an image of the pure signal, respectively. In practice, a balanced detection configuration is utilized by including an additional polarizing beamsplitter and two CCD cameras in the second output of NPBS2 to acquire two additional images: is then reconstructed with the sample moved out of the field of view, and the reconstructed image of the sample is divided by that of the blank to obtain the effect of the object:
OQM has a field of view of in the image plane and a phase error of approximately 9%. The phase resolution was determined using binary phase gratings fabricated with high-quality fused silica substrates that have a transmission phase error less than one-tenth wave across their diameter before fabrication. The overall profile was measured with a surface profileometer, and the phase through the grating was measured to be using light. The error was calculated by comparing the actual phase change through the target with the measured phase change of using OQM.
Confocal Reflectance Microscopy
The krypton-argon, argon-ion, and titanium-sapphire laser, in low-power, continuous-wave mode, are used for illumination in confocal reflectance microscopy (CRM). Scanning in the illumination path and descanning in the detection path are performed with a laboratory breadboard version of a commercial raster scanner (VivaScope 2000, Lucid, Inc., Rochester, New York). The VivaScope is the commercial version of an original video-rate confocal scanner that was developed for imaging skin.6, 7, 8 The scanner consists of a fast rotating polygon for line-scanning and a slower nonresonant oscillating galvanometer for frame-scanning and provides a raster scan with a frame rate of . Relay telescopes are placed between the polygon scanner and the galvanometric scanner and between the galvanometric scanner and the objective lens to ensure that the pupils at the scanners are conjugate to each other and to the pupil of the objective lens. A silicon avalanche photodiode (APD, Model C5460, Hamamatsu) is used for the detection of the reflected light through a pinhole. This pinhole diameter is approximately 5 times and 3 times larger than the diffraction limited lateral resolution for the two most commonly used objectives, NA and NA oil immersion, respectively, and illumination wavelengths . The scanner also includes a polarizing beamsplitter, BS1 in Fig. 1 (Chroma Tech, Burlington, Vermont), and a broadband QWP to maximize light throughput. The light-throughput limitations for such a broad band of wavelengths from blue/green to near-infrared are discussed in Sec. 2.6. A cold mirror is positioned in the infinity space of the microscope to couple the lasers with the microscope axis.
The field of view of the scanning modes is at the image plane. The optical section thickness (equivalent to the axial resolution under diffraction-limited conditions) of a confocal microscope is in terms of the full width at half maximum (FWHM) of the point spread function, and the lateral resolution is . For an NA of 0.75 and wavelengths that are generally used for imaging embryos, the theoretical optical section thickness is and the theoretical lateral resolution is . The signal-to-noise ratio (SNR) for CRM is limited primarily by quantum noise. Based on Mie scattering calculations for backscatter from a spherical object and the measured laser power and losses within the system, the SNR is approximately .
Confocal Fluorescence Microscopy
The krypton-argon and the argon-ion lasers are used for excitation in confocal fluorescence microscopy (CFM).9 The illumination path for CFM is the same as the path for CRM, except the broadband polarizing beamsplitter BS1 is often taken out of the path to maximize light throughput. From the specifications, -polarized light incident upon BS1 has a transmission of 63%, thereby providing the single greatest loss in the transmission path. BS1 also produces a severe loss in the detection path for a popular fluorophore such as FITC (excitation at , emission at ), which is coupled with an additional 50% loss because fluorescent light is randomly polarized, thereby providing a maximum detection of 31% of the returning green/yellow fluorescent light. Removing BS1 significantly improves detection of weaker fluorophores and autofluorescent signatures in biological samples, but BS1 is kept in place when the CRM and CFM modes are used in tandem. Removing and replacing BS1 is possible with minimal loss of alignment in both illumination and detection paths because the mount is precisely located with mechanical pins on the baseplate of the VivaScope scanner.
The returning fluorescent light retraces the illumination path until reaching the dichroic mirror M2. The dichroic mirror is mounted within a six-position motorized filter wheel (FW102, ThorLabs, Newton, New Jersey) and reflects the illuminating wavelengths while transmitting the returning fluorescent wavelengths. Any bleedthrough, i.e., stray illumination, backscattered light, or unwanted fluorescent or autofluorescent light, is further rejected by emission filters placed in another motorized six-position filter wheel F1 that is positioned in the detection path after M2.
Fluorescent light that has transmitted through F1 is focused through a pinhole with an focal length lens. This pinhole diameter is approximately 14 times and 10 times larger than the diffraction-limited lateral resolution of the two most commonly used objective lenses, NA and NA oil immersion, respectively, and an illumination wavelength of . The pinhole used in CFM is larger than that used in CRM because fluorescent signals tend to be weaker than reflective signals under confocal conditions. Light that has traveled through the pinhole is detected with a head-on photomultiplier tube (PMT) (Model HC124-02, Hamamatsu). The PMT’s active detection area is in diameter with a specified photosensitivity of , which is approximately 2 orders of magnitude more sensitive than that of the APD used for CRM. Since the PMT is sensitive and has a large active area, blocking stray light is imperative for an optimal SNR. Thus, a custom-made mount was fabricated for the PMT and pinhole with a ULM-TILT tilt mount and a 420 Series linear positioner (Newport Electronics).
The optical sectioning and lateral resolution of CFM is dependent on both the illumination and detected wavelengths. Using the detected wavelength, the theoretical optical sectioning and lateral resolution are approximately and , respectively, for an NA of 0.75. Based on the analysis of fluorescence emission, light throughput, and the detected power the SNR is estimated to be approximately .
The titanium-sapphire laser is used in ultrafast pulsed mode for illumination in two-photon microscopy (2PM).10 The scanned illumination path for the 2PM mode is exactly the same as the titanium-sapphire path in CRM with a hot mirror (Table 1 ) replacing the cold mirror in the infinity space. To minimize throughput losses and increase detection sensitivity, the returning fluorescent light is not descanned, but transmitted through the hot mirror to a detector mounted to one of the camera ports of the microscope. This enables the shortest detection path. The detector is a PMT (HC124-02, Hamamatsu) with a photosensitivity of that is similar to the one used in CFM. For multiple-labeled samples, precision filtering must be preformed with glass filters positioned within the dichroic filter cubes provided in the microscope for epi-fluorescence.
To avoid scanning the returning fluorescent light across the PMT’s active area, the PMT cathode is placed conjugate to the pupil of the objective lens, where the light is stationary. If the PMT were placed conjugate to the image plane, the fluorescent light would scan across the PMT’s cathode such that any nonuniformities in the active area would appear in the image. Since the camera ports of the microscope are designed for detectors to be placed conjugate to the image plane, a specially machined mount was fabricated with a lens to place the PMT in a pupil plane.
In 2PM, the optical section thickness may be estimated by the FWHM of the axial point spread function for the illumination, which is approximately . For an NA of 0.75 and the illumination wavelength of generally used, the theoretical optical section thickness is approximately . Based on the analysis of fluorescein equivalents per sphere and the measurement of detected power and losses, the SNR is estimated to be approximately .
Optical interfaces must be carefully chosen in a multimodal microscope to combine various sources and detectors while maximizing light throughput and maintaining image registration. The first optical interface is a cold mirror M1 that combines the titanium-sapphire laser and the argon lasers. Selection of this mirror is critical because excessive losses can degrade image brightness.
The second optical interface is the broadband polarizing beamsplitter BS1. This beamsplitter was chosen to transmit and reflect the broad range of wavelengths from . Across such a wide range of wavelengths, polarizing beamsplitters are not truly achromatic, so the light throughput is not consistently high and varies with wavelength. For the chosen BS1, the light throughput is 63% at , 87% at , and greater than 90% for wavelengths from , with the exception of a -wide dip centered at where the throughput drops to approximately 80%. If the CRM mode is not used in an experiment, BS1 may be removed since the mount is located with two pins in the baseplate of the scanner. This allows precise, repeatable placement with minimal need for realignment.
Following the polygon and galvanometric scanners, the scanned beam must be interfaced to the Nikon microscope. This requires the polygon and galvanometric mirrors to be optically conjugate with the entrance pupil plane of the objective lens to minimize beam walk across the pupil of the objective and to minimize vignetting. Two relay telescopes are used with antireflection coated lens for broadband transmission. The first is positioned between the polygon mirror and the galvanometric mirror, and the second between the galvanometric mirror and the objective lens.
An important optical interface for coupling the scanning modes into the microscope is the mirror that is placed in the infinity space of the microscope, between the objective and the epi-fluorescence filter turret that reflects the scanned beam toward the objective lens. Either a cold mirror or a hot mirror may be selected depending on the modalities being used (Table 1). A cold mirror enables CFM and visible CRM with the argon lasers, and a hot mirror enables 2PM and CRM with the titanium-sapphire laser, and DIC. When a mirror is not extended into the infinity space, the port selector underneath the epi-fluorescence filters directs the light to the appropriate detector.
Computer interfacing in the multimodal microscope is necessary for automating mechanical functions. Two computers are connected to a hub to ensure fast acquisition rates. The main computer (Cockpit) controls most automated functions, while the computer connected to the confocal scanner (VivaScope) acquires images from all of the scanning modes with an Imaq 1408 framegrabber (National Instruments, Austin, Texas).
All image capture commands are performed with the Cockpit computer using socket programming written in Python to communicate between the Cockpit and the VivaScope computers. This type of programming allows the two computers to communicate through the network with a socket server program operating on the Vivascope machine, along with a socket client program on the Cockpit. Through this configuration, the Cockpit accesses all of the commands of the VivaScope for operating the scanner and image acquisition. The Cockpit computer acquires approximately in communication mode, while the VivaScope computer captures approximately in stand-alone mode. Since socket programs are fully network compatible, it is possible to operate the microscope remotely, such that any remote host computer may operate the socket client program in conjunction with the server on the VivaScope computer to capture live images.
The Cockpit computer hosts several hardware components. Two framegrabbers are used, one for the CCD cameras for OQM (Matrox Genesis LC board) and the other for the SPOT Diagnostics Cooled CCD camera for DIC and epi-fluorescence. The Cockpit computer also contains a spectrometer (Ocean Optics) that measures the output wavelength of the titanium-sapphire laser and a GPIB board used for general testing, which are connected through a universal serial bus (USB) connection. The piezo-electric crystal-activated stage, motorized -translation stage, and mechanical shutters, which control illumination in all of the modes, are connected and controlled through a serial interface.
The multimodal microscope includes a live-cell chamber, motorized -translation stage, piezo-electric stage, linear positioner for the flip-up mirrors in the infinity space, shutter servo controllers, and motorized filter wheels containing the dichroic and bandpass filter sets. All of the mechanical interfaces are combined using the Python socket software to allow for automated long-term, time-lapse, live-cell imaging.
A heated and perfused stage/chamber has been implemented for live-cell imaging. The culture dish system (Delta T4, Bioptechs) is mounted on a custom baseplate allowing it to be positioned in three dimensions ( , , and ) by the automated -translation stage and the piezo-electric stage. The system maintains constant temperatures from and gas concentrations (we use a mix for embryo growth) in the cell growth chamber. The system has been demonstrated to maintain embryo health for periods up to , which is sufficient to image from the single-cell stage to the blastocyst stage in mouse embryos.
A motorized -translation stage (ProScan, Prior Scientific, Cambridge, UK) with a maximum travel range of was implemented to allow the imaging of larger samples and the construction of ultra-widefield mosaics. A single-axis piezo-electric crystal-activated stage (Model PZ400, Piezosystems Jena, Jena, Germany) with a maximum travel of was mounted to the -translation stage to translate the sample along the optical axis while keeping the objective fixed. Several holders were also machined to accommodate a wide range of samples.
Flip-up Mirror Control
The cold and hot mirrors that couple the scanning lasers into the microscope are mounted consecutively on a motorized linear positioner (Zaber Technologies, Ottawa, Canada) and positioned in the infinity space below the objectives. The linear positioner provides a lateral precision of with rapid and repeatable interchangeability. The holder is attached to the linear positioner via a rotary positioner (GM-1R, Newport Electronics) to place the reflected beam accurately into the pupil of the objective. The configuration is also programmable such that multimode experiments can be executed without extensive realignment of the mirrors between modes and to allow for long-term automated time-lapse imaging with various light sources without constant user presence.
Imaging of Embryos
Figure 3 shows images of a multicellular mouse embryo at the blastocyst stage in DIC, CRM, 2PM, CFM, and OQM. The embryo was stained using Mitotracker 488 (note the absence of signal from the blastocoel, which is a vacuole in the embryo devoid of cells) and Hoechst 33342. The Mitotracker dye binds to the mitochondria in the cells and was excited using light in CFM, while the Hoechst dye binds to the DNA within the nuclei and was excited with light in 2PM (both Mitotracker and Hoechst labeling were limited to the trophectoderm). The images were captured using the NA objective lens. The CRM, CFM, and 2PM images each represent one image from a stack of 25, captured in steps. Due to the noise in the PMTs and the fast pixel time , 10 frames were averaged to produce an image with an improved SNR. For this data set, the averaged images were captured at approximately one per second, thereby requiring to capture an image stack.
Specifications of mirrors.
|Mirror||Part No.||Transmission spectrum||Reflective spectrum||Modes|
|Cold mirror||03MHG009(Melles Griot)||CFM,visible CRM|
|Hot mirror||03MHG009(Melles Griot)||2PM, CRM (withTi-sapphire laser)|
Conclusions and Future Work
The multimodal optical microscope provides multiple imaging modalities on one common platform. Multimodal optical microscopy may prove useful for the assessment of embryo viability, but a smaller, portable design will be necessary for use in a clinical setting. A simpler design with more efficient optical interfaces will also improve light throughput and detected signal across a broad band of wavelengths. Spatial registration of images may be improved with fusion methods such that features unique to one imaging mode may be localized in another image of a different mode. In addition to embryo viability studies, this multimodal microscope is capable of imaging live samples in up to six different modes for at least for particle tracking, drug delivery, and morphological development. The fusion of the imaging modalities may provide information that is not evident by observing images from individual modes alone.
The microscope website is http://www.keck3dfm.neu.edu/.
The author acknowledges the following people for their help in creating the multimodal microscope: Gustavo Herrera, Rafal Norton, Matt Bouchard, Judy Newmark, Martina Comiskey, and Dr. Carol Warner. Funding for the multimodal microscope was provided by the W. M. Keck Foundation and by CenSSIS, the Bernard M. Gordon Center for Subsurface Sensing and Imaging Systems, under the Engineering Research Centers Program of the National Science Foundation (Award No. EEC-9986821).