Spectral imaging is also known as imaging spectroscopy, which refers to the technology that integrates conventional imaging and spectroscopy methods to obtain both spatial and spectral information of an object. It was originally defined by Goetz in the late 1980s and discussed for remote sensing of the Earth.1 Spectral imaging can be divided into multispectral imaging, hyperspectral imaging (HSI), and ultraspectral imaging according to its spectral resolution, number of bands, width, and contiguousness of bands. Multispectral imaging systems generally collect data in few and relatively noncontiguous wide spectral bands, typically measured in micrometers or tens of micrometers. These spectral bands are selected to collect intensity in specifically defined parts of the spectrum and optimized for certain categories of information most evident in those bands. While HSI systems can collect hundreds of spectral bands, ultraspectral imaging systems collect even more. Figure 1 shows the concept of the hypercube data captured by a spectral imaging system. The spectral imaging data can be visualized as a three-dimensional (3-D) cube or a stack of multiple two-dimensional (2-D) images because of its intrinsic structure, in which the cube face is a function of the spatial coordinates and the depth is a function of wavelength. One of the important advantages of this technique is that it can acquire reflectance, absorption, or fluorescence spectrum for each pixel in the image, which can be used to detect the biochemical changes of objects that cannot be identified with traditional gray or color imaging methods. Spectral imaging technology has been originally substantiated within remote sensing fields, such as airborne surveillance or satellite imaging, and has been successfully applied to mining and geology, agriculture, military, environmental, and global change research.2
According to the electromagnetic theory, different biochemical constituents commonly have different spectral signatures.3 These signatures are usually generated by the interactions between materials and electromagnetic waves, such as electron transition, atomic and molecular vibration or rotation. The biological and pathological changes in tissues and organs also have a close relationship with the spectra. Spectral characteristics in different wavelength regions yield a distinguishable spectral signature, making pathological changes distinguishable. Therefore, the spectral imaging technology also can be extended to the biomedical engineering field to estimate the physiological status of biological tissues, since it can take advantage of the spatial relationships among the different spectra in a neighborhood. This technology opens new prospects for life science by which scientists can identify and quantify the relationships among biologically active molecules, observe living organisms noninvasively, perform histopathological and fluorescent analyses, and enhance biological understanding of diseases. In the past decades, researchers have developed various spectral imaging systems for the biochemical analysis of various biological organs and tissues. These studies have shown that the extension of this technology to the biomedical engineering field has incredible potential for researchers to get more information on organs, tissues, and even cells than those traditional optical imaging methods (such as CCD cameras and light microscopes). Biomedical spectral imaging technology has attracted more and more attention and has gained importance in research today.
This paper summarized the origins and major developments of spectral imaging technology used in the biomedical field. The development of biomedical spectral imaging systems, comparison of different spectral imaging methods, system performance, in vivo, fluorescence, and histological analyses, as well as current limitations and challenges of this technology are discussed. The purpose of this paper is to highlight the advantages and disadvantages of biomedical spectral imaging reported previously and to identify fundamental and applied research issues that occurred in the current and future biomedical applications.
Advantages of Biomedical Spectral Imaging
Most traditional biomedical optical imaging methods can only capture gray or color images of biological samples. The targets of interest in these kinds of images are generally analyzed by their spatial properties such as size, shape, and texture. It has been widely recognized that the monochrome and RGB color imaging methods have limitations in the early detection and identification of tissue abnormalities.4,5 The obtained diagnostic information is poor, since in most cases, the metabolic or compositional alterations occurred during the progress of the disease and do not significantly alter the color characteristics of abnormal tissues.6 Another generally used optical method is the spectroscopic diagnostic technology, which can obtain an entire spectrum of a single tissue site within a wavelength region of interest. This method is often referred to as the point measurement method, which cannot provide the spatial information of samples. Different from those traditional optical diagnostic methods, biomedical spectral imaging technology can capture the contiguous spectrum for each image pixel over a selected wavelength interval. This character makes it possible not only to detect some physiological changes of biological tissues by their reflectance or transmittance spectral signatures, but also for early diagnosis of some diseases since the shapes of the spectra yield information about biological samples. Some advantages of biomedical spectral imaging (multispectral, hyperspectral) technology over conventional monochrome, RGB, and spectroscopy are presented in Table 1. From the table, it can be seen that the biomedical spectral images contain more information than the traditional monochrome, RGB, and spectroscopy methods. Biomedical spectral images make it possible to take advantage of the spatial relationships among the different spectra in a neighborhood, which allows more elaborate spectral-spatial models for a more accurate segmentation and classification of the image.7 Therefore, the spectral imaging technology can find potential applications in pathology, cytogenetics, histology, immunohistology, and clinical diagnosis.
Comparison of monochrome, RGB, spectroscopy, multispectral, and hyperspectral features.
Spectral Imaging Methods
In the past decades, different kinds of spectral imaging methods and related technology have been proposed in acquiring the spectral image data of natural materials. To make the content and structure clear, this review mainly focuses on four typical approaches: the whiskbroom, pushbroom, staring, and snapshot, which have been commonly used in the remote sensing field and now have been extended to the biomedical imaging applications. Other related ones, such as the Raman, the miscellaneous, and Hadamard methods, will not be discussed in detail in this review.
The whiskbroom imaging mode, also known as the point-scanning method, was used originally by the Earth Resources Technology Satellites and then implemented by the Airborne Visible/Infrared Imaging Spectrometer. In the whiskbroom sensor, rotating mirrors were often used to scan the landscape from side to side perpendicular to the direction of the sensor platform. The rotating mirrors redirect the reflected light to a point where a single or just a few sensor detectors are grouped together. As shown in Fig. 2(a), a single point is scanned along two spatial dimensions ( and ) by moving either the sample or the detector. Then the reflected light is dispersed by a prism and recorded by a linear array detector. The spectral image data cubes (, , ) can be acquired by scanning over the 2-D scene (, ) and by dispersing in the wavelength domain (). In the whiskbroom mode, two-axis motorized positioning tables are usually needed to finish the image acquisition, which makes the hardware configuration complex. In addition, this kind of imaging mode is also usually time-consuming because it needs to scan both in the and spatial dimensions. Therefore, another scanning approach with high acquisition speed and high sensitivity has been proposed, that is, the spectral confocal laser-scanning method. This method is usually accomplished by illuminating and acquiring images through confocal or conjugate pinholes to restrict the in-focus optical section thickness and by incorporating multiple lasers and wavelength dispersive spectrophotometers to get spectral information. An advantage afforded by this method is the ability to control depth of field, eliminate or reduce background information away from the focal plane, and the capability to collect serial optical sections from thick specimens. Another advantage of the whiskbroom imaging is that it has fewer sensor detectors to keep calibrated as compared with other types of sensors. Most commercial spectral confocal scanning instruments belong to this kind of imaging mode.
The pushbroom approach, also known as line scanning, has been used by some earth observation satellite systems operating from space, such as the SPOT system and the Advanced Land Imager (ALI). Different from the whiskbroom method that scans one point at a time, the pushbroom method can simultaneously acquire a slit of spatial information, as well as spectral information corresponding to each spatial point in the slit for one scan, that is, a special image with one spatial dimension () and one spectral dimension () can be taken at the same time with a matrix detector. As shown in Fig. 2(b), the pushbroom method collects a slit image from the object dispersed onto a 2-D detector, in which the spatial information is displayed along one axis and wavelength information along the other. Then the spectral image data cube can be obtained by scanning the slit in the direction of another spatial axis. A pushbroom scanner can get more light than the whiskbroom one, since it can stay at a particular area for a longer time providing a long exposure on the matrix detector and a relatively high spectral resolution as well.8 However, in most of the pushbroom imagers, either the camera or the object should move accordingly as one line of the image is acquired at each scan and the move should be synchronized with the frame acquisition rate of the array detector in order to ensure the measurement of a smooth image. Another approach to perform the pushbroom procedure is to put a movable slit aperture or a rolling shutter into the conjugate image plane to eliminate out-of-focus light. This method allows for higher illumination transmission while exhibiting only a slight increase in the confocal depth response, which has been widely used in multispectral line confocal imaging systems.9
The staring approach (also known as band sequential method) is a spectral scanning method that acquires a single band 2-D gray-scale image (, ) with full spatial information at once. As shown in Fig. 2(c), this mode generally uses filters [such as filter wheels containing fixed bandpass filters, linear variable filters (LVF or wedge filters), variable interference filter (VIF), and tunable filters] instead of a grating or prism in front of a matrix detector. The light passes through the focusing optics. Then it is filtered, which induces only a narrowband segment of the spectrum that impinges on the focal plane of the detector (typically is a matrix CCD). Therefore, the 2-D image in one wavelength can be captured at a given time, and the image cube is filled by tuning the output wavelength of the filter as a function of time.10 In the staring mode, the entire scene is imaged onto the focal plane, one spectral band at a time, and goes through the spectral wavelengths, which is different from the whiskbroom and pushbroom approaches. Therefore, the number of spectral bands acquired is completely up to the user. Also, and importantly, a dynamic range can be preserved, since different exposures can be employed at different wavelengths.
The snapshot (also known as single-shot) method is intended to record both spatial and spectral information on an area detector with one exposure. Unlike the whiskbroom, pushbroom, and staring modes that need to scan either in the spatial dimension or in the spectral dimension, thereby limiting their temporal resolution, the snapshot mode is an imaging technique with no need to scan at all. As shown in Fig. 2(d), the snapshot mode can acquire a complete spectral data cube in a single integration by directly imaging the remapped and dispersed image zones onto a CCD detector simultaneously.11 Although this mode can acquire data directly with minimal postprocessing to build a 3-D data cube, its spatial and spectral resolutions are limited as the total number of voxels cannot exceed the total number of pixels on the CCD camera. Therefore, for a given CCD, one can always increase the spatial sampling at the expense of spectral sampling, and vice versa.12
Comparison of Different Approaches
Among these four kinds of spectral imaging approaches, each one had both advantages and disadvantages when they were applied to analyze biological tissues. Table 2 shows the comparison of the whiskbroom, pushbroom, staring, and snapshot imaging modes. There is no “absolutely” best mode, but in order to select the most suitable one for a given biomedical application, the characters of different biological tissues and the detection objective must be considered.
Comparisons of whiskbroom, pushbroom, staring, and snapshot approaches.
|Point of comparison||Whiskbroom||Pushbroom||Staring||Snapshot|
|Scanning||Both spatial scan (x and y dimension)||One of the spatial scan||Spectral scan||No scan|
|Spectral dispersion element||Dispersive element [prism, grating, prism grating prism (PGP)]||Dispersive element (prism, grating, PGP)||Tunable filters (AOTF, LCTF, circular and linear variable filters), interferometer||Diffractive elements (prism, grating, digital hologram)|
|Wavelength ranges||Wide (†††)||Wide (†††)||Mediate (††)||Wide (†††)|
|Spectral resolution||High (†††)||High (†††)||Mediate (††)||Low (†)|
|Throughput||High (†††)||High (†††)||Low (†)||High (††)|
|Data cube collection timeb||Long (†††)c||Long (†††)||Short (††)d||Fast (†)|
|Complexity||Complex (†††)||Complex (†††)||Simple (†)||Mediate (††)|
|Cost||Low (†)||Low (†)||High (†††)||Mediate (††)|
|Substitutability||Hard (†††)||Mediate (††)||Easy (†)||Mediate (††)|
|Coupled optics||Camera lens, microscope||Camera lens, microscope||Camera lens, colposcope endoscope, microscope||Fundus camera, endoscope, microscope|
Note: Some parameters of applicability and utility for accurate use are compared using the following relative ranking marks: †, worse or difficult; ††, better or permitted; and †††, approach is ideally suited.
Some whiskbroom and pushbroom systems allow selection of spectral range and resolution such as Zeiss and Nikon systems listed in Table 4.
Refers to the time spent capturing the same size of data cube (x, y, λ).
Some laser scanning systems (for example, the Zeiss and Nikon point scanning systems) have as fast a speed as the snapshot systems. Some systems with conventional mechanical x-y scanning will take a longer time to capture a data cube (x, y, λ).
Systems based on liquid crystal tunable filter (LCTF) usually take a longer time than the laser scanning systems because the switching speed is limited by the relaxation time of the crystal. Acousto-optical tunable filter (AOTF) can switch wavelengths <1 ms but might be limited by photon count.
From Table 2, it can be seen that the whiskbroom and pushbroom modes use a dispersive element [prism, grating, prism-grating-prism (PGP), etc.] to split the light, a factor that gives an advantage of uniformly high efficiency, low scatter, and low cost to these kinds of imagers. Another advantage of the pushbroom mode is that the collected images reflect only the plane, making a single line of light feasible as the source for image illumination.13 This approach is suitable for applications of searching an object with specific spectral signatures as the entire spectrum of each pixel in the slit image is available in real time. In addition, systems that are based on dispersion elements, such as prisms and gratings, are also efficient for measuring the spectral images through confocal microscopes, which implement the scanning procedure by scanning optics.14 The whiskbroom and pushbroom modes also have disadvantages. The optical design is complex and the data cube collection time is long for conventional mechanical scanning systems. Although some optical scanning systems such as the Zeiss and Nikon laser scanning systems can capture data as fast as the snapshot systems, the acquisition of high-contrast images commonly accompanies strong photobleaching, which is of particular concern in prolonged imaging experiments.15 In general, the whiskbroom and pushbroom imagers were often coupled with microscopes and used for fluorescent and histopathological analyses.
The most important advantages of the staring mode are the short collection time of data cube and the ease of coupling with other medical optical instruments, as no relative spatial move is required between the sample and the detector. Another advantage is the flexibility of selecting the optimal spectral range for each application. These advantages make it suitable for a wide range of biomedical applications, such as detecting organs, tissues, and cells in vivo or in vitro when it is coupled with a camera lens, endoscope, or microscope.13,16 There are also some disadvantages of using this mode, such as a lower spectral resolution, lower transmission rate (typically reaches only 35 to 60% depending on the type of device and model), and a higher cost than the whiskbroom and pushbroom modes (except the laser scanning systems). Although tunable filters attempt to shorten the total capture time by selectively acquiring spectral bands, or by altering each channel’s exposure time as a function of wavelength so that the total data capture time can be saved in bright channels, there are still shortcomings that can be difficult to overcome, such as the low throughput and sequential acquisition mode, particularly in multifluorophore imaging, which requires at least 30 spectral channels to be captured.15 The interferometer-based spectral imaging systems, however, usually have high optical throughput, high and variable spectral resolution, mechanical/temperature stability, and flexible selection of wavelength range. They still have limitations for practical applications, such as the varying sensitivity in the entire spectral region and the computation intensity for data transform.17
For the snapshot mode, no scanning in either spatial or spectral domains is needed for obtaining a spectral data cube, making it attractive for applications requiring fast spectral image acquisitions. Therefore, it is an ideal solution for applications where imaging time is a major factor. It is suitable for time-sensitive imaging experiments, such as the observation of fast-diffusing molecules or the determination of temporally resolved dynamic biological processes. It is also suitable for detecting oxygen saturation maps in the retina of human eyes when it is coupled with a fundus camera. This has been demonstrated by Johnson et al. with computed tomography imaging spectrometer (CTIS) with an exposure time of .18 However, this mode is still in the early stages and not fully developed. Only a few implementations that rely on complicated fore-optics design and computationally intensive postprocessing for image reconstructions are currently available, with limitations for range and resolution of spatial and spectral dimensions.17
In summary, each spectral imaging mode has advantages and disadvantages when used in biomedical analysis. For example, some whiskbroom and pushbroom systems can get high spectral and spatial resolution and are low costing, but their hardware assembly is usually complex. The staring mode can be assembled compactly and coupled easily with other optical instruments, but it is more expensive than the others. The snapshot mode can quickly capture spectral data cube, but its spectral and spatial resolutions are limited. However, the advantages of different modes can be exploited fully if they are selected correctly and used according to the objectives of different biomedical analyses. We will discuss this in detail in the following sections.
Biomedical Spectral Imaging System
To acquire a 3-D data cube, spatial (2-D) and wavelength as a third dimension of a biological tissue, a spectral imaging system suitable for biomedical applications needs to be developed. As shown in Fig. 3, the hardware architecture of a typical biomedical spectral imaging system generally consists of four parts: collection optics or instruments, spectral dispersion element, detector, and system control and data collection module.
The collection optics or instruments part refers to the collection and imaging optics such as microscopes, endoscopes, camera lens, etc., which can produce an image on the spectral dispersion element. The spectral dispersion element is the heart of the system that enables the separation of the light into different wavelengths. Dispersive spectrometers and spectral filters are two kinds of commonly used wavelength dispersion devices. The dispersive spectrometer is a collection of transmitting or reflecting elements separated by a distance comparable to the wavelength of the light under investigation. It often refers to prisms, gratings, PGP, or beam splitters, which can be used with whiskbroom, pushbroom, and some snapshot imaging modes. Spectral filters have the characteristic to pass radiation through a very narrow bandpass, or spectral bin, and this can be spectrally tuned over a wide spectral range, usually in a very short time. Acousto-optical tunable filter (AOTF), liquid crystal tunable filter (LCTF), and circular and linear variable filters (CVF and LVF) are the most popularly used tunable filters in spectral scanning instruments. Another generally used spectral dispersion element is the interferometer, which can obtain spectral images by changing optical path-length difference (OPD) between two beams, recording the interference intensity as a function of the OPD (interferogram), and Fourier transforming the interferograms pixel by pixel.19 For example, the Michelson interferometer and the Sagnac interferometer have been used to develop some spectral imaging systems for different biomedical research.20,21 The detector is used to acquire the light intensity required to measure the spectrum at each pixel in the image. Some sensors such as the photomultiplier tube, linear CCD, area CCD and CMOS, focal plane array, and electron multiplying charge coupled device (EMCCD) can be used as a detector according to different imaging requirements. In recent years, the scientific CMOS (sCMOS) has been introduced and also used in biomedical imaging as it offers an advanced set of performance features, such as high resolution, without compromising read noise, dynamic range, or frame rate. These features make the sCMOS ideal for high-fidelity, quantitative scientific measurement. Therefore, some commercial sCMOS [for example, the ORCA-Flash2.8 and Flash4.0 camera (Hamamatsu), Andor and PCO series sensors, and Rolera™ Bolt Camera (QImaging)] can also be used as a detector of spectral imaging system.
In the past decades, researchers have developed various spectral imaging systems for biochemical analysis on various biological organs, tissues, and cells. Table 3 shows some selected spectral imaging systems used in the biomedical field. We will break up the discussion of instrument configuration according to the imaging mode in the following subsections.
Current biomedical spectral imaging systems.
|Imagers||Spectral dispersion elements||Coupled optical instrument||Imaging mode||Detector||Wavelength range (nm)||FWHM (nm)||Spatial resolution||Acquisition time||Applications||Reference(s)|
|Hyperspectral fluorescence imaging||Grating-based imaging spectrograph||17 mm lens or NIR lens||Pushbroom||512×512 or 2048×506 CCD||400 to 1000||0.6 to 2||9 to 12 μm||Mouse tumor tissue||22 to 24|
|Hyperspectral microarray scanner||Grating-based imaging spectrograph||Infinity-corrected objective||Pushbroom||576×288 DV465 detector||490 to 900||3||10 to 30 μm||5 min||Microarray DNA||25 to 27|
|Hyperspectral microscopy||Grating-based coded aperture spectral engine||Microscope (Nikon)||Pushbroom||Santa Barbara Instrument Group ST-7XME CCD array||550 to 665||1||1.54 to 7.7 μm||CdTe quantum dots||28 to 30|
|Hadamard transform spectral microscopy||Grating-based monochromator||Fluorescence microscope||Pushbroom||S7031-0907 TE 512×122 CCD||580 to 745||0.3||Breast tumor and quantum dots (QDs)||31 to 33|
|SPLE||Prism-based SPLE||Zeiss 150FC colposcope||Pushbroom||Monochrome CCD camera||400 to 800||12 to 28||Stained cervical biopsies sections||34, 35|
|Hyperspectral tongue imaging system||PGP-based imaging spectrograph||Camera lens||Pushbroom||652×499 CCD||400 to 800||5||Tongue, optic nerve head||36 to 41|
|Hyperspectral fluorescence imaging microscope||Grating- or PGP-based spectrograph||Microscope (Olympus)||Pushbroom||1536×1024 CCD or EMCCD||380 to 780||2||0.18 μm (100×)||1.2 s/line or 340 ms/line||Human heart, artery section||13, 42 to 46|
|MPHI||PGP-based imaging spectrograph||Microscope (Nikon Co. Ltd.)||Pushbroom||460×300 CCD||400 to 800||2||1.125 μm (40×)||Retina sections, oral cancer cell||47 to 51|
|Prototype open-frame MACROscope||Grating||Microscope||Whiskbroom||Linear PMT array||500 to 720||6.8||1 μm||200 μs/spectrum||Heart tissue section||52, 53|
|Hyperspectral imaging system||Spectrograph or prism MEMS scanner||Lens||Whiskbroom||CCD or PMT||387 to 707 550 to 700||6||2 mm||90 s||Colon cancer, tongue, epithelium||54, 55|
|NIRSI||LCTF [Cambridge Research & Instrumentation (CRI) Cambridge MA]||Nikon microlens/microscope||Staring||512×512 CCD camera||400 to 720 650 to 1050 960 to 1700||5 to 10 nm||0.2 mm||536.8 ms/band or 90 s||Skin, heart, stained tumor sections, blood in tube||56 to 68|
|In vivo spectroscopic imager||LCTF (CRI)||50 mm, f/l.4 or a 60 lens (Nikon)||Staring||768×512 cooled, CCD||420 to 750||6.8 at 550 nm||0.45 mm||50 ms/band||Skin||69 to 74|
|Active DLP hyperspectral system||DLP-based spectral illuminator and grating||Standard 50 mm lens (Nikon)||Staring||1392×1040 CCD||380 to 780||5||0.13 to 0.36 mm||0.26 to 11.32 ms/line||Visualized ischemic changes||75, 76|
|Multispectral imaging system (MSIS)||LCTF (CRI)||18 to 108 mm lens||Staring||CCD||400 to 1000||3 nm||40 ms/band||Cervix, skin||6, 77|
|Spectral microscope system||VIF monochromator||Microscope||Staring||Megapixel CCD||400 to 1000||7.5 at 500 nm||Breast cancer, lymphocytes||78, 79|
|Hyperspectral imaging system||AOTF (Brimrose)||Camera lens||Staring||CCD||400 to 880 nm||2 at 543 nm||50 ms/band||Human tongue, hands, tooth||4, 80, 81|
|NIR hyperspectral imaging system||AOTF (Brimrose, VA-200)||50 mm short-wave infrared lens||Staring||320×256 InGaAs focal plane array||1000 to 1650||10 nm||Tooth||82, 83|
|Endoscope imaging system||AOTF (Brimrose)||Endoscope, flow cytometry system||Staring||512×512 CCD or intensified charge coupled device||400 to 700||1 to 4 nm||Skin, brain, tissues, flow cytometry||5, 16, 22, 84 to 86|
|Hyperspectral video endoscope||AOTF (Brimrose)||Endoscope||Staring||1004×1002 EMCCD||400 to 650||5 nm||Surgeries, cancerous tissues||87, 88|
|Hyperspectral imaging microscope||LCTF (CRI)||Microscope||Staring||CCD||400 to 720||10 nm||Live-viable versus white blood cell, skin||89 to 93|
|Molecular hyperspectral imaging system||AOTF (Brimrose)||Microscope (Nikon)||Staring||CCD||550 to 1000||2 to 5 nm||15 ms/band||Blood cells, retina sections, nerve, lung||94 to 98|
|AOTF microscope||AOTF (ChromoDynamics)||Microscope (Carl Zeiss)||Staring||1317×1035 CCD||450 to 750||0.35 μm||Binucleated Swiss 3T3 cell, brain||99 to 105|
|Fourier transform imaging spectroscopy||Interferometer||Fluorescence microscopy||Staring||100×100 CCD||400 to 800||4 to 16 nm||30/Mμm||5 to 50 s||Fluorescence, skin, karyotyping||106 to 110|
|MSIS||Color filter wheel||Camera lens||Staring||CCD/InGaAs camera||Red, Green, Blue, NIR||Skin, ovary, tooth||111 to 113|
|Hyperspectral fundus camera||LCTF (CRI)||Fundus camera (Canon, CF-60z)||Staring||CCD||500 to 650||2 nm||10 to 15 min||Retina||114 to 119|
|IRIS||Wollaston prism polarizing beam splitters||Fundus camera (Canon, CF-60z)||Snapshot||CCD||575 to 615|
|CTIS microscope||CGH disperser||Microscope (Olympus IMT-2)||Snapshot||CCD||450 to 750||10 nm||1 μm||RIN cells||120, 121|
|Spectroscopic sensitive multiaperture camera||Lenslet array and filter array||Fundus camera||Snapshot||1392×1040 12-bit CCD||540, 560, 576, 600, 680||20 nm||200 ms||Retina, skin||122 to 124|
|ISS/IMS||Image mapper and prism||Microscope (Zeiss Axio Observer A1)||Snapshot||4096×4096 CCD||450 to 650||5.6 nm||0.45 μm||Fluorescent beads||12, 15, 125|
Note: CGH, computer-generated-hologram; CTIS, computed tomography imaging spectrometer; DLP, digital light processing; IMS, image mapping spectrometer; IRIS, image replicating imaging spectrometer; ISS, imaging slicing spectrometer; MPHI, molecular hyperspectral imaging; NIRSI, visible, near-infrared, and long wavelength spectroscopic imaging; PGP, prism-grating-prism; PMT, photomultiplier tube; SPLE, spectrally programmable light engine; VIF, variable interference filter.
Current Biomedical Spectral Imaging Systems
Whiskbroom and pushbroom systems
Whiskbroom (point-scanning) and pushbroom (line-scanning) biomedical spectral imagers generally use the spatial scanning configurations with prisms, gratings, or beam splitters to create spectral discrimination. These kinds of imagers are popular for remote sensing applications as an airplane or satellite flying over the surface of the Earth provides a natural scanning procedure. The biomedical applications of whiskbroom and pushbroom systems are mainly used in microscopic forms because the scanning procedure can be carried out by moving of the microscope stage, and the acquisition time is usually not a problem for some microscopic observations. In the past decades, various microscopic multispectral or HSI systems have been developed and used in biomedical engineering. Huebschman et al. developed a prototype HSI microscope by coupling a grating-based SpectraPro 556i spectrograph (Acton Research Corp., Acton, MA) to a microscope. The system allows measurement of wavelengths approximately from 400 to 780 nm by a -pixels Photometrics-Synsys 1600 air-cooled CCD camera (Photometrics Inc., Tuczon, AZ) with the stage incrementally moving in 0.1-mm or larger intervals.13 This is the earliest report about the development of the pushbroom microscopic HSI system. Then they analyzed the characteristics and capabilities of this kind of system and tested it with a range of samples relevant to cytogenetic, histologic, fluorescence in situ hybridization (FISH), cell fusion, and tumor cells in their recent studies.4243.–44 Some commercialized PGP-based imaging spectrograph also can be used to design a HSI microscope. For example, Li et al. have developed the microscopic pushbroom HSI system by coupling an ImSpector imaging spectrograph (ImSpector series) produced by Spectral Imaging Ltd. (Oulu, Finland) to a microscope.4748.–49 Some researchers also developed a short-wave infrared hyperspectral pushbroom imaging system for quality control of herbal medicines.126 Another kind of spatial scanning systems, the spectral confocal microscopes, also have been developed and widely used in biological analysis. The spectral confocal microscopes were originally developed in different laboratories as application-specific instruments in early 1990s.127,128 Since then, a number of point-scanning and line-scanning spectral confocal microscopes have been introduced that expand upon these capabilities by allowing the acquisition and unmixing of data from multiple fluorescent probes with overlapping fluorescence emission spectra.129130.–131 Some commercial spectral confocal microscopes also have been provided by different manufacturers such as Leica (TCS SP), Olympus (FluoView), Nikon (C1si), and Zeiss (LSM). Although each of them has their own spectral detector design for detection of wavelength-dependent signals, three different types of spectral units, that is, dispersive prism, diffractive grating, and multichannel array, are usually used by these instruments. Recently, some new types of spectral confocal microscopes have been proposed, such as the wide spectral range confocal microscope,132 the multispectral line confocal imaging microscope,9 the confocal-spectral microscope,133 and the multichannel spectral imaging laser scanning confocal microscope.134 These types of equipment have the capacity to obtain unique fluorescence spectra of specific dye molecules that will increase our understanding of the biological fluorescence emission process. Also, some other similar microscopic whiskbroom or pushbroom systems have been presented in recent studies, such as the hyperspectral fluorescence lifetime imaging microscope,45 the Hadamard transform spectral microscope,31 the hyperspectral microscopic system,51 the prototype open-frame MACROscope,52,53 etc. Besides microscopes, some other optical devices such as colposcope, ophthalmological lens, endoscope, and camera lens also have been coupled to the monochromator-based imaging spectrograph to form different kinds of whiskbroom or pushbroom biomedical spectral imaging systems.24,34,36,37,41,54,55,135,136 In addition, some spectral confocal scanning laser ophthalmoscopes also have been developed for retinal imaging.137,138 One of the advantages of the pushbroom and whiskbroom imagers is that they can scan the target in one or two spatial dimensions, which makes the imaging range of these two modes larger than the staring and snapshot ones. Therefore, the spatial scanning imager is suitable for coupling with a microarray scanner to analyze spotted microarrays. The pushbroom hyperspectral microarray scanner has the advantage of scanning an entire microarray at high spectral and spatial resolutions, allowing the identification and elimination of unwanted artifacts. It also improves the accuracy of microarray experiments greatly.13,25,26 There are also some disadvantages with traditional spatial scan imagers—low light throughput due to slit caused vignetting, nonuniform efficiency, being at maximum at the blaze wavelength, limited spectral range due to the grating order superposition, etc. In addition, some grating-based monochromators coupled with microscopes will require very bright light sources in order to compensate for the low throughput of the grating and complicated calibration methods.78
Staring biomedical spectral imaging systems
The staring biomedical spectral imagers use spectral filters (e.g., filter wheels containing fixed bandpass filters, AOTF, and LCTF) to acquire a single-band 2-D gray image with full spatial information and form a spectral data cube by scanning in the spectral domain through a number of wavelengths. The most basic implementation of staring spectral imaging is the use of a rotatable disk called a filter wheel carrying a set of discrete bandpass filters. Afromowitz et al. have developed a color filter wheel–based multispectral imaging system to evaluate burn depth on human palms and feet.112 This is the earliest study on the staring biomedical spectral imager and its applications. In recent studies, similar systems have been developed for quantitative analysis of the human tooth and ovary.111,113 The advantage of these filter wheel–based imagers is that they are easy to use and relatively inexpensive. However, they also have some disadvantages, such as slow wavelength switching, narrow spectral range and low resolution, mechanical vibration from moving parts, and image misregistration due to the filter movement. Therefore, some other spectral filters need to be used in order to overcome these limitations. There are mainly two filters widely used in the staring biomedical spectral imager: AOTF and LCTF. AOTF is a rapid wavelength scanning solid-state device that operates as a tunable optical bandpass filter based on light–sound interactions in a crystal. The major function of AOTF is to isolate a single wavelength of light from a broadband source in response to an applied acoustic field. The main advantages of AOTF are good transmission efficiency, fast scan times, large spectral range, and flexible controllability and programmability. AOTF is also attached easily to a microscope, endoscope, and camera lens. Schaeberle and Treado, et al. have developed multispectral and HSI microscopy based on AOTF.139,140 Then, Wachman et al. developed an AOTF microscope to map hemoglobin saturation and oxygen tension in mouse brains.99,100 Since then, more AOTF-based microscopy HSI systems have been proposed and used for different biological analysis, for example, the HSI system,86 the molecular HSI (MHSI) system,96 the HSI microscope,141 etc. Besides microscope, endoscope and camera lens also have been attached to AOTF to detect biological tissues in vivo. For example, some AOTF-based endoscope spectral imaging systems have been developed and used to detect skin tumors, thorax surgeries, and cancerous tissues.5,16,84,85,87,88 Furthermore, some camera lens with standard interface can be coupled to an AOTF adapter directly and used for in vivo imaging.8081.–82 The advantages of AOTF-based imagers include quiet and vibration-free operation, high switching speed, spectral selectivity, spectral purity, and flexibility. On the other hand, LCTF is a solid-state instrument that utilizes electronically controlled liquid crystal cells to transmit light with a specific wavelength and the elimination of all other wavelengths. The LCTF approach also has been used in coupling with the microscope,65,8990.91.–92 fundus microscope,114,117118.–119 and camera lens5657.–58,60,62,64,6869.–70,77 to create spectral image cubes for biological imaging in vivo and in vitro. Although the electronic controller of the LCTF is able to shift the narrow bandpass region throughout the entire wavelength range covered by the filter unit, the wavelength switching speed depends on the relaxation time of the liquid crystal as well as the number of stages in the filter. Typically, it takes tens of milliseconds to switch from one wavelength to another, which is far longer than the response time of the AOTF.17 The commercial AOTF and LCTF products also have been widely used to acquire biomedical spectral images in the visible (VIS) and near-infrared (NIR) spectral regions. For example, some researchers have developed different kinds of systems based on the AOTF component (Brimrose, Sparks, MD),5 the AOTF imaging modules (ChromoDynamics, FL),99,141,142 and the CRi LCTF (Cambridge Research and Instrumentation, Woburn, MA).91,92 In addition, some commercial instruments have also been manufactured based on these components, such as the HSi-300 and HSi-440C HSI systems (Gooch & Housego, FL), the Nuance,™ and Vectra™ multiplexing microscopes (CRi). Compared to the fixed interference filters used in the filter wheels, AOTF and LCTF can be flexibly controlled by a computer. They do not have moving parts and, therefore, do not suffer from the problems associated with the rotating filter wheels, such as speed limitation, mechanical vibration, and image misregistration. However, the AOTF- and LCTF-based imagers are in all cases expensive and have delicate and low throughput.12 Therefore, an interferometer also has been used for spectral imaging as it usually has high optical throughput, high and variable spectral resolution. In general, it is possible for Michelson interferometer and Sagnac interferometer to acquire 2-D spatial and one-dimensional spectral information of objects by applying a Fourier transform spectrometer algorithm and the Van Cittert-Zernike theorem. For example, Inoue et al. discussed the performance of Fourier-transform spectral imaging based on the Michelson interferometer in early 1990s. Potter et al. developed an FTIR system by coupling mid-IR Michelson-type step-scan interferometer to an IR microscope.143 Magotra et al. developed an hyperspectral biomedical imager imaging system using a spatially modulated Fourier transform interferometric device. This device is based on a Sagnac interferometer, which creates an interferogram that contains the desired spectral information.144 Fisher et al. obtained spectral images of a tooth by combining imaging optics, a Sagnac interferometer, and an imaging detector.145 Besides the Michelson and Sagnac interferometers, researchers found that Fizeau interferometer, based on double Fourier interferometry, can also be used for spectral imaging.146 In recent years, various interferometric spectral imaging systems and sensors have been developed and used for biomedical analysis, such as the spectral karyotyping (SKY), dynamic monitoring of biomolecular interactions, and live epithelial cancer cells analysis.20,109,147,148 Besides the interferometer, some other devices such as the rotating direct vision prism,149,150 VIF,78 CVF, LVF,42,151 and the dispersion of optical rotation152 can also be used to form biomedical staring imagers. Although the staring biomedical spectral imagers do not need to scan in the spatial dimension, most of them still need to scan in the spectral dimension. The scanning mechanism decreases their temporal resolution and limits their potential use in some real-time imaging. However, the staring spectral imaging systems are still the most popular ones in current biomedical applications.
Snapshot biomedical multispectral imaging systems
A snapshot biomedical multispectral imaging system can acquire both spectral and spatial information in a single exposure. This kind of imaging system can capture a scene of multispectral images faster than most of the spatial scanning and spectral scanning imagers. In recent years, researchers have proposed some methods for biomedical snapshot spectral imaging, such as the CTIS,120 image mapping spectrometer,125 coded aperture snapshot spectral imaging,153,154 image replicating imaging spectrometer (IRIS),114115.–116 imaging slicing spectrometer,12 and Fourier transform–based snapshot imaging systems.155,156 A major advantage of this kind of imager is speed. The faster speed makes it suitable for studies in endoscopy detection, fluorescent probes analysis, retina oxygen saturation measurement, and rapid processes that other spectral imaging instruments can hardly handle. This kind of imager has some other advantages, among them time-resolved spectral imaging of transient phenomena possibility, high reliability and robustness as a result of not having moving parts, high signal throughput as no multiplex losses, etc.116 However, the dimensions of the spectral data cube obtained by snapshot imagers depend on the size of the CCD sensor they used, as the total number of voxels cannot exceed the total number of pixels on the CCD camera. Therefore, the spatial resolution and spectral resolution trade-offs of the system need to be considered according to different biomedical applications. Although improvements are needed to address the issue of spatial or spectral resolution, the snapshot system can capture hypercubes without any scanning that represents a new trend in instrument development for biomedical spectral imaging techniques.
In addition to the different kinds of experimental spectral imaging systems developed by researchers in laboratories, some commercial biomedical spectral imaging systems have also been produced in recent years. Table 4 lists some commercial biomedical spectral imaging systems manufactured by different companies. Most of these systems were developed based on the pushbroom and staring imaging mode and were designed to meet a wide range of biomedical applications. Some systems were also designed for a particular application that can perform in-depth analysis on a specific tissue or disease with custom software. These commercial systems generally have modular hardware and friendly man–machine interface software, which make them more reliable than those experimental systems developed in laboratories, and these systems are suitable for doctors and medical researchers with little technical background and expertise. For example, the DySIS™ (Dynamic Spectral Imaging System, DySISmedical, UK) has been used to discriminate high- from low-grade cervical lesions and non-neoplastic tissues.157,158 The Nuance™ Multispectral Imaging Systems (CRi) has been used to assess blood flow and oxygenation in implantable engineered tissues,159 and to evaluate morphological distributions of miRNAs and their putative targets in thin tissue sections with the inForm™ image analysis software.160 However, most of the commercial softwares generally provide basic spectral image processing and analysis functions, and can hardly meet some particular research requirements. Therefore, the secondary software development is often needed by the users to implement their new analysis algorithms. The Gooch & Housego, Inc. considered this problem and has offered an open-source application software platform for imaging and control of automated microscopes on multiple operating systems (Windows, Mac, and Linux). Surface Optics Inc. also provides the HyperSpect™ cross-platform operating software, which can export data with a format supported by the third-part software such as ENVI or MATLAB. This characteristic is very important for the commercial systems in promoting the research implementations in different biomedical areas.
Some commercial biomedical spectral imaging systems.
|Products||Dispersion elements||Imaging mode||Wavelength range (nm)||FWHM (nm)||Software||Applications||Manufacturers||Website|
|C1si Spectral Imaging Microscope System||Gratings||Whiskbroom||400 to 750||2.5/5/10||Customized C1si system software||Fluorescence photons, fluorescence redistribution after photobleaching (FRAP) observation||Nikon||http://www.nikon.com/|
|LSM series||Multichannel array||Whiskbroom||Selectable||1||ZEN Efficient Navigation software||Multiple fluorescence, live cell imaging, FRAP, fluorescence lose in photobleaching||Carl Zeiss||http://corporate.zeiss.com/|
|CytoViva® System||Diffraction grating||Pushbroom||400 to 1000||2.8||Customized software, a modified version of the ENVI||Drug delivery, toxicology, fluorescent in live cells||CytoViva||http://www.cytoviva.com|
|710VP Imager||Grating||Pushbroom||400 to 1000||4.7||HyperSpect™ operating software||Biological analysis||Surface optics||http://surfaceoptics.com/|
|PARISS® System||Curved prism||Pushbroom||365 to 920||1||PARISS software, developed in a LabVIEW environment||Histology, histopathology, CFP/YFP FRE, fluorescent||LightForm||http://www.lightforminc.com|
|Benchtop Scanning System||Prism||Pushbroom||400 to 1000||2.4||Spectronon Software, users can write their own plugins||Pharma, ischemic wounds, multiple dyes identification||Resonon||http://www.resonon.com|
|HSi-300/HSI-440C||AOTF||Staring||450 to 800||1.5 to 3||Open source application software platform on multiple OS||Immunohistochemistry, QDs, FISH, FRET, SKY||Gooch & Housego||http://www.goochandhousego.com|
|Nuance™/Vectra™ System||LCTF||Staring||420 to 720 450 to 950||3 to 10||inForm™—Intelligent Image Analysis Software||In vivo fluorescence, QDs, overstained specimens||CRI||http://www.perkinelmer.com/|
|IRIS™ retinal imaging system/IMA™||Bragg gratings||Staring||400 to 1000||2||Particularly developed for ophthalmological studies||Retina, dermatology, mapping of copper indium gallium selenide solar cells||Photon etc||http://www.photonetc.com/|
|DySIS™ Imager||VIF||Staring||400 to 1000||7.5||Particularly for cervices studies||Cervix diseases detection||DySISmedical||http://www.dysismedical.com/|
|SpectraCube system||Interferometer||Staring||400 to 1000||5 to 20||SKYView software can be upgraded with add-ons||Karyotyping, fluorescence, skin, cells, tissues||ASI Inc.||http://www.spectral-imaging.com/|
|VNIR-20/VNIR-90 hyperspectral imager||HyperPixel™ (HPA™)||Snapshot||450 to 675 500 to 910||11||Analysis software compatible with ENVI||Retinal imaging, colonoscopy, and skin cancer detection||Bodkin Design & Engineering||http://www.bodkindesign.com/|
It is important to evaluate the performance of a biomedical spectral imaging system since different applications usually require different performance parameters. The main performance parameters of a biomedical spectral imaging system generally include spatial resolution, spectral range, spectral resolution, dynamic range, sensitivity, etc. These parameters depend mainly on the capability of the optical system (the optical head, the spectral dispersion element) to produce an undistorted, unaberrated, and intense data cube of a scene, as well as the ability of the detector to convert an image into a digital form with low noise and fine spatial or spectral sampling.161 Some performance parameters of current biomedical spectral imaging systems have been listed in Table 3. A brief discussion of the most important performance parameters and their relationship with the applications follows.
In biomedical imaging, the spatial structure information is one of the important factors for histological analysis and disease diagnosis. The inclusion of spatial information can increase the biological information content derived from data acquired with spectral imaging systems. As shown in Fig. 4, different biological objects usually have different spatial scale requirements for detection. Spatial resolution of a spectral imaging system determines the closest distinguishable features in the objects. With a proper selection of the system configuration, useful spatial information and quantification can be achieved by sufficient biological sampling, as different spectral imaging methods usually have different spatial resolutions ranging from a few micrometers to a few centimeters. For example, cells expressing each color of fluorescent protein can be located and clusters of cells expressing two or three colors usually reinforce the need for increased spatial resolution () to facilitate single cell detection.26 A spectral imaging system with spatial resolution of 0.45 mm and 17-cm-diameter field of view (FOV) is sufficient for hemoglobin oxygen saturation detection in sickle cell disease patients and in some other clinical investigations.70,73 Therefore, proper selection of spatial resolution is important for different kinds of biomedical applications.
Spatial resolution defines the level of spatial detail depicted in an image, which determines the closest distinguishable features in an object. The theoretical spatial resolution mainly depends on two factors: the wavelength () and the numerical aperture (NA) of optics. In addition, the pixel size of the detector and the magnification also play a role because they determine the sampling frequency, which must be sufficiently high to achieve full resolution.162 The spatial resolution of most spectral imaging systems can dominantly deteriorate by astigmatism, which is an off-axis aberration that increases with the square of both the off-axis angle and NA.163 Significant loss in spatial resolution due to prism mount misalignment in a chromotomosynthetic HSI system has been analyzed by Bostick et al. They found that the relative effect is less in the shorter-wavelength region, which indicates that spatial performance is also affected by wavelength.149 However, astigmatism is not a problem in a spectral confocal laser scanning system because the laser submits single points from the FOV sequentially, not multiple points simultaneously. For a line-scanning spectral imaging system, the spatial resolution in the direction of the slit width is affected by the slit width, whereas the spatial resolution along the slit length is usually affected by the magnification and camera pixel size,44 while for a spectral filters-based imaging system, the spatial resolution is also related to the types of tunable filters (AOTF, LCTF, etc.) the system used.4,70,87 Currently, spectral images acquired by microscopic biomedical spectral imaging systems are usually in the order of 0.3 μm or finer, whereas those systems coupled with camera lens generally acquire higher spatial resolution data, usually in the order of 0.45 mm or finer.44,70
There is also another problem to consider in the bright-field light microscopy and fluorescence microscopy combined with spectral imaging function. That is, how to produce high spatial resolution or clear images of a focal plane deep within a thick tissue. For example, a fluorescent cell might be 5 to 15 μm thick, whereas the thickness of the imaging plane of a high-NA objective () is only or less. Thus, the vast majority of the cell volume is out of focus and becomes blurred.164 Optical sectioning may be one of the approaches used in solving this problem. It can get better background rejection, improved contrast, and more accurate spectral characterization by reducing out-of-focus artifacts. Compared to common imaging with wide field, optical sectioning generally led to greater than improvement in the rejection of out-of-focus fluorescence emissions and greater peak-to-background ratios in biological specimens, yielding better contrast and spectral characterization, which is important in unmixing signals in biological samples and suppressing a phototoxicity and bleaching of fluorophores.165 The commonly used methods for optical sectioning are confocal laser scanning microscopy and two-photon microscopy. These methods are powerful means to eliminate the background caused by out-of-focus light and scatter from images and are well matched to the needs of spectroscopy, providing both convenient formats and high optical throughput. Conchello et al. and Svoboda et al. have summarized these techniques for optical imaging of thick tissues.164,166 Some new optical sectioning techniques, such as planar illumination and structured illumination have been proposed in recent years. These methods are particularly good for applications involving high speeds, large FOVs, or long-term imaging, which have been reviewed by Mertz.167 Structured illumination is a way to make it possible to perform optical sectioning with a conventional microscope, which is generally radiated through a grid of horizontal lines and is then reflected off a beam splitting mirror. Then the image of the thick sample with both wide-field and optical section containing grid lines passes back through the beam splitter to the camera. Thus, the optically sectioned images can be reconstructed by some algorithms from a conventional wide-field microscope.168 Most of these techniques can also be used for spectral imaging of thick tissues. For example, Oruanaidh et al. proposed a Bayesian spectral analysis method to obtain optical sectioning using structured light imaging in a conventional wide-field microscope.169 Webb et al. presented a wide-field fluorescence lifetime imaging system with spectral resolution and optical sectioning to achieve five-dimensional fluorescence images of biological tissues.170 In addition, the digital micromirror device also has been employed as a spatial excitation light modulator for either spectral imaging or fluorescence lifetime imaging.171,172 In recent years, single-point optical sectioning systems for microspectroscopic analysis have been reported, some of which have been adapted for scanning, particularly for Raman spectroscopy.173 The optical sectioning spectroscopic systems using slit scanning have been applied both for transmission and Raman spectroscopic measurements also.174 The programmable array microscope based on programmable spatial light modulators, developed by Hanley et al., is another kind of optical sectioning microscope of particular interest for spectroscopy.165 These studies show that optical sectioning has improved spatial resolution in the intensity images by removing out-of-focus blur. With optical sectioning, a high spatial resolution image generally demands a greater signal-to-noise ratio (SNR) than a conventional intensity image does. This will make high S/N ratios imperative when applying the technique to thick biological tissues.175 Therefore, suitable specification of the optical system is generally needed to produce optimal trade-offs between reducing the speckle noise and preserving the optical sectioning to get clear images. Some of these studies also demonstrate that the spectral imaging technology combined with optical sectioning is essential for spectroscopic characterization. For example, the spectral signature changes of tissues are generally associated with their pathologic states. The ability to localize these changes within cells and tissues will improve the diagnostic capabilities of such method. In some cases, the localization of probe molecules within a specimen requires that they can be distinguished from sample autofluorescence or from out-of-focus probe-fluorescence. Therefore, it is important for the spectral imaging microscope to reduce both types of background through the use of optical sectioning methods to improve the detection and accurate localization of targets.165 However, there are no hard and fast prescriptions on which technique is most suitable for optical sectioning. The researchers must consider what factor is important in the biomedical analysis (such as speed, resolution, and FOV) as the optimization of one factor often leads to degradation of the others.167
Spectral range refers to the range of wavelengths over which the biomedical spectral imaging system can measure the spectrum. As shown in Tables 3 and 4, different biomedical spectral imaging systems generally work in different spectral ranges as the properties of the optics are different. The spectral range is usually given by the minimum and maximum wavelengths at which the spectral response exceeds some threshold (for example, 10%). Generally, it is determined by the kind of illumination source, optical head (such as microscope, endoscope, camera lens, etc.), spectral dispersion element, and detector that the system uses. First, the transmission of a microscope objective, endoscope optics, and camera lens will limit the wavelengths of the incident light.152 Second, different illumination sources and spectral dispersion elements can only work in different wavelength ranges. For instance, a single AOTF unit generally covers a specific wavelength range, as the components for constructing it have different characteristics, which can only accommodate a particular spectral region. To acquire spectral images that cover the spectral region from 450 to 2500 nm, three different commercial imaging AOTF modules (Gooch & Housego) working at different wavelength ranges (i.e., 450 to 800 nm, 900 to 1700 nm, and 1300 to 2500 nm) are needed. At last, the spectral response of the detector is also an important characteristic that determines the working wavelength range of the system. For inspection of biological tissues using spectral information in the VIS and short-wavelength NIR regions, the silicon-based CCD detectors have been used in reflectance and transmittance spectral imaging systems.4,44,111 For an NIR spectral region (900 to 2500 nm), the InGaAs image sensor, which is made of an alloy of indium arsenide (InAs) and gallium arsenide (GaAs), is usually used.82,113,176 With the exception of the CCD and InGaAs spectral range, system cost will be significant primarily due to the required customization. In the visible and near infrared (VNIR) spectral range, CMOS cameras can be used, but only for the low-end applications.
According to the electromagnetic theory, different biochemical constituents, biochemical changes, and labels generally have different spectral signatures in a certain wavelength range.177 Therefore, a proper spectral range selection is usually needed for a particular analysis purpose. For example, to map the oxygen saturations in living tissues, a spectral range (such as 500 to 800 nm) which can cover the spectral signatures of and Hb can usually be used for in vivo imaging. For fluorescent analysis, a spectral range from 400 to 750 nm is usually selected as the excitation spectral signatures of most of the fluorescent probes are located in this wavelength region. Another wavelength region from 900 to 1700 nm is usually used for teeth diseases diagnosis, as dental enamel manifests high transparency for the light in the NIR spectral region. In general, flexibility in selecting the optimal spectral range for a particular application is important. We will discuss this in the following sections.
Spectral resolution describes the ability of a biomedical spectral imaging system to distinguish two signals that are separated by a small spectral difference. It can be represented by the full width at half maximum (FWHM), which can be checked by measuring the spread in the spectrum of a very narrow source.178 The spectral resolution can be easily determined in any instrument by using a light source that emits a spectrum with a pure monochromatic line. Spectral resolution of a spectral imaging system generally depends on the type of the spectral dispersion element that the system uses. As shown in Tables 3 and 4, most grating-based pushbroom spectral imaging systems have the higher spectral resolution, as it is mainly determined by the slit width and the optical aberration. For those AOTF-based spectral imaging systems, the spectral resolution is uneven across the whole wavelength range. Shorter wavelengths generally have higher resolutions than longer wavelengths.17 For example, the corresponding spectral resolutions of an AOTF adapter with spectral range from 550 to 1000 nm are generally in the range of 2 to 20 nm.84,96,179 For an LCTF-based system, the spectral resolution is typically of the order of several nanometers, although a narrower bandpass can also be constructed.180 In some applications, the spectral resolution that is needed is limited, or there is a need for only few wavelengths that are well known. For example, diagnostic imaging might be fine with tens of wavelengths over a visible range, but 10 signaling probes would require 30 wavelengths with 2 to 4 nm spectral resolution, according to the theoretical analysis and experiments proposed in Refs. 181 and 182. In such a case, it is possible to use a rather simple method that splits the few spectral bands of interest to the different regions in space, so that the whole information can be measured simultaneously. For instance, a snapshot imager usually can work in a broad wavelength range with high spectral resolution at the cost of scarifying the spatial resolution, yet it is suitable for retinal imaging and detecting fast metabolic processes.
There are also some other performance parameters to consider in the spectral imaging system, such as photon collection efficiency, data acquisition time, dynamic range, etc. The photon collection efficiency can evaluate the light level that the spectral imaging system can achieve. The photon collection efficiency of a spectral imaging system is usually different for each wavelength as the illumination, the NA of objective, the spectral response of the detector, and the transmission of the optics are not uniform. The sources detected by the spectral imaging systems tend to be weak and spatially incoherent, which makes the photon collection efficiency very critical in biological imaging. However, some spatial or spectral scanning spectral images usually have low photon collection efficiency for incoherent sources, which leads to a very small absolute number of collected photons. For example, the pushbroom spectral imager images only one slit at a time and will reject the out-of-slit photons. Therefore, a given element in a scene of spectral image tends to contain very few photons and hence has a poor SNR.28 The emission spectra in fluorescence measurements are usually inherently noisy when compared to broadband measurements as the narrow-band filtering required to image the fluorescence emission at many discrete wavelengths naturally results in decreased signal strength.141 In response to this problem, researchers have proposed theoretical analysis and a number of different designs to maximize the light gathering efficiency of the systems. R. Neher and E. Neher indicated that there are theoretical problems for the highest possible photon collection efficiency originating from the fact that both emission and absorption spectra of simultaneously present dyes’ overlap and their signal quality is limited by photon shot noise, as well as by photobleaching. They also defined the figure of merit (FoM) to measure the efficiency of a given system or measurement configuration.181 Some high–photon collection efficiency or high-throughput spectral imaging systems also have been developed for working with the weak, incoherent sources common to biomedical applications.19,28,183 These studies show that there are generally some ways to compensate for the photon collection efficiency, such as the use of a higher–quantum efficiency detector and a high-power light source to increase the exposure time. However, long exposure time may not be appropriate in applications where the scene is changing on a time scale of the acquisition.180 The exposure time also influences the data acquisition time, another parameter that depends on the effective single band time and on the total number of bands that should be measured. One of the fair comparisons of different acquisition methods can be based on the data acquisition time in which each method obtains information in order to satisfy the requirement of the measurement. As shown in Table 3, the snapshot imaging mode usually has the shortest data acquisition time as it does not need to scan to capture multispectral images. However, the data acquisition time of the whiskbroom mode is generally longer than other modes since it needs to scan in both and dimensions. In general, the measurement of spectral images requires significantly more time than the time required in acquiring a single frame using a traditional optical imaging system since the spectral images must be measured at different wavelengths. The time limitation of spectral imaging technology can be critical: for example, the high throughput that is required in many biomedical applications, the photobleaching influence in fluorescence, and the phototoxicity or image motion in applications such as retinal imaging. The dynamic range of a spectral imaging system refers to the largest difference in the intensity of incident light that can be observed in one scene of the spectral image, which depends largely on the dynamic range of the detector the system uses. In addition, due to the temperature and illumination fluctuation, the performance of the spectral imaging system usually gets worse in a real working condition. There are some methods that can be used to improve the system performance. One method is to add collimating lenses in front of each detector to reduce the wavelength shift and improve SNR. Another one is to use a cooling system to stabilize the temperature fluctuation. However, these methods will make the system more complex and cumbersome.151 The balance between system performance and complexity should be considered when developing a biomedical spectral imaging system.
System Calibration and Data Preprocessing
After the spectral imaging system was developed, one of the important steps was to ensure that the system would provide an accurate measurement of the spectrum. That is, the spectral imaging systems need to be calibrated and corrected to produce a flat spectral response before further experimental measurements. These steps are important for ensuring reproducible results and for comparing spectra across multiple systems.141 One way to determine the wavelength accuracy of the spectral imaging system is to calibrate it with a low-pressure discharge lamp that covers the wavelength range from 365 to 850 nm. However, the pure lamp usually emits deep UV light that is dangerous to the exposed skin and can cause blindness to unprotected eyes, thus making the measurement difficult to perform. Therefore, an eye-safety, multiion discharge wavelength calibration lamp (MIDL) distributed by LightForm Inc. can be used for wavelength calibration from 365 to 912 nm simultaneously.163 Lerner and Zucker have proposed the MIDL calibration method in detail and have derived reference spectra from the MIDL data to accurately predict the spectral resolution, ratio of wavelength to wavelength, contrast, and the aliasing parameters of any confocal spectral imaging system.184 This method has also been used by Leavesley et al. to calibrate the HSi-300 HSI system (ChromoDynamics, Orlando, FL).141 In addition, some other spectrally well-known light sources, such as lasers (for example, 632.8 nm by HeNe lasers), fluorescent lamps, and broadband lamps equipped with interference bandpass filters, can also be used for wavelength calibration.17 Other calibration methods also have been presented recently. For example, three National Institute of Standards and Technology traceable Spectralon standards (Labsphere, North Sutton, NH) of different colors have been used to calibrate the multiaperture system, which can capture six identical images of the human fundus at six different spectral bands.122 Xu et al. developed a digital tissue phantom (DTP) platform for quantitative calibration and performance evaluation of spectral wound imaging devices and demonstrated the feasibility of such DTP platform by both in vitro and in vivo experiments.136 After the spectral calibration process, the wavelengths for the pixels along the spectral dimension of the data cube can be defined and the relationship between the true spectral position of the incoming light and the observed effect can be determined for further processing.
Besides the spectral calibration, a data preprocessing procedure is also needed to remove the effects of noises and artifacts and make the images suitable for quantitative analysis. During the spectral image acquisition, the noise counts accumulated on the detector may influence the pixel values beyond the true intensities. Various image artifacts can be generated by factors such as nonuniform illumination, dust on the lens surface, and pixel-to-pixel sensitivity variations of the detector.17 These effects in the original data can be corrected by some flat-field correction methods. In the past decades, different data correction methods have been proposed along with the development of the biomedical spectral imaging technology. Most of these methods utilize the radiance of standard white diffuse reference panels (for example, WS-1, Ocean Optics, FL) and the dark current data for the correction, which can be performed using the following equation:83,122,141,176,18568 Although this equation is proposed for reflectance image correction, transmittance image data can also be corrected similarly. For example, a thin diffusing glass slide with transmittance has been used to correct the transmittance images captured by spectral imaging microscopy.48,91,96 Then, the relative (or percent) reflectance instead of the absolute intensity data can be acquired for further data analysis.
Some commercial software has also been developed for biomedical spectral images analysis. For example, the CytoSpec program developed by Peter Lasch is a specialized software package with a large number of different functions, such as data import, spectral and spatial preprocessing, and uni- and multivariate image segmentation.186 Some other software associated with the commercial spectral imaging hardware also provide the basic calibration and preprocessing functions, such as the inForm™ intelligent image analysis software developed for the Nuance™ and Vectra™ multispectral imaging system, the HyperSpect™ cross-platform operating software for the 710VP Hyperspectral Imager, and the PARISS® software for the PARISS® system.
Biomedical Applications and System Requirements
Spectral imaging systems are potentially useful for those applications in which subtle spectral differences exist in chemical constituents within a scene of spectral images. It has enabled applications in a wide variety of biomedical engineering studies ranging from individual biochemical species observed in in vitro samples to organs of living people. This section will concentrate on biomedical applications involving microcosmic and macroscopical, in vivo and in vitro. Some of the pioneering research works are mentioned here, but it is by no means a complete list.
In Vivo Spectral Imaging
In vivo optical imaging allows a noninvasive observation of living organisms and helps in understanding metabolic processes and disease-related changes in the body. As one of the optical diagnostic technologies, spectral imaging has been used for in situ tissues diagnosis in recent years. Among the openly accessible tissues, skin is one of the most suitable ones for in vivo spectral imaging. Afromowitz et al. developed a multispectral imaging system to evaluate burn depth of skin.112 They found that the spectral imaging method was more accurate in predicting burn healing than the traditional method. After that a spectral image-analysis system based on Fourier transformed spectroscopy combined with image processing has been used for the in vivo study of porphyrin localization in human skin lesions.107 These are some earliest studies on in vivo analysis on skin. Then Sowa et al. evaluated the hemodynamic changes in the early postburn period of skin by NIR reflectance spectroscopy and imaging.59,60,64 Cancio et al. further presented a study in which a spectral imaging system was used to assess the cutaneous manifestations of significant systemic events.67 Martinez proposed a noninvasive spectral reflectance method for mapping blood oxygen saturation in various types of wounds, ranging from burn injuries and diabetic-related ulcers, to skin grafts.187 Some other studies include determining skin ulcers,188 and spectral imaging of cancer skin fluorescence and reflectance.5,16,22 In vivo confocal scanning laser microscopy has also been used to analyze the skin annexes, as well as cutaneous cells from different epidermal layers by both reflectance and fluorescence mode.189 More recent studies focused on 3-D HSI on small human skin areas,68 quantification of skin oxygen saturation and its relation to the traditional measures of wound healing,123 subcutaneous adipose tissue observation,176 ischemic wound assessment,136 and breast tumors.142 These studies show that noninvasive in vivo detection of skin surface using spectral imaging technology may be of value in the early assessment of burn injury and skin tumors.
Another important in vivo application of spectral imaging is retina detection. Some earlier studies focused on the development of the retinal spectral imaging system coupled with a standard ophthalmic fundus camera.144,190 Some researchers also discussed the analysis methods in detecting and characterizing the extent of retinal abnormalities according to the spectral images.41,191,192 A major disadvantage of the spectral imaging systems used in these studies is that they are fundamentally unsuitable for recording phenomena that are changing on a timescale that is shorter than the duration required in reconstructing the spectral data cube. Therefore, the snapshot spectral imaging technology has been proposed and used to capture retinal spectral images in recent studies. Harvey et al. developed an image replicating imaging spectrometer to capture spectral images of the retina and proposed a physical model that enables the oximetry map of retina.114,116 Johnson et al. and Ramella-Roman et al. obtained oxygen saturation maps of retina with a three-wavelength algorithm and found that the oxygen saturation were for healthy subjects’ arteries and 30 to 35% less for the veins.18,122 Although the snapshot mode can capture the retinal spectral imaging quickly, the spatial and spectral resolutions are limited. In recent studies, LCTF-based retinal HSI systems have been developed to calculate the oxygen saturation of retinal vessel and detect retinal lesions.117,118,193 These studies show that a spectral imaging technique can be adapted to measure retina in vivo and map relative oxygen saturation in retinal structures. The confocal scanning laser ophthalmoscopy has also been used to evaluate retinal perfusion in the human eye.137,138 In addition, it can be combined with high-resolution spectral-domain optical coherence tomography (OCT) for simultaneous multimodal imaging of retinal pathologies, such as fluorescein and indocyanine green angiographies, IR and blue reflectance (red-free) images, fundus autofluorescence images, and OCT scans.194 Future work in this area will include clinical studies focused on spectrally characterizing retinal tissue, its diseases, and on the detection and tracking of the progression of retinal disease.118,191
The in vivo spectral imaging technology has also been used for tongue surface identification and oral cancer diagnosis. Li et al. introduced the spectral imaging technology to tongue diseases diagnosis for the first time.36 Yu et al. presented in vivo spectral images from a hyperkeratotic lesion on the ventral surface of the tongue to demonstrate clinical applicability.54 In recent studies, different kinds of HSI systems have been developed and used for tongue body segmentation, tongue color analysis and discrimination, tongue cracks extraction and classification, sublingual veins analysis, and malignant changes associated with oral cancer detection.4,3738.–39,80,195 These preliminary experiments show that the spectral oral imaging system is superior to the traditional CCD-based methods because it can provide more information about the oral surface. However, further studies are needed on how to extract the quantitative features of the tongue surface and model the relationship between these features and certain diseases.
Besides the exterior organs and tissues, some intracorporeal organs also can be detected in vivo by the spectral imaging system coupled with colposcope, endoscope, and gastroscope. Parker et al. developed a noncontact in vivo hyperspectral diagnostic imaging device for the detection and localization of cervical intraepithelial neoplasia.135 Balas described an LCTF-based multispectral imaging system for the in vivo early detection, quantitative staging, and mapping of cervical cancer and precancer.77 Commercial systems such as the dynamic spectral imaging colposcope (DySIS, Dynamic Spectral Imaging System; Forth Photonics, Livingston, UK) have also been developed and used to detect cervical lesions.157,158,196 Similarly, Vo-Dinh et al. developed an AOTF-based endoscopic HSI system and evaluated it by a model of bronchial tubes (APM Inc.) and esophageal cancer tissues.5,16,22,84,85 Arnold et al. evaluated the hyperspectral video endoscopy demonstrator during several measurements in a clinical surrounding for thorax surgeries.87 In recent study, this technique has also been used in gastrointestinal diagnosis.88 Although most of these studies were mainly concentrated on the system design, and only preliminary results have been obtained, they have demonstrated the possibility of inspecting inner regions of the human body with additional diagnostic information by spectral imaging technology. There are also some other in vivo applications of spectral imaging systems, such as hemoglobin saturation evaluation in tumor microvasculature and tumor hypoxia development,91 characterization of activity-dependent changes in oxyhemoglobim, deoxyhemoglobim, and light scattering in brain,197 analysis of the oxygenation and blood flow through microvessel networks,198 normal and malignant muscle tissues detection,84 tooth decay detection,82,113,185 and flow cytometry.86 Guidance of surgical intervention, treatment, and real-time assessment of tissue response to therapy is another promising application of in vivo spectral imaging.16,199 Especially with the development of clinically relevant quantum dots (QDs) contrast agents suitable for spectral multiplexing, in vivo spectral imaging can help the surgeon to delineate tumor margins and to detect residual tumor cells or micrometastases. Singhal et al. summarized the advances in nanotechnology and optical instrumentation and how these advances can be integrated for applications in surgical oncology applications, such as tumor localization, tumor margins, important adjacent structures identification, mapping of sentinel lymph nodes, and detection of residual tumor cells.200 From the review, it can be found that the combination of nanoparticles and in vivo spectral imaging has particularly promising opportunities in surgical oncology, and it will be an important research trend in the future studies.
In addition, some researchers also use the diffuse spectral imaging system to perform the in vivo diagnosis. This technology has been used for early diagnosis and screening of malignant changes in the oral cavity. Some experimental results show the potential use of this method in routine clinical practice for instantaneous screening of oral cancer.201,202 Prasanth et al. investigated the in vivo inflammation mapping of periodontal disease using the diffuse reflectance spectral imaging method and estimated the specificity and sensitivity of the in vivo technique in inflammation mapping.203 Recently, some new kinds of diffuse reflectance spectral imaging methods have been proposed and used for normal and malignant breast tissue–mimicking phantoms.204205.–206 All these studies show that the spectral imaging technology has the potential for in vivo diagnosis on diseases without the need for sample excision and histological processing. Both the spatial and spectral diagnostic information can be available noninvasively and in real time for medical diagnostic applications where rapid in vivo detection of complex samples is required. Therefore, this approach is expected to enhance the dynamic range of the image contrast in identifying the earlier changes of live tissues.
For the in vivo applications, the proper selection of wavelength range is important to match the identification of the object. For example, the spectral imaging method has been used to map the blood oxygen saturation in various types of surface wounds (such as burn injuries, diabetic-related ulcers, and skin grafts), retinal structures, and the blood vessels of a living animal as we have mentioned above. The main basis of this application is that the hemoglobin is found primarily in two forms in the blood, that is, oxyhemoglobin () and deoxy-hemoglobin (Hb). Light that emerges from the body is modified by the absorption of hemoglobin and contains information about the amount of oxygen contained in blood. There are usually different spectral signatures from 500 to 800 nm between the spectra of and Hb, which can be used for oxygen saturation detection.207 One of the wavelength regions between 700 and 800 nm can be used for mapping the blood oxygen saturation in skin surface as the states of hemoglobin at an isosbestic (800 nm) and oxygen-sensitive (760 nm) wavelength are different. In addition, it allows for deeper penetration of light at the longer wavelengths and minimizes the influence of other absorption chromophores below 630 nm.187 For the blood oximetry in the retina, wavelengths ranging between 500 and 650 nm can be selected. The reason is that the optical depth of changes from being less than to being greater than the thickness of the larger retinal arteries from 500 to 650 nm, and the reflectance spectrum of different locations across the retina is dominated by and Hb in both choroidal and retinal vessels, and by melanin components that vary depending on subjects and ethnicity.114,117 Another example is tooth diseases’ diagnosis using the spectral imaging method in the NIR spectral range. Studies show that dental enamel manifests high transparency (the mean free path of the photons being at 1310 and 1550 nm, while being for visible wavelengths) for the light in the NIR spectral region and the tooth caries process usually increases absorption and scattering of the incident light. Therefore, the NIR spectral range (for example, 900 to 1700 nm) is suitable for noninvasive high-contrast imaging of the caries located both beneath the enamel surface and on the surface between adjacent teeth.82,113 For the clinical study of the uterine cervix, spectral imaging and analysis show that the maximum contrast between acetic acid responsive and nonresponsive areas can be obtained at . Therefore, this spectral range can be selected to get the maximum diagnostic information for mapping the cancerous and precancerous lesions of cervix.208 These studies demonstrate that it is important to select the proper wavelength range for the spectral imaging system in the in vivo biomedical applications as different biological changes usually have different spectral signatures at different wavelengths.
Since spectral imaging technology can collect a set of images at continuous wavelengths revealing the spectral characteristics of multiple tissue components, the in vivo applications of this technology opens a new avenue for noninvasive and real-time assessment of the tissues. One advantage of the in vivo spectral imaging is that both the functional and the structural pathologic changes of the tissues can be detected and used for the early diagnosis and grading of diseases without the need of a sample excision. Another advantage is that the spectral imaging systems used for in vivo detection can support a wider wavelength range (including the NIR band) than those designed for histological applications, as the front optics are usually simple and easy to change. However, spectral imaging has a significant limitation when used for in vivo detection, that is, the penetration depth, due to the strong scattering of optical radiation in tissues. Therefore, those openly accessible sites (such as skin, retina, cervix, tongue, teeth, and tumor microvasculature) and endoscopy-accessible organisms (such as stomach and colon) are suitable for in vivo spectral imaging.
Fluorescence and Cytogenetics
In traditional fluorescence detection methods, it is necessary to excite all the dyes, but the emission spectral range cannot overlap with the excitation range. This means that part of the spectral range that is used for excitation cannot be used for detection. With the spectral information, spectral imaging technology makes it possible to resolve the spectra of fluorescence images without switching optical filters and unmix the cross-talk between overlapping fluoresence emissions.209 One of the most widespread applications of spectral imaging technology is SKY, which is based on FISH and revolutionized mouse and human cytogenetics. Schrock et al. developed a spectral imaging system based on Fourier spectroscopy and used it to resolve spectrally overlapping chromosome-specific DNA probes and identify all human chromosomes simultaneously.110 This is the earliest study on multicolor SKY of human chromosomes with citations since first published in 1996. Full karyotype single interphase cells have also been implemented by a rapid enumeration procedure based on the hybridization of chromosome-specific probes and spectral imaging analyses.210 SKY was recently used for evaluation of chromosomal rearrangements in post-Chernobyl papillary thyroid carcinomas,211 identification of constitutional chromosomal abnormalities,212 and characterization of giant marker and ring chromosomes in a pleomorphic leiomyosarcoma.213 These studies show that spectral imaging technology has the ability to identify and detect complex karyotypes, subtle chromosome translocations, and aberrations of chromosomes that have been extracted from a metaphase nucleus and hybridized with chromosomes probes, without spectra overlapping. Therefore, it is a valuable tool for the cytogenetic analysis of a broad range of chromosome abnormalities associated with a large number of genetic diseases and malignancies.214 There are also a number of different labeling strategies that have been combined with SKY, such as the fluorescent dyes [for example, Rhodamine, Texas Red, Cy5, fluorescein isothiocyanate (FITC), and Cy5.5], the fluorescent proteins, QDs, and some commercial products (such as the fluorescent polymer “Brilliant Violet” series, a set of FISH-probes developed by the Applied Spectral Imaging, ASI). With the appropriate spectral image capturing and analysis methods and tools (for instance, SpectraCube™ and SKYView™, ASI), these probes can represent all 24 chromosomes (SKY Paint) together.20,215
The spectral imaging technology also provides a unique capability for detecting spectral changes in fluorescent indicators [for example, an indicator for fluorescence resonance energy transfer (FRET)], microenvironmental changes in cellular compartments, fluorescence from natural pigments, and effective detection of multiple fluorescent labeling in biological tissues. For example, Haraguchi et al. studied the capability of the spectral imaging system for the spectral and temporal resolution of fluorescently stained living cells and found that the linear unmixing of the spectral images can separate the cyan fluorescent protein (CFP) staining of the nuclear membrane and yellow fluorescent protein (YFP) staining of STAT1 in some cases.216 However, the traditional FRET sensors were limited in the range of glutamine concentration that can be detected due to relatively low affinity, and the commonly used FRET pair [enhanced CFP and YFP (ECFP and EYFP)] is difficult to multiplex with other FRET pairs. Some new fluorophores, such as the teal fluorescent protein mTFP1, may overcome these limitations as it provides a narrow and single-peaked emission spectrum, improved brightness and photostability, and single exponential lifetime decay. The new FRET pair, mTFP1 and venus, has also been proposed and used to analyze the properties of transporters expressed in a given cell as it can improve quantum efficiency and pH stability compared to the more commonly used FRET donor, ECFP and EYFP.217 The mTFP1–venus FRET pair is suitable as a replacement for ECFP and EYFP and can be duplexed with another FRET pair, mAmetrine and tdTomato, opening up the possibility for real-time imaging of another molecule.218 The time course of fluorescence from a coral during a photobleaching experiment has been observed by using the spectral imaging method. It provides a hint of the photophysics of the intact coral leading to a better understanding of the likely individual functions of the many fluorescent and nonfluorescent proteins following a green fluorescent protein (GFP)-like motif isolated from different varieties of coral.219 Studies by Livet et al. show that different neuronal clones could be distinguished in the inventors of Brainbow mice according to color and intensity gathered by combining tissue/temporally inducible Cre recombinase, multiplex recombination sites encompassing multiple fluorescent protein gene cassettes, and spectral confocal microscopy.220 In addition, the spectral confocal microscopes have also been used for fluorescence evaluation and quantification in different tissues, such as brain slices, mouse ovaries, skin sections, kidney slice, and breast cancer tissues.134,221,222 Neher et al. demonstrated that quantitative FRET can be performed by taking advantage of steady-state intensity measurements, but it only allows determination of the apparent FRET efficiency, the relative concentration of donor, and the concentration of the acceptor.223 They also developed a nonnegative matrix factorization (NMF) algorithm to detect and separate spectrally distinct components of multiple labeled fluorescence images.182 In addition, Fisher et al. examined the fluorescence changes in the human oral tissues by a Sagnac Fourier transform imaging spectroscopy combined with a fluorescence microscopy.224 In cell signaling, Grant et al. presented an approach for simultaneous imaging of two different FRET sensors in the same cell with minimal spectral cross-talk, which can be used to report on both unimolecular FRET biosensors and protein–protein interactions in the same cell.225 Studies by Kawano et al. show that a combination of the coiled-coil tag-probe labeling method and spectral imaging can enable a stoichiometric analysis of the oligomeric state of membrane proteins on living cells by using monomeric, dimeric, and tetrameric standard membrane proteins.226 Draux et al. tested different optical substrates that would best preserve cell integrity and allow direct measurement of Raman spectra at the single living cell level. They also performed Raman spectral imaging to show the distribution of nucleic acids, lipids, and proteins within the cell.227 There are also some other applications that have been demonstrated in separating multiple fluorescence signals,120,141 separating autofluorescence and fluorescence signals,12,85 extracting dye distributions from the images,228 FRET imaging,53,229 and tissue imaging.45 These studies show that spectral imaging is best appreciated as a tool for detecting spectral changes in fluorescence and eliminating problems associated with autofluorescence. It extends the possibilities in discriminating distinct fluorophores with highly overlapping emission spectra, thus providing the possibilities of multicolor imaging.229
There are also some requirements for the spectral imaging system in detecting and separating fluorescence signals. Theoretical analyses on these requirements have been done along with the development of spectral imaging fluorescence microscopy. Garini et al. presented the SNR to evaluate the multiple-color fluorescence microscopy method for the particular case of SKY.230 R. Neher and E. Neher discussed the influence of photobleaching on the resolution and on the choice of imaging parameters for optimal fluorophore separation. They introduced a new measure called FoM for the efficiency of a given microscope or measurement configuration and found that a small number of spectral channels (two to four) is sufficient to extract spectral information and works almost as well as when using many channels, as long as the spectral ranges are optimally selected.181 They also provided theoretical analysis on how to separate fluorescence of FRET pairs from those originating from unpaired donors and acceptors by spectral fingerprinting and on how to select imaging parameters to optimize the SNR of the estimates.223 These theoretical analyses are very important for evaluating the fluorescence separation ability of spectral imaging fluorescence microscopy. With these theoretical analyses, R. Neher and E. Neher found that for the autofluorescent proteins ECFP, enhanced green fluorescent protein, and EYFP and two sequential exposures at 458 and 514 nm, the FoM is , which means that times more photons should be excited and detected to reach the resolution than if the dyes were not mixed and if their fluorescence could be measured without background noise.181 Quantitative analysis of fluorescently labeled cells in highly autofluorescent lung vasculature revealed that GFP may be detected when the GFP signal contribution is as little as 15% of the peak autofluorescence intensity.141 For a spectral imaging system with 7.2 nm wavelength resolution, at least a 90% confidence level can be obtained to resolve all fluorochrome dyes that have an FWHM and whose maxima are at least apart, such as Cascade Blue, 4’, 6-diamidino-2-phenylindole (DAPI), GFP, FITC, Rodamine 123, GFP, Cy3, tetramethyl rhodamine isothiocyanate (TRITC), Lissamine Rodamine, Texas Red, and Cy5.42 The number of the labels that can be used simultaneously generally depends on the excitation and emission paths, the spectral properties and stability of the fluorophores, quantum efficiency and the noise properties of the system and the amount of confounding autofluorescence, etc. In general, the degree of multiplexing is 4 to 5 in immunohistochemistry and in immunofluorescence.44,231 The spectral range selection of a spectral imaging system is also important in fluorophores separation, as it is essential in distinguishing the weak emitted light from the strong excitation light at different wavelengths. The fluorochromes have to be excited at certain spectral ranges, which limit the spectral ranges available for detection. For instance, better spectral unmixing of gastric tissues can be got by using excitation rather than using emission spectra. This is because wavelength-dependent variations in tissue absorption and scattering over the near infrared range are far less than in the visible range.232 Overall, a spectral range from 400 to 750 nm usually has been selected for fluorescent studies as the excitation spectrum of most of the fluorescent probes are located in this wavelength region. In addition, as time-dependent photobleaching will alter the measured spectral signature, there are also limits to the maximum acquisition time tolerated by the spectral imaging fluorescence microscopy. Wachman et al. has discussed the imaging speed and spectral versatility of AOTF microscopy in the investigation of millisecond and submillisecond dynamics in living biological systems using multiparameter fluorescence. They found that polystyrene beads brightly labeled with Cy3, Cy5, and both Cy3 and Cy5 can be separated according to the image set acquired in 3.5 ms.99 In general, there is a trade-off between achieving a high identification accurate through increased image acquisition time and minimizing spectral photobleaching artifacts. Therefore, the parameters should be properly selected according to the characteristics of different fluorescence probes.
These applications and analyses demonstrate that one of the advantages of spectral imaging compared to conventional microscopy methods is the increased number of distinct fluorescent molecules that can be simultaneously detected and distinguished. For example, by knowing the fluorescence spectra of each of the dyes, up to seven observed mixed spectra can be unmixed to each component by the spectral imaging technology.111 Another advantage is its discriminatory abilities, as the linear unmixing analysis often results in improved SNR of the system. Compared to single-channel images, there is commonly a two- to threefold increase in SNR after unmixing.141 In addition, the advent of spectral imaging will also provide an impetus for the development of a new class of fluorescent dyes. There are, however, some limitations induced by factors, such as image background and detector noise, and the appropriate selection of the number and bandwidth of the detection channels with respect to the overlap of the fluorescence to be distinguished. These limitations become most critical when imaging living cells, where the signals of interest are usually weak. So far it is parallel and thus extends not only to the abilities and performance of multicolor imaging, but also to the use of more efficient donor–acceptor fluorophore pairs for FRET imaging.160 Moreover, an inherent limitation is that quantification improvements rely heavily not only on prior knowledge of the absorption spectra of the tissue chromophores and the magnitude of their relative contributions. In cases where this information is not available, modeling errors can propagate into the inversion process, potentially negating any gains attributable to the spectral constraints.233 Another limitation of the fluorescence spectral microscopy is that the observed fluorochromes have to be excited at certain spectral ranges, limiting the spectral range available for detection. In addition, both the emission and the absorption spectra of simultaneously present dyes overlap and the signal quality is limited by photobleaching. This problem has been formulated by Zimmermann et al. in the case of live-cell spectral imaging, which introduces even stronger restrictions on the acquisition time.229 A more detailed method for solving this problem has been proposed by R. Neher and E. Neher also in the case of multiple fluorochromes by taking into account the bleaching effects.181 Despite this, spectral imaging microscopy and corresponding spectral analysis are useful methods for detecting weak fluorescent signals within a high autofluorescence background.
Histology is an essential tool of biology and medicine, which is commonly performed by examining cells and tissues by sectioning and staining, and followed by structural examination under a light microscope. To visualize different components of the tissue under a microscope in bright field, the sections are usually dyed with one or more stains. The histological staining of tissue sections enables the microscopic visualization of tissue structures and substances onto which the stains are attached without the need for selective spectral excitation as in fluorescence. It has been widely used in pathology, where the visual assessment of complex staining patterns constitutes the basis for the identification and classification of the status of the examined tissue.78 However, the visual assessment by the traditional light microscope can usually obtain qualitative and subjective results, which leads to only part of the diagnostic information of the stained tissues being explored.234
The spectral imaging technology may provide a new tool for histological analysis. It can take advantage of the spectral features of various stains used in histological sections to segment features of interest for subsequent morphometric measurement. In recent years, the method has been successfully applied to a wide range of histological stains and immunolabeling reaction products and has proved capable of detecting any color-stained object. Ornberg et al. presented spectral analysis of stain combinations such as Safranin O/hematoxylin for proteoglycans in articular cartilage, Oil red O in adipose tissue, peroxidase-diaminobenzi-dine immunostain with either fast green or hematoxylin counterstain, alkaline phosphatase-AP-Red, and -Vector Red immunostain and counterstains in a variety of tissues.65 In addition, Papadakis et al. discussed the possibility of using spectral imaging method for quantitative assessment of the uptake by histologic samples of stains used in pathology to label tissue features of diagnostic importance.78 This method has also been used to identify emissions on printed DNA microarrays and tissue microarrays.26,235 Li et al. presented the analysis method on spectral images of retina sections dyed with hematoxylin for evaluating the protective effect of erythropoietin on diabetic retinal cells and the spatial-spectral method for automated white blood cells segmentation.49,97 Kiyotoki et al. developed an objective method to detect gastric cancer using HSI technology combining spectroscopy and imaging.236 Larsen et al. presented the hyperspectral image analysis combined with the histology method for detection and characterization of advanced atherosclerotic plaques in vitro.237 Maeder et al. introduced a confocal laser-scanning microscope equipped with a detection unit that provides high spectral resolution method to visualize and evaluate the spatial distribution of spectral information in skin sections.221 In a more recent study by Nallala et al., an IR spectral imaging system was used for histopathological recognition in colon cancer diagnosis.238 Fourier transform IR spectral imaging system has also been used to perform the histopathology mapping of biochemical changes in myocardial infarction, live epithelial cancer cells, histologic and biochemical correlations in tissue engineered cartilage, and histomorphological evaluation of oral squamous cell carcinoma of the oral cavity.109,239 Moreover, the spectral imaging technology allows the investigator to separate dyes with overlapping emission spectra and to eliminate from the display certain dyes in a sample.13 In some cases, spectral images of unstained tissue sections may also provide useful information similar to the results available from stained histological methods.98,240,241 Potentially, this is a way to identify abnormalities within tissue at an earlier stage disease, as a change to the molecular composition of a cell most likely occurs before a morphological change. Furthermore, some Raman spectral imaging methods also have been introduced by researchers for different kinds of biological analysis, as it can visualize molecular vibration with high sensitivity and high contrast, allowing label-free imaging of biological samples. For example, Amharref et al. investigated molecular changes associated with glioma tissues by Raman microspectroscopy and then compared the spectroscopic markers obtained from C6 glioma tissues with conventional histological and histochemical techniques.242 Tollefson et al. evaluated whether Raman molecular imaging could be used to identify histological differences in patients with Gleason 7 prostate cancer who progressed to metastatic disease and died from prostate cancer.243 More recent study by Ozeki et al. has used the Raman scattering microscopy in various imaging modalities for histological analysis, such as 2-D spectral imaging of the rat liver, spectral imaging of several sections of intestinal villi in mice, and in vivo spectral imaging of mice ear skin.244 Kong et al. have developed electro-optical tunable Lyot filter–based multicolor stimulated Raman scattering microscopy for live HeLa cells analysis.245 In addition, there are also some commercial spectral imaging systems that have been used for visualization and quantitative analysis of multicolor tissue sections, for instance, the Nuance™ multispectral imaging system and morphologic image analysis packages, such as the inForm™ image analysis software (CRi), used for assessing morphological distributions of miRNAs and their putative targets in thin tissue sections,160 the Prism and Reflector Imaging Spectroscopy System (PARISS®) hyperspectral imaging system (LightForm) used for studying apoptosis,246 and the HSi-300 hyperspectral imaging system (Gooch & Housego) used for preclinical research and clinical pathology.231
These studies show that the histological spectral imaging method can improve both morphometric and biochemical analysis by enabling the evaluation of subtle cytological and histological features, which in turn can yield useful structural and medical information for diagnostic and prognostic evaluation. Moreover, the spectral imaging method can also identify the combination of stains by determining the spectral signatures of each stain on the slide and can select two or more wavelengths at which the differential absorption between stain and counterstain is greatest. It allows researchers to analyze multistains in bright-field mode and calculate the concentration of each and separate the complex color-image into a set of single color images, where each one of them shows the sample as if it were stained with only a single stain. One advantage of histological spectral imaging is that many stains can be used simultaneously, while still making it possible to analyze the histological section as if it were stained with a set of single stains. Another advantage is that it can provide information on the colocalization of structures, proteins, and other components by correlating the single stain distribution.162 To identify the stained objects in histological sections, the wavelength ranges should be selected to cover red (R), green (G), and blue (B) light, i.e., at wavelengths of , 545, and 435 nm, respectively.65 The reason is that most of the commonly used dyes are selected to highlight the stained objects of interest above a uniform background to match the spectral response of the human eye, which restricts in visible light. Therefore, a wavelength range covering the R, G, and B light, such as 400 to 650 nm, is usually needed for the histological applications. Another requirement for histological application is the proper spectral resolution. Spectral analysis is based on the ability of histological and cytological specimens to absorb, reflect, or emit photons in ways characteristic to their interactions with specific stains, whose characteristic absorption and emission bands usually occur in 10 nm or less narrow-wavelength ranges.215 Therefore, the spectral resolution of the spectral imaging systems generally needs to be to enable the demarcation of features and the evaluation of quantities from the spectral information of the histological specimen.
In traditional histological analysis, a good visual contrast in double immunoenzyme staining plays an essential role for identification by unaided eyes or RGB camera. In some cases, an RGB camera is sufficient to distinguish the two basic colors of double staining results. However, typical stain combination colors such as red/blue and turquoise/red have the drawback of containing one diffusely localized and rather insensitive/inefficient chromogen, which makes it hard to perform successful double staining with antibodies that show obscure and weak staining by nature or staining of tiny cellular constituents.247 With the spectral imaging technology, one can unmix the original observation according to the different spectral characteristics of chromogens and nuclear counterstain rather than the visual contrast. Although the histological spectral imaging method is well founded on the physical principles of light absorption and is simple and straightforward to perform, there are still some difficulties to quantitatively unmix up to four different markers in a few cases. This may be due to the fact that spectral imaging does not work ideally with transparent chromogens, and the number of stains present in the sample as well as the characteristics of the stains themselves will affect the performance of spectral imaging analysis methods.247,248 With the improvement of system performance and the advance of new identification algorithms, this problem may be solved. Moreover, it has been recognized that the quantitative assessment of the complicated histological staining patterns is sometimes ineffective and that its qualitative nature is capable of only partially exploiting the diagnostic content of the tissue staining techniques.78 One of the reasons is that histological dyes are usually difficult to control in terms of intensity of stain, from cell to cell and from section to section. As most staining reactions involve a chemical union between dye and the stained substance, the degree of stained tissue sections could be determined by conditions such as the staining time, the temperature, or pH of the solution. Therefore, the intensity of pathological images acquired using different time or facilities varies greatly depending on circumstances.249 These factors will make the quantitative assessment of the concentration of single or of multiple stains hard or not straightforward. Different from those histological dyes, some immunohistochemical reagents have the potential to support quantitative analysis. For example, the alkaline phosphatase (AP) and horseradish peroxidase are enzymes that are allowed to run as endpoint assays, which establish the level of a quantifiable effect indicative of a biologic process. An immunohistochemical dye is usually analogous to an enzyme-linked immunosorbent assay (ELISA) test performed in the clinical lab, and ELISA tests are widely recognized as being truly quantitative.250 Although immunohistochemical analysis has some practical advantages over immunofluorescence, the optical properties of the chromogens will affect the linearity and dynamic range of the assay. For instance, the popularly used brown diaminobenzidine (DAB) stain exhibits scattering behavior similar to that of melanosomes and does not obey the Beer-Lambert law (BLL), which underlies that the unmixing approach applies only to pure absorbers. In fact, it is hard to separate DAB from melanin pigmentation spectrally since their spectra arise from the same optical properties.250 Other chromogens used mostly make precipitating products, such as Vector Red, and also have a similar problem as it may not obey the BLL. However, this does not mean that quantitative immunohistochemistry is out of the question if the entire process, from sample acquisition to analysis and interpretation, is properly integrated, standardized, and combined with spectral imaging technology. For example, the Nuance™ system and inForm™ software (CRi) have been used for automated quantitative scoring of immunohistochemistry on tissue microarray recently.251 With the advances of spectral imaging in immunohistochemistry, it will become a routine assay for the assessment of therapeutic selection markers, which are likely to increase in number and be applicable to some diseases.
Summary and Conclusions
Spectral imaging technology has been and continues to be used successfully in the area of remote sensing to provide important insights into the study of the Earth.178 Fundamental to the concept is the ability to both spatially and spectrally resolve a heterogeneous surface topography for the purpose of detecting and classifying otherwise indiscernible features or targets, natural and manmade, by exploiting known spectral reflectance characteristics.252 Extension of this technology to biomedical engineering applications will allow novel exploration of anatomy, physiology, biochemistry, and pathology. In this review, we have assessed major efforts and advances of spectral imaging techniques in biomedical and biological scientific research spanning from microscopy to clinical in vivo imaging. The features of spectral imaging and traditional optical diagnosis methods, such as monochrome, RGB, and spectroscopy, were compared. Four major approaches (whiskbroom, pushbroom, staring, and snapshot) for acquiring spectral images were presented. Moreover, the advantages and disadvantages for the biomedical application of each imaging mode were discussed. Current biomedical spectral imaging systems, the performance of these systems, and their biomedical applications were also reviewed. This review shows that although the development of spectral imaging sensors and data analysis techniques are time-consuming and arduous, the types of information obtained by the spectral imaging techniques can enable us to clearly see characteristics of various tissues and organs in healthy and diseased subjects, which previously have not been able to be investigated so directly.252 With the refinement of current technologies and the development of new techniques, additional information will be available in helping dissect biomedical analysis in the future.
Current studies show that spectral imaging is a powerful technology that can be used to detect the biological changes in living organisms and has the potential for the early diagnosis of diseases. However, there are several limitations and issues in current studies that have blocked its widespread applications in biomedical engineering. The first one is the standardization of system and software. In the past decades, various biomedical spectral imaging systems have been developed and used for different biomedicine studies. However, there is no unified standard to make these systems compatible, comparable, and able to be shared. Further research is needed on the standardization of the system in areas of calibration methods, data formation and normalization, etc. The second one is the image shifting during wavelength tuning. According to the dispersion theory of light, the focus area will move with the increasing wavelength accordingly.253 In remote sensing applications, this influence can be neglected as the object distance is generally greater than the image distance. However, it cannot be neglected in microscopic spectral imaging applications as the object distance is usually less than the image distance and will lead to some single-band images becoming blurred with the wavelength increasing. Finally, the biomedical spectral imaging analysis mechanism is not clear. Spectral images contain a wealth of information, but interpreting them requires an understanding of exactly what properties of biological tissues we are trying to measure and how they relate to the measurements made by the spectral imaging sensor. Most of the current studies just extracted some spectral signatures and identified different kinds of tissue compounds according to these signatures. However, few studies have been done to explain what these spectral signatures mean. Further studies are needed to explore the implications of these spectral features. There are also some other issues for the biomedical spectral imaging system such as the time-consuming acquisition, expense, delicate and complicated, etc.161,254 Therefore, further studies are still needed on the development of biomedical spectral imaging systems. New instrument design concepts will be continually introduced, and current instruments and systems can also be improved to achieve better performance. The advances in biomedical spectral imaging systems will inspire the future development of spectral imaging technology.17 In addition, it should be mentioned that the feature extraction methods, the combined spectral and spatial analysis and classification algorithms are not involved in this review as most of them are similar to those in the remote sensing field.
It is difficult to get the detailed prospect of the biomedical spectral imaging technology. However, several trends can be anticipated. The first one involves the trend toward acquiring more accurate measurements, in spatial, spectral, and time of the state of the biological tissues and organs. New instrument design concepts will be introduced, and current instruments and systems can also be improved to achieve better performance for biomedical applications. The second trend is the surgical perspective. As a kind of optical analysis method, the spectral imaging examines only areas of tissue near the surface in vivo. This is obviously less of a problem to the surgeon, who is directly observing most of the organs of his/her interest. With the incorporating of spectral imaging and nanometer-sized particles such as QDs and colloidal gold, the surgeon can confirm a diagnosis and evaluate surgical therapy in an ongoing manner in the future. The third one is the dual- and multimodality spectral imaging. Even with the progress achieved, research on histology, genomics, and proteomics has not made it possible to obtain all biological information of tissues to meet the requirements of a complete biomedical analysis by using only one kind of imaging method. Therefore, multimodality imaging that combines several kinds of imaging methods to get more information about biological samples is needed. Spectral imaging combined with other biomedical imaging modalities can help in the interpretation and prediction of the behavior of proteins, organelles, and cells with different kinds of biological parameters, which will then need to be confirmed in living tissues and animals. This will be a clinically proven approach and will replace conventional diagnostic devices in the future. The final trend is the multiexpression, colocalization analysis with spectral QDs imaging. Analysis of the function and interaction of genes and of their protein products in cells, organ systems, and organisms is important for immunohistochemistry, clinical, and mechanistic biology. There is a need for an in situ method capable of detecting several genes or proteins at the same time, thus enabling good spatial and cellular resolution at a morphological level. A wide spectral range and high spectral resolution of a spectral imaging system may make it possible to identify the unknown multiple emission spectral signatures of some QDs, making it particularly well suited to multiple marker studies.32,255 Combined with advanced spectral unmixing and deconvolution algorithms, spectral imaging comprises the technology of choice in separating multiple QDs with overlapping spectra and is used to stain different targets within the same tissue sample.254 This method can be used in a wide range of in vitro and in vivo applications, enabling single molecule tracking, high resolution in vivo tracking, and multiplex imaging for biomedical investigations. In addition, the biomedical applications of longer wavelengths imaging should also be considered in future studies as the wavelength ranges usually have a higher transparency than the visible wavelengths in some tissues and organs. For example, the NIR spectral region (900 to 1700 nm) is suitable for noninvasive high-contrast imaging of the tooth and bone beneath the enamel surface of for disease diagnosis.83 The midwave IR (MWIR) region from 3 to 5 μm and the long-wave IR (LWIR) spectral region from 8 to 14 μm should also be explored for biomedical applications. As the MWIR and LWIR imaging generally are sensitive to thermal emissions and need no artificial illumination of any kind, they are suitable for detecting hot targets. For example, the MWIR and LWIR imaging are more advantageous for measurements of the pulsatile heat patterns of human vital signs.
In summary, continual improvements in the instrumentation, software design, and algorithmic developments of spectral imaging technology will result in achieving more complex analytical results of pathology, histology, genomics, and clinical diagnosis. It is anticipated that more and more researchers and physicians will be attracted to try this technology in their biomedical research and the market demand for commercial biomedical spectral imaging instruments can be expected to increase. Spectral imaging methods and technology will become progressively widespread and will be utilized routinely for biological/biomedical research and related arts in the near future.
This work is supported in part by the National Natural Science Foundation of China (Grant No. 61177011, 61377107), the Project supported by the Shanghai Committee of Science and Technology, China (Grant No. 11JC1403800), and the Project supported by the State Key Development Program for Basic Research of China (Grant No. 2011CB932903). The authors would like to thank Ms. Carmen Banton, New York State Psychiatric Institute, New York, for her valuable suggestions.