Clinical molecular imaging (MI) covers a broad range of techniques, including fluorescence, positron emission tomography, single photon emission computed tomography (CT), and magnetic resonance imaging.1 MI aims to improve diagnosis by utilizing targeted reporters (smartprobes) for specific disease targets in tissue. In this work, we focus on advancing fluorescence-based MI microendoscopy of the distal lung (gas-exchanging alveolar regions).2 Optical molecular imaging (OMI) has proven to be effective against several disease targets, such as bacterial infection,3 inflammation,4 and cancer.5 Several candidate smartprobes have been designed targeting neutrophil recruitment,6 bacterial detection,7 and fibrogenesis.8 Our evolving strategy is to deploy custom smartprobes alongside pulmonary microendoscopy in vivo to image disease relevant targets.
One particular area where disruptive optical technologies would potentially have high impact is in the rapid diagnosis of lung inflammation and infection in critically ill-ventilated patients, which has high level of mortality, often above 70%.9 Postmortem, pneumonia is pathologically defined on biopsies of alveolar (distal lung) tissue infiltrated with bacteria and inflammatory cells. The most common method to determine the bacterial burden in respiratory critical care relies on the culture of bronchoalveolar lavage fluid (BALF). The major limitations of BALF in critically ill patients are the time taken to yield a result, which can be up to 48 h,10 and contamination by proximal airways sampling. During this intervening time, patients may be prescribed inappropriate therapy or deteriorate rapidly while awaiting a confirmatory diagnosis. Therefore, there is a clinical unmet need11 for immediate information that can determine the absence or presence of bacteria and markers of inflammation such as infiltrating innate immune cells (e.g., monocytes and neutrophils) associated with suspected pneumonia in critically ill patients.
Our approach is based on using smartprobes coupled with miniature fiber-optic imaging bundles12 to access distal alveolar regions and transmit the image from the distal end of the fiber to the rest of the optics. The technique is used in conjunction with flexible bronchoscopy (FB).13 FB is able to access trachea, bronchi, and bronchioles, but cannot access the distal lung. FB was introduced by Ikeda et al.,14 who pioneered fiber bundle based bronchoscopy in 1968 and subsequently worked on improving the imaging by introducing charge couple device (CCD) cameras to the tip. Flexible bronchoscopes have tips 4 to 6 mm in diameter and deploy white light endoscopy for navigation, with some systems utilizing narrow-band spectral imaging to increase contrast on superficial blood vessels.15,16 Autofluorescence of the lung is complementary to white light endoscopy in providing a convenient way to navigate to the site of interest. For illumination in the 460 to 490 nm range, the lung autofluorescence emission spectra peak between 520 and 570 nm. Autofluorescence endoscopy has been used to detect pathologies in tissue since 1990s, especially in the gut.17 However, interpatient variations in lung autofluorescence pose a serious problem in using autofluorescence alone clinically,18 which significantly reduces specificity.19 Autofluorescence from the lung originates from elastin,20 and the autofluorescence spectra demonstrate that the red region (650 to 700 nm) has lower autofluorescence intensity (5 to 10 times lower) than the green region (500 to 600 nm). This provides a strong motivation for the design of smartprobes in the red and near-infrared (NIR) spectral ranges (600 nm to 1000 nm).
Microendoscopy can be used in bronchoscopy by inserting the narrow fiber imaging bundle through the working channel of the bronchoscope. The diameter of the working channel available on modern flexible bronchoscopes ranges from 1 to 3 mm depending on the manufacturer. For distal lung imaging, narrower microendoscope probes are necessary to enable safe access to the alveolar space. The distal alveolar regions are reached by performing a transbronchial pass. Bronchoscopic interventions usually last 20 to 30 min, and prior to the procedure, navigation is often guided by x-ray radiographs or CT scans, allowing for targeted regional exploration. Optical microendoscopy enables the exploration of pulmonary segments by allowing multiple endobronchial passes into the distal lung. This is ideally suited to image the diffuse conditions seen with suspected pneumonia or pulmonary inflammation. This methodology has been demonstrated in ex vivo large animal models using smartprobes8 and in vivo in humans for label free parenchymal assessment of the distal lung.20–22
Previous work in colonoscopy,23 ovarian cancer,24 detection, and preclinical mouse imaging25 have been reported, and multicolor fluorescence endoscopy has a long history.26,27 Our instrument is based on a single-color fluorescence endoscopy platform28,29 with the fluorescence filter-set exchanged for that of an integrated two color, green and red, fluorescence spectral range prior to collecting the image on a color CMOS camera. Confocal microendoscopy is an alternative method30 deployed both in academia31 and clinics20–22 and provides optical sectioning, permitting higher contrast imaging, as tissue background fluorescence is largely suppressed.32 Two-color confocal microendoscopy has also been explored, demonstrating good contrast on a variety of tissues.33 However, the scanning required for confocal systems is inherently slower than camera-based imaging methods.
While single color confocal microendoscopy has been evaluated for bronchial airway and distal lung imaging,34 multicolor widefield camera-based microendoscopy remains to be fully characterized. Fiber bundle imaging and bacterial dynamics, in the case of mycobacteria, have also recently been studied.35 In this work, we show that camera-based systems provide a robust and economical route to multicolor fluorescence detection. White light endoscopy dominates bronchoscopy procedures, and our aim is to enhance fluorescence endoscopy by integrating smartprobes in multicolor scenarios. Here, the ability of the widefield system to operate in the distal lung by imaging targets ranging in size from 2 to such as bacterial colonies, monocytes, neutrophils, and the fungus Aspergillus fumigatus deposited in ex vivo human alveolar lung tissue is demonstrated. Also, the blinking effect of fluorescing objects smaller than the fiber core and cladding is exploited to augment bacterial imaging.
Materials and Methods
The diagram of the optical system is shown in Fig. 1. The architecture extends standard widefield endoscopy to dual color while taking advantages of improved imaging and illumination technologies.29 All components are off-the-shelf commercial items. The proximal end of the fiber imaging bundle (Alveoflex™, Mauna Kea Technologies, Paris, France) is brought to the focus of the objective by mounting it onto a 30-mm caged -axis translation mount (SM1Z, Thorlabs) enabling fine focusing and optimized coupling of the image from the distal end of the fiber to the rest of the optics. The Alveoflex™ imaging fiber has approximately 30,000 cores without any lensing on the tip (i.e., works by contact imaging). Figure 2(a) shows the fiber cores of the Alveoflex™ imaging bundle. Two LEDs at 470 and 625 nm (M470L3 and M625L3, Thorlabs) are used as illumination sources coupled to achromatic condenser lenses (ACL2520-A, Thorlabs for 470-nm LED and ACL2520-B, Thorlabs for 625 nm LED), see Fig. 2(b) for LED spectra. The collimated beams are combined using a dichroic beamsplitter (FF506-DI03-25X36, Semrock). Two-color epifluorescence is achieved using a two-color filter-set (emission, dichroic, and excitation filters in XF454, Horiba, UK) before being directed to an infinity-corrected microscope objective with NA 0.3 and working distance 10 mm (RMS10x-PF, Thorlabs). Emission and excitation filter spectra are shown in Figs. 2(b) and 2(c), respectively, along with normalized absorption and emission spectra of example fluorescent probes used in this study (PKH67 and Cellvue® Claret; see text below).
Fluorescence exiting the microscope objective is imaged onto the color CMOS camera (GS3-U3-23S6C-C, Grasshopper3, Point Grey Research, Canada). A color imaging camera used in conjunction with an appropriate multiband filter set was chosen to simplify the optical design for application in a clinical environment. The camera has a universal serial bus 3 (USB3) connection to the computer allowing up to 162 frames per second (fps) at full resolution, . The color filter is the standard Bayer pattern filter [see Fig. 2(c)] deployed in many consumer, machine vision, and microscopy cameras. The CMOS camera sensor is a Sony IMX174 which has seven photoelectrons () read noise.36 We refer to the green and red channel as the green (510 to 560 nm) and red fluorescence (660 to 700 nm) as captured from the setup onto the camera and filtered by the Bayer filter on the CMOS chip. As shown on Fig. 2(c), the Bayer color filter overlaps with the green channel of the two-color filter-set used (XF454, Horiba, UK), while the overlap in the red channel is smaller. Smaller overlap in the red channel is offset by greater separation of the two channels (), which reduces spectral mixing.
Optical System Characterization
The system performance was characterized by measuring the lateral spatial resolution, the spectral throughput of illumination and fluorescence collection, chromatic focus shift, verification of the camera read noise, and performance at high-frame-rate acquisition (e.g., ). The aim was to achieve the optimal contrast on targets ranging in size from bacteria (0.5 to ) to cells, such as monocytes and neutrophils (10- to ), and to fungi ( or more). The lowest detectable light intensity defines the visibility of the object against the noise floor, while chromatic aberrations potentially affect contrast levels for each spectral range detected. The required frame rate for optimal assessment in fluorescence microendoscopy is suggested as 10 to 15 fps.37 While this is easily achievable with CMOS camera technologies in terms of speed and noise,38 it is imperative to assess how such frame rates compromise the limit of detection. High frame rates could be applied to tissue elastography and assessing alveolar compliance. In this study, frame rates of 12–20 fps and camera resolution were chosen to readily permit real-time (video rate) visualization by operators. Commercial microendoscopy systems operate at this rate.20 Higher camera resolution () was used whenever fiber core visibility was needed. All tissue imaging was done at , unless stated otherwise. Higher frame rates up to 200 fps were also evaluated, which is the upper limit imposed by the hardware, notably the camera, memory transfer speeds, and hard disk space available on the computer.
Optimal spatial resolution was measured by contacting the distal end of the Alveoflex™ fiber to a standard chrome on glass USAF 1951 target (Thorlabs, part number R1DS1N). This measurement was done in transmission (through the USAF target) by allowing the fluorescence from a target (fluorescent block, Chroma) in the spectral range 510 to 560 nm (Thorlabs, MF510-42) to pass through the test target having been illuminated via the excitation delivered through the Alveoflex™ fiber and the target imaged back through the optical system in the conventional manner. An intensity line profile was taken over the image obtained from the test target to estimate contrast reduction with increasing resolution.
Spectral throughput measurements were undertaken by inserting a fiber probe connected to the spectrometer ( fiber connected to a USB2000 spectrometer, Ocean Optics) on the following locations: (1) distal side of Alveoflex™ (in front of the light source), (2) proximal side of Alveoflex™, and (3) the camera imaging plane. The distal end of Alveoflex™ was illuminated by a white LED lamp. Fluorescence collection efficiency was measured by illuminating the distal end of the fiber by a white LED measuring the optical power at (1) the proximal side of Alveoflex™ and (2) the camera imaging plane.
The limit of detection (LOD) for fluorescein (Sigma-Aldrich) and Cy5 (Sigma-Aldrich) solutions was determined. 1, 10, 50, and 100 nM water solutions were prepared. The Alveoflex™ was inserted into the solution, and 10 images were taken for each concentration with the exposure time set to 80 ms. The background image was obtained by illuminating the Alveoflex™ by both LEDs as in normal operation, but without any fluorescence or stray light entering the tip. Fifty background images were saved and their mean subtracted from images acquired from solutions. A area was selected from the central area of the Alveoflex™ imaging fiber bundle. Mean and standard deviation were calculated for the area, for a single 80 ms exposure time image indicating signal-to-noise (SNR).
Chromatic effects are of particular importance for multicolor fluorescence microendoscopy. We assess the chromatic effects by loading the distal end of the Alveoflex™ with 0.3% relative intensity standard fluorescence microspheres (green Inspeck™ I-7219 and red Inspeck™ I-7224, Life Technologies). Fluorescence microspheres in solution readily attach to the distal tip, and this provides a good target for evaluation of contrast and contrast degradation. To optimize the focus of fluorescence onto the camera, we adjust the position of the tube lens with respect to the CMOS sensor camera and the position of the proximal fiber end with respect to the microscope objective. The tube lens was prealigned as in Ref. 26 and kept fixed. The proximal end of the Alveoflex™ was moved to three locations. Optimal focus for green fluorescence was found and moved around this location.
Camera read noise was measured by taking 100 bias frames (0 ms exposure time) in dark room conditions with a metal C-mount cap on the sensor.39 Camera gain was set to 0 dB and the camera video mode to 7 (lowest noise mode for this particular camera40). The mean of all bias frames was subtracted from a sample bias frame and the standard deviation provides the read noise in analog-to-digital units (ADU). We used as the system gain41 to derive the read noise in . We also used the monochrome version of the sensor for read noise measurements (part number GS3-U3-23S6M-C, Point Grey Research, Canada). Dark noise was measured by taking a 3 s exposure time image in dark room conditions as above. The average bias frame was subtracted from the 3 s exposure frame, and the standard deviation of the resulting image was divided by 3 to obtain the dark noise for a 1 s exposure.
For all experiments, a background image was acquired prior to imaging sessions and subtracted from the acquired images. The resulting image was contrast enhanced to allow for optimal illustration of image SNR. No frame averaging or image processing was performed, although the potential improvement of image processing is discussed further below.
Microsphere and Tissue Experiments
Current methods to determine the bacterial burden in suspected ventilator-associated pneumonia (VAP) rely on the culture of BALF. The growth of colony forming units (CFU) above 42,43 or 44,45 is considered diagnostic of VAP. These are the accepted clinically relevant limits, and we have based our system on being able to perform at this limit of detection. As broncholavolar lavage involves the instillation and retrieval of fluid, a dilution effect is seen. The true bacterial burden in the infected distal lung is likely to be to .9 Therefore, to establish the suitability of the instrument for the lower limit of bacterial detection in the distal lung, we initially performed experiments in two scenarios whereby we used microspheres as surrogates for bacteria: (a) imaging of microspheres alone in black Eppendorf tubes (to avoid tube fluorescence) and (b) imaging of microspheres in ex vivo lung tissue placed in a well plate. 0.3% Inspeck™ microspheres (green Inspeck™ I-7219 and red Inspeck™ I-7224) were used as a fluorescence emission standard indicative of labeled bacteria. Green 0.3% Inspeck™ microspheres have equivalent emission to green OMI smartprobe7 labeled bacteria (Pseudomonas aeruginosa), and red 0.3% Inspeck™ microspheres are 1.5 times weaker than PKH red (Cellvue® Claret, Sigma-Aldrich) labeled bacteria (Pseudomonas aeruginosa). Blackened microtubes were filled with concentrations , , , and to derive the detection limit of the system without the lung tissue present. Tubes were centrifuged for 10 s before each experiment. Excised lung tissue was placed in well plates with 0.3% Inspeck™ microspheres deposited at the following concentrations: , , , and . Controls included excised lung tissue alone, empty microspheres (unstained control beads supplied with green Inspeck™ I-7219, Life Technologies) per milliliter in lung tissue, distilled water alone and lastly, empty microspheres per milliliter in distilled water. The controls were used to minimize false positive detections, especially in the green channel, where small parts of tissue could be mistaken for bacteria.
Each sample was imaged for 30 s. The fiber was moved every 2 to 3 s to image as much of the volume as possible in the limited observation time. Detection of weakly fluorescent microspheres was assessed visually through user interpretation of the real-time images acquired during the 30 s experiment. A successful detection was defined as visual identification of at least a single target with pattern recognition similar to those in the controls. The camera settings were set as follows: 8 bit image acquisition, gain 24 dB, exposure time 80 ms (12 fps).
Labeled Targets, Tissue, and Bacterial Imaging Experiments
Excised human lung tissue was placed in well plates with five experimental conditions: (1) freshly isolated human monocytes labeled with PKH red (Cellvue® Claret, Sigma-Aldrich); (2) freshly isolated human neutrophils labeled with PKH green (PKH67, Sigma-Aldrich); (3) Aspergillus fumigatus labeled with PKH red; (4) labeled monocytes, neutrophils, and fungus (Aspergillus fumigatus) together in single well; and (5) bacteria (Pseudomonas aeruginosa) at labeled in PKH green (PKH67, Sigma-Aldrich).
Neutrophils and monocytes were isolated by discontinuous percoll gradients from the peripheral blood of human volunteers using methods previously described.46 As mentioned above, bacteria used were Pseudomonas aeruginosa (ATCC 47085), and fungi used were Aspergillus fumigatus (clinical isolate, Royal Infirmary of Edinburgh, Scotland). Bacteria or fungi were counterstained with PHK67 or Cellvue Claret dyes (Sigma-Aldrich, St Louis, MO) in accordance to the manufacturer’s instructions. Briefly, cells were washed three times in PBS and re-suspended in of Diluent C, then added to of Diluent C containing of stock dye. Following 3 min, bovine serum albumin was added to terminate the reaction, and cells were washed a further three times and re-suspended as required for experiments. Human tissue was used with regional ethics committee (REC: 13/ES/0126) approval and was retrieved from the periphery of specimens taken from lung cancer resections. Monocytes and neutrophils were isolated and purified from peripheral blood of healthy volunteers (REC: 08/S1103/38). Each sample was imaged for 30 s, and the camera settings were as follows: 8 bit image acquisition, gain 24 dB, exposure time 80 ms (12 fps). PKH red and green were used as per manufacturer’s instructions.
Well plate 5 with PKH green labeled Pseudomonas aeruginosa at was additionally imaged at 5 ms exposure time (200 fps) to study the dynamics of fiber core blinking events which are observed when imaging objects smaller than the fiber core. Furthermore, to study the effect further, we dissolved of Constellation™ microspheres (C-14837, Life Technologies) in of distilled water and imaged the solution in (1) free space—fiber bundle was removed from the setup in Fig. 1 and solution was placed on the slide and covered by a coverslip—and (2) with fiber bundle, that is, conventional microendoscopy imaging. Constellation™ microspheres have a range of sizes and colors and are a good candidate for cross comparison of imaging effects on small (less than fiber core size ) and large objects (greater than several fiber core sizes ). Fiber bundle imaging of Constellation™ microspheres was performed in a microtube, as contact with the glass slide could potentially damage the fiber bundle tip.
Optical System Characterization
Below, we present the measurement results for system spatial resolution, camera read noise, fluorescence collection, fluorescence spectral throughput, chromatic focus shift, and an evaluation of high-speed performance on ex vivo lung tissue.
Figure 3(a) shows the USAF test target with an even fluorescent illumination, and Fig. 3(b) shows the line profile across group 6 elements where the line thickness of the smallest resolvable line (group 6, element 6) is . Individual test chart lines are clearly distinguished at this resolution. Fluorescence ranging from 510 to 570 nm is imaged and focused in Fig. 3. Fiber core size, cladding size, and core to core coupling47 limit the system’s spatial resolution and hence image quality; see Fig. 2(a) for a demonstration of fiber core packing density. Read noise was measured to be and dark noise . As the frame rates used are at least 12 fps, the influence of dark noise is minimal. The signal is read noise dominated for the detection of up to about , where read noise is 35% of the Poisson noise. The uniformity of the field of view (FOV) is as follows: for the green channel, edge intensity is 54% of the central intensity, and for the red channel, edge intensity is 76% of central intensity. While future work will need to address the uniformity of the green channel, in the current systems this did not seriously affect the imaging quality. The optical power exiting the distal fiber and fluorescence collection efficiency are summarized in Table 1. The maximum optical power coupling to Alveoflex™ for 470 nm LED is 2.4 mW, but we limit this to 1.8 mW due to the intrinsic red autofluorescence of the Alveoflex™; see fiber background discussion below. For 625 nm LED, the optical power measured at the distal end of the fiber is 3.4 mW. Total optical power is within acceptable limits of maximum exposure as defined in Ref. 48. For fluorescence collection, we measured 53% efficiency for both channels.
Illumination and fluorescence collection efficiency.
|470 nm LED optical power exiting the distal end of Alveoflex™||1.8 mW|
|625 nm LED optical power exiting the distal end of Alveoflex™||3.4 mW|
|Fluorescence collection efficiency on green channel||53.1%|
|Fluorescence collection efficiency on red channel||52.8%|
Spectral throughput is analyzed in Fig. 4. Fiber fluorescence throughput is above 85% across the spectral range studied, 500 to 750 nm. System fluorescence throughput was measured at the camera image plane and includes attenuation by the fiber bundle, objective, dichroic, emission filter, and the mirror (see Fig. 1). It reflects what would be expected from the dichroic and emission filters used. Figure 4 shows Bayer filter spectra superimposed to illustrate expected reduction of the signal onto the camera.
SNR analysis of the uniform solutions of fluorescein and Cy5 are given in Table 2. Fluorescein has a LOD concentration of 10 nM, and Cy5 has a LOD concentration of 50 nM. Other solutions, such as in Ref. 21, obtain lower detection limits, predominantly due to deploying several EMCCD cameras with optimal spectral ranges on each camera, whereas we have a single color camera with a higher inherent read noise. We verified this by using a scientific CMOS camera (PCO.edge mono 4.2LT, PCO AG, Germany) and LOD improved to 1 nM for fluorescein and 10 nM for Cy5.
SNR for FITC and Cy5 solutions.
|1 nM fluorescein|
|10 nM fluorescein|
|50 nM fluorescein|
|100 nM fluorescein|
|1 nM Cy5|
|10 nM Cy5|
|50 nM Cy5|
|100 nM Cy5|
Figure 5(a) shows the contrast degradation with defocus for green and red fluorescence. Defocus is obtained by moving the proximal fiber side. Figure 5(b) shows the line profile across the green microspheres on the green channel. It indicates that the position is the optimal focus for green microspheres, judging by contrast. Figure 5(c) shows the line profile across the red microspheres on the red channel. It demonstrates that is the optimal focus for the red microspheres. This axial displacement for the red channel over the green channel is likely due to chromatic aberration. As indicated previously,49 some imaging fiber bundles have detectable fiber background autofluorescence. The modulation on Fig. 5(c) for the position is due to red fluorescence from our fiber cores and provides further evidence that this is the optimal focus for the red channel. The compromise is to place the proximal end of the Alveoflex™ between the two optimal focal locations, minimizing the chromatic effect with our 0.3 NA microscope objective. The presence of chromatic aberration is well understood and is, in part, a consequence of a single channel detection platform. However, by placing the proximal end of the Alveoflex™ between the two optimal focal locations, we minimize the chromatic effect with our 0.3 NA microscope objective, and image reconstruction is not compromised significantly for either of the imaging channels.
Figure 6 demonstrates fast image acquisition of ex vivo lung autofluorescence. While 3% Inspeck™ microspheres are visible at both 20 and 200 fps (see arrows in Fig. 6), the weaker fluorescing 0.3% microspheres are barely visible at both 20 and 200 fps. Despite the fact that faster frame rates result in scaling down the fluorescence captured on the sensor, the autofluorescence and tissue structure are still visible at 200 fps. Faster frame rates require faster display hardware (display rate on monitors is currently limited to 60 to 120 Hz). One interesting observation from the 200 fps video sequence (see accompanying Video 1) is the possibility to extend microendoscopic imaging to tissue elastography.50 Also, contrast often increases with increasing frame rate despite lower fluorescence signal. This is due to better localization of the object. Slower frame rates can “smear” the contrast of an object under study due to the fast moving endoscopic environment.
Microsphere and Tissue Experiments
For both red and green 0.3% Inspeck™ beads, the LOD was determined to be 104 microspheres per ml. Tables 3 and 4 show that LODs for green and red 0.3% Inspeck™ microspheres embedded in the excised human lung tissue are equivalent to and , respectively. The controls show no detected microspheres in both cases. The higher LOD for green microspheres is due to high level of lung tissue autofluorescence in the green. Autofluorescence of the human lung elastin and collagen enables anatomical navigation between the airways and alveoli. However, once the distal lung is reached, it would be desirable to remove the background autofluoresence if the object we are trying to image has broadly equivalent excitation wavelengths. In fact, this is the major reason to resort to designing molecular probes in red and NIR in the lung coupled with microendoscopy as the background fluorescence drops dramatically with higher wavelengths.
Well plate experiment for green 0.3% Inspeck™ microspheres.
|Experiment||Number of duplicate well plates||Number of successful detections|
|green 0.3% Inspeck™ microspheres per ml in lung||4||0|
|green 0.3% Inspeck™ microspheres per ml in lung||4||0|
|green 0.3% Inspeck™ microspheres per ml in lung||4||2|
|green 0.3% Inspeck™ microspheres per ml in lung||4||4|
|empty microspheres per ml in lung||2||0|
|empty microspheres per ml in distilled water||2||0|
Well plate experiment for red 0.3% Inspeck™ microspheres.
|Experiment||Number of duplicate well plates||Number of successful detections|
|red 0.3% Inspeck™ microspheres per ml in lung||4||1|
|red 0.3% Inspeck™ microspheres per ml in lung||4||4|
|red 0.3% Inspeck™ microspheres per ml in lung||4||4|
|red 0.3% Inspeck™ microspheres per ml in lung||4||4|
|empty microspheres per ml in lung||2||0|
|empty microspheres per ml in distilled water||2||0|
Labeled Targets, Tissue Experiments, and Observations on Bacterial Blinking
Figure 7 illustrates the capability of the two-color widefield system to perform fluorescence imaging of multiple targets of immediate relevance to pulmonary inflammation and infection. Figure 7(a) shows PKH red labeled monocytes in ex vivo lung tissue. Figure 7(b) shows PKH green labeled neutrophils in ex vivo lung tissue. Figure 7(c) shows the strong signal from Aspergillus fumigatus. Due to its large size, Aspergillus fumigatus is a relatively easy target to image. Simultaneous detection of monocytes, neutrophils, and Aspergillus fumigatus is demonstrated in the accompanying Video 2. As tissue autofluorescence has a strong green component, MI of targets in green is more difficult. However, for cellular targets in size or more, this is entirely feasible providing signal to background is greater than 10%.51
Smaller objects such as bacteria are also visible at high signal to background; however the fast moving microendoscopy environment requires a large number of dots for them to be immediately observable by a clinician. Figure 8 shows two images of ex vivo lung, one without bacteria, Fig. 8(a), and one with bacteria, Fig. 8(b). Bacteria are visible as blinking fluorescing dots; see Videos 3 and 4. The number of blinking dots is less in Video 3 (no bacteria) when compared to Video 4 (with bacteria). Blinking can be explained as follows: If a fluorescing object is smaller than the fiber core and cladding, then the inherent relative motion of the distal end of the fiber bundle with respect to the tissue will induce the object to move in front of the core (visible) and cladding (not visible). In order to explore the phenomenon of blinking of dynamic targets over the fiber cores, we also evaluated frame rates up to 200 fps in imaging ex vivo human lung tissue with PKH green labeled Pseudomonas aeruginosa. Figure 9 demonstrates this mechanism in three sequential images, each 5 ms exposure time, from Video 5. A core lights up in the left image of Fig. 9, then goes :off: in the middle image, and then lights up the neighboring core in the right image. Figure 10 illustrates the effect further. Constellation™ microspheres were utilized to demonstrate that in free space imaging the blinking effect is absent in both large and small fluorescing objects. However, when imaged through the fiber bundle, blinking dominates the imaging of smaller objects. Objects larger than the core and cladding do not blink. Top and bottom figures in Fig. 10 are not on the same FOV. Videos 6 and 7 demonstrate the effect clearly.
This mechanism can be put to good use providing the bacterial colonies are small enough and bright enough. One can even think of engineering fiber cores and cladding (or grids for ex vivo study of assays) to match the size of the object that needs detecting, thus maximizing blinking. The key attraction is that, to our experience, blinking is readily detectable by observers at SNRs even less than 1. One also has to be cautious; any object smaller than the fiber cladding will eventually blink, whether bacteria or not, so false positives are likely. The same applies to scenarios where ultrafast imaging is performed, which will accentuate blinking due to motion effects and low SNR (blinking due to shot noise). Additionally, many bacterial colonies are large in size (), and these will not blink. Despite limitations, we believe the technique could be useful if appropriately integrated into MI algorithms involving microendoscopy.
A key strategy for future work is to use autofluorescence for structural imaging in the in vivo human lung and deploy smartprobes to evaluate detection of specific pathology.7,8 The limit here is that green autofluorescence interferes with green fluorescing smartprobes, creating challenges to disentangle the two signals in vivo in situ. Even if the smartprobes were emitting strongly, signal levels in excess of 10% of the background level are required for quantitative assessment.47 Furthermore, the nonhomogenous and dynamic nature of the background signal imposes limits on the uniformity of pathology under investigation, potentially decreasing the limit of detection at the lower limit. This motivates the move to red and NIR fluorescence where background is much weaker.52 In the red and NIR, fluorescence collection efficiency and the camera read noise become limiting, both of which can be further optimized, for example by deploying sCMOS cameras which have read noise .53 Depending on the molecular target, spatial resolution (both axial and lateral) can be sacrificed. For example, molecular sensing of enzyme activity using a smartprobe displaying fluorescent amplification upon substrate cleavage does not need resolution, just the ability to see fluorescent amplification from an “off” state to an “on” state. A smartprobe recently designed to detect active fibrogenesis8 is a good example—the general level of green fluorescence increases throughout the diameter FOV only when the smartprobe is applied to human lung tissue with active fibrogenesis indicated by increased lysyl oxidase (LOX) enzyme activity. This is due to the fact that the LOX enzyme activity is detected extracellularly and the fluorescence amplification takes place over a large area of tissue and is not localizable to a specific organelle inside a cell, which would require high-resolution imaging. Other examples in oncology and beyond can be found elsewhere.54
The effect of fiber fluorescence in the red channel when illuminated with 470 nm LED is shown in Fig. 11. The experiment consisted of imaging the ex vivo lung tissue with 0.3% Inspeck™ beads. Constant 625 nm LED illumination was provided, and 470 nm LED was ramped up and down (from 1 to 2.4 mW) to illustrate the effect of spectral mixing of red fluorescence of the Alveoflex™ with imaging of Inspeck™ microspheres. 1 mW passing through Alveoflex™ causes noticeable red fluorescence, but Inspeck™ 0.3% microspheres are still visible in Fig. 11(a) (see the zoomed inset in top left of Fig. 11(a) as well as microspheres indicated by white circles). When 470 nm LED was set to 2.4 mW, red fluorescence from the fiber was saturating the red channel of the sensor. Fluorescence from fiber bundles has been analyzed before,45 and Fig. 11(c) shows the fiber fluorescence spectrum from 500 to 850 nm. The spectrum was obtained by using a broad FITC filter-set (FITC-LP01-Clinical-000, Semrock) and spectrometer placed on camera image plane (USB2000 spectrometer, Ocean Optics). Most of the fiber bundle fluorescence is concentrated in the region 600 to 700 nm, and further work is needed to reduce or remove it while keeping multicolor capability.
As we mentioned in Sec. 2, no image processing was used; instead we focus on the raw performance of the underlying optical system at clinically relevant limits (42,43 or 44,45). It has been demonstrated that confocal microendoscopy benefits from image processing.55 The main methodology is to oversample the fiber core pattern, calculate the signal from each core, and interpolate the random core pattern into a rectangular image. This provides a reliable summary of information from each core, while in the images above we present the oversampled raw images. Future work will explore such techniques to deploy real-time processing for improved multicolor detection of labeled disease targets.
Multicolor fluorescence microendoscopy promises to provide instant pathology to clinicians during time-limited in vivo in situ interventions. We have shown that by utilizing a simple imaging platform designed from off-the-shelf components, it is readily achievable to detect important pathological targets with little or no image processing. With suitable fluorescent smartprobes it will be possible to disentangle background autofluorescence and derive vital information related to pathology. A prospect of particular importance to us is that the low-cost technologies required to perform this (LEDs and CMOS sensors) are maturing to such an extent that devices such as the one presented here could be developed for widespread use.
We would like to thank the Engineering and Physical Sciences Research Council (EPSRC, United Kingdom) Interdisciplinary Research Collaboration grant EP/K03197X/1 for funding this work. We would also like to thank the Wellcome Trust & Department of Health HICF award (HICF-0510-069) for supporting the staff involved in this study. We would like to thank PCO AG and Jason Marvin from Photon Lines, UK, for loaning us the PCO sCMOS camera. We also acknowledge Thorlabs technical support in the implementation of optomechanics and LEDs. Datasets related to this publication can be found at http://dx.doi.org/10.7488/ds/1372. Due to storage requirements some datasets may not be complete, but full datasets and scripts for all videos can also be obtained from N.Krstajic@physics.org.
Nikola Krstajić is an EPSRC IRC Proteus postdoctoral research associate working in fluorescence endoscopy and detector integration. His interests include both photonics system design and component design (detectors and sources). After industrial work in medical and scientific instrumentation, he received his PhD in physics from the University of Surrey in 2007 and has worked since in biomedical optics at the University of Sheffield, University of St Andrews, and University of Edinburgh.
Ahsan R. Akram is a clinician working in the field of pulmonary optical molecular imaging. He received his medical degree in 2005 from the University of Edinburgh and his PhD in translational optical imaging in 2015. His emerging research interests include the use of optical modalities in pulmonary medicine.
Tushar R. Choudhary is an EPSRC IRC Proteus postdoctoral research associate. He received his PhD at Heriot-Watt University and the University of Glasgow in human retinal oximetry using multispectral imaging. His research interests include optical and spectral imaging and fluorescence lifetime in research and clinical applications.
Neil McDonald received his PhD in 2006 at the University of Dundee. He worked in Calgary as a postdoctoral research assistant and then returned to Edinburgh to work as postdoctoral research assistant and project manager in the pulmonary optical molecular imaging group supported by a Wellcome Trust Health Innovation Challenge Fund to develop bacterial imaging agents.
Michael G. Tanner is an EPSRC IRC Proteus postdoctoral research associate. He received his PhD at the University of Cambridge, collaborating with the Hitachi Cambridge Laboratory where he continued as a postdoctoral research scientist developing silicon architectures for quantum computing. In 2009, he expanded into quantum optics, moving to Heriot-Watt University and then the University of Glasgow to work on single photon detectors and their applications, including optical fiber sensors.
Ettore Pedretti is an EPSRC IRC Proteus postdoctoral research associate. He received his PhD from the Université de Provence in 2003. He pioneered imaging with densified pupil interferometry and had a key role in developing the infrared camera and data acquisition system at the IOTA interferometer while being a Smithsonian predoctoral fellow. As a Michelson postdoctoral fellow, he implemented the infrared camera and developed the real-time control system for the MIRC beam combiner.
Paul A. Dalgarno received his MPhys degree from Heriot-Watt University in 2001, and his PhD in physics in 2005. He began his career studying self-assembled quantum dots, specializing in time resolved measurements, before moving to St Andrews in 2010 to work on biopolymer folding and organic polymers. Since 2012, he has been an assistant professor at Heriot-Watt, leading the Advanced Microscopy Group, which works on high resolution, multispectral, and 3-D microscopy for the life sciences.
Emma Scholefield studied environmental biology and has 15 years’ biological research experience at the University of Edinburgh. She has spent the last six years within the Pulmonary Optical Molecular Imaging group developing novel optical imaging probes to detect infection and inflammation in the distal lung. Currently, she is the quality and operations manager enabling the transition from bench to bedside to allow first-in-man studies of optical imaging agents.
John M. Girkin is the professor of biophysics at Durham University, UK. Having worked in industry designing ophthalmic instruments, including the puff tonometer and diode laser photocoagulator, his research concentrates on the development of optical instrumentation for the life sciences. His recent work has focused on optical microscopy combining adaptive optics with SPIM and multiphoton imaging. A theme running through his research is to make practical instruments for use in the laboratory or clinic.
Anne Moore studied mathematics at Cambridge before receiving her PhD in signal processing from the University of Edinburgh. She then worked for 16 years in industry on experimental phased array radar projects. In 2007, she returned to academia completing her PhD in molecular plant science at the University of Edinburgh. Currently, she manages the EPSRC IRC Proteus project.
Mark Bradley is a professor of chemistry at the University of Edinburgh. He is the director of the EPSRC IRC Proteus Project and cofounder of three active university spinouts. The main focus of the Bradley group is the application of the tools of chemistry to allow the manipulation, control, and understanding of specific biological processes and functions and to address specific biological questions.
Kevin Dhaliwal is a clinician scientist. He leads the IRC Proteus Interdisciplinary hub in the MRC Centre for Inflammation Research at the University of Edinburgh. He trained in medicine in Edinburgh before completing general profession training in London. He received his PhD supported by a MRC clinical training fellowship. He was appointed as a senior clinical lecturer in pulmonary molecular imaging in 2013. His work focuses on the clinical translation of optical approaches to improve pulmonary medicine.