10 July 2018 Low-cost high sensitivity pulsed endomicroscopy to visualize tricolor optical signatures
Author Affiliations +
A highly sensitive, modular three-color fluorescence endomicroscopy imaging platform spanning the visible to near-infrared (NIR) range is demonstrated. Light-emitting diodes (LEDs) were sequentially pulsed along with the camera acquisition to provide up to 20 frames per second (fps) three-color imaging performance or 60 fps single color imaging. The system was characterized for bacterial and cellular molecular imaging in ex vivo human lung tissue and for bacterial and indocyanine green imaging in ex vivo perfused sheep lungs. A practical method to reduce background tissue autofluorescence is also proposed. The platform was clinically translated into six patients with pulmonary disease to delineate healthy, cancerous, and fibrotic tissue autofluorescent structures. The instrument is the most broadband clinical endomicroscopy system developed to date (covering visible to the NIR, 500 to 900 nm) and demonstrates significant potential for future clinical utility due to its low cost and modular capability to suit a wide variety of molecular imaging applications.



Optical molecular imaging (OMI) promises to generate insights into disease processes in vivo1 with the potential to provide in vivo molecular information in real-time based on signatures of disease. However, to deliver on this potential, there is a requirement for the development of imaging agents that allows the fluorescent detection of disease targets alongside a versatile imaging platform with multiplexing capabilities.

The core aim of this work was to demonstrate a modular engineering solution for pulmonary fluorescent imaging23.4 with cellular and bacterial imaging5,6 capability. We demonstrate that a modular, low cost, clinically translated, optical imaging platform can delineate structural, bacterial, and cellular imaging in situ and in vivo. The technical configuration advances our previous endomicroscopy platform7 as follows: (i) the number of colors is expanded and extended into near-infrared (NIR); (ii) spectroscopy is incorporated into the detection channel; (iii) the limit-of-detection (LoD) is improved; (iv) a background suppression technique is introduced (droplet capture); (v) an intuitive graphical user interface (GUI) is introduced; and (vi) the unit was translated into clinical pulmonary imaging.

While clinical practice mostly deploys white-light endoscopy, molecular imaging is augmented by fluorescence-based imaging. As our immediate aim is to interrogate molecular processes in the distal human lung in vivo,7 we have focused on fluorescence endomicroscopy with access to the distal lung (alveolar space), mediated via a transbronchial pass with a narrow diameter device (<2  mm).

Our approach complements other fluorescence endomicroscopy platforms, which are at various stages of development. Scanning fiber endoscope (SFE) systems have been developed,8 with a high-resolution SFE endomicroscopy system achieving bacterial imaging in the gut,9 but this endoscope platform is not suitable for accessing the distal lung due to its wide bore. Miniaturized multicolor SFE10,11 has also been implemented in imaging atherosclerotic plaques in ex vivo murine tissue.12

Fiber-based endomicroscopy platforms utilizing proximal camera imaging have been investigated1314.15 and translated into clinic in a single-color mode1617.18 with the described system (called Versicolour henceforth), significantly extending this capability as the first multicolor endomicroscopy system to be translated into humans for in vivo imaging. In this manuscript, we characterize our three-color configuration and demonstrate its performance in bacterial imaging in ex vivo models and imaging pulmonary structure in humans in vivo in situ.


Materials and Methods


Device Description

The layout of the optics and electronics inside Versicolour is shown in Fig. 1(a). A laptop controls the image acquisition sequence by triggering the LEDs in sequence, as shown in Fig. 1(b). To avoid spectral mixing, the fluorescence image from each channel is acquired with only one LED turned on at a time, with the camera triggered to acquire an image synchronously when the LED is turned on. Therefore, to acquire a three-color image, the trigger unit (NI9402, National Instruments, United States) was programmed to generate a set of three individual trigger pulses, where the pulse period dictated the exposure time (our preprogrammed exposure times were 15, 25, 50, 100, and 250 ms corresponding to 20, 10, 6, and 1.3 fps three-color imaging), see Fig. 1(b). All three channels had identical exposure times and gain due to camera firmware limitations. The pulsing architecture supersedes simultaneous two-color imaging7 by acquiring spectrally unconstrained acquisitions for each spectral channel. A previous technical configuration7 had been limited by the Bayer filter inherent in most color camera solutions, which preselects the spectral range for each channel and completely removes any NIR capability.

Fig. 1

(a) Electro-optical architecture of Versicolour and (b) triggering sequence for 25-ms exposure time for each individual channel resulting in 10 fps imaging speed for combined three-color images (minor delay between each pulse inserted for camera hardware to handle the image transfer).


The technical configuration of Versicolour was based on our previous work7 with significant improvements in adapting the architecture to three-color fluorescence endomicroscopy and molecular imaging applications. The improvements include the addition of a NIR imaging and a parallel spectroscopy channel (Flame-S spectrometer, Ocean Optics, United States). Three LEDs (470 nm LED M470L3, 625 nm LED M625L3, 780 nm LED M780F2, all from Thorlabs, United Kingdom) were combined using achromatic condensers for collimation (ACL2520U-A for 470 nm and 625 nm LEDs and ACL2520U-B for 780 nm LED) and dichroics for combination. A dichroic filter (FF506-DI03-25X36, Semrock, United States) combined collimated blue light from the 470-nm LED with red and NIR collimated beams, which were combined using a separate dichroic (FF652-DI01-25X36, Semrock). The custom multiband filterset was manufactured by Chroma, United States. The excitation and emission curves are shown in Figs. 2(a) and 2(b) along with absorption and emission curves from representative fluorescent dyes: nitrobenzoxadiazole (NBD), fluorescein, cyanine 5 (Cy5), methylene blue, and indocyanine green (ICG). Apart from methylene blue, we have tested all dyes listed. Current human in vivo imaging uses either approved (fluorescein and ICG) or nontoxic (methylene blue) dyes.19 For both NBD and Cy5, we are developing a range of clinically translatable SmartProbes.6,20,21

Fig. 2

(a) Pass and stop bands of the custom three-color excitation filter along with the absorption curves of representative fluorescent dyes (NBD, fluorescein, Cy5, methylene blue, and ICG) are shown and (b) the pass and stop bands for the custom three-color emission filter and dichroic are shown along with the emission curves of representative fluorescent dyes for each channel (NBD, fluorescein, Cy5, methylene blue, and ICG).


A multiband excitation filter passes the collimated LED light to the imaging fiber bundle via a multiband dichroic and a microscope objective (MO) (RMS10X-PF, Olympus). The fluorescence generated at the distal end of the fiber was transmitted through the fiber and via a multiband emission filter and tube lens (ITL200, Thorlabs, United Kingdom) to a fast monochromatic CMOS camera (GS3-U3-23S6M-C, Grasshopper3, Point Grey Research, Canada). On the emission path, a 50:50 nonpolarizing beamsplitter (BS013, Thorlabs) transmits fluorescence into a spectrometer (Flame-S, Ocean Optics). The exposure time for spectrometer measurements was set to 2 s.

Custom real-time software was written in C++ with wxWidgets user interface classes. An intuitive GUI was developed for clinical use and the Versicolour unit was placed on a medically approved cart (see Fig. 3). To enable clinical translation, Versicolour was developed within the framework of a quality management system following the principles of BS EN 13485:2012. Risk management and mitigation were organized around BS EN 14971:2012. Software development followed the international standard BS EN 62304:2006 Medical Device Software principles. Versicolour passed electromagnetic compatibility (EMC) tests for radiated and conducted emissions (York EMC Services, Grangemouth, United Kingdom) and all locally required optical and electrical safety testing for use in the interventional pulmonology suite. Instrument enclosure mechanical designs (EG Technology Ltd., Cambridge) ensured ingress and electrical safety compliance following BS EN 60601-1.

Fig. 3

The medically approved cart with large display and the Versicolour unit can be easily transported between hospital departments.


The total cost of the bill of materials for one-off manufacture is $15000. The projected unit cost for 10 units (based on quotations) is <$5000 without spectrometer and <$8000 with spectrometer. As the design is breadboard based with relatively expensive electronics (e.g., National Instrument Cards), we believe that a commercial imaging design can be produced for less than $3000 for the manufacture of 10 units (without spectrometer). Furthermore, by deploying standard product design and optical engineering techniques (e.g., single metal block design for optics rather than breadboard based design), this can be reduced further. A similar single-color confocal scanning unit sells for 100,000$.22 One needs to take into account that regulatory aspects related to Food and Drug Administration approval in USA and Conformité Européene marking in the European Union can take up a significant fraction of the cost of the end unit. However, even in this respect we believe that the sheer simplicity of the design presented here warrants faster translation to clinic.



The LoD for fluorescein (F6377, Sigma-Aldrich), Cy5 (777323, Sigma-Aldrich), and ICG (21980, Sigma Aldrich) solutions was determined using aqueous solutions of the fluorophores (5, 10, 20, and 30 nM in deionized water). The tip of the imaging fiber bundle was inserted into the solution, and several images were taken for each concentration with a 100-ms exposure time. A 100×100  pixel area was selected from the central area of the imaging fiber bundle. Mean and standard deviation (SD) were calculated for the area and their ratio was ascribed as the signal-to-noise ratio (SNR). To remove variation attributable to the fiber bundle pattern, the SD value was derived from the SD of the same area but for a difference of two subsequent images. The difference removes the pattern while the noise inherent to image acquisition increases (by a factor of 2). To simplify matters, we defined the LoD as the detectable concentration of respective dyes resulting in a 21 SNR.7,23


Chromatic Shift Test

To estimate the chromatic focal shift for all color channels (green, red, and NIR), the fiber imaging bundle was dipped into solutions of fluorescein, Cy5, and ICG (30 nM). A Z axis stage (SM1Z, Thorlabs), which places the proximal side of the fiber imaging bundle into the focus, was moved along a 50  μm distance in steps of 1  μm. A bandpass filter between 520 and 560 nm was placed in the fluorescence emission path for green channel measurements due to red fluorescence from the fiber at 470-nm excitation.7,24 This allowed the fluorescence intensity to be evaluated from a single core versus focal position in front of the MO. Maximum fluorescence from the respective solutions indicated the optimal focus for a given channel.

A diluted solution of Constellation™ microspheres (ThermoFisher Scientific, United States) was loaded onto the distal tip of the imaging fiber and allowed to dry (5  min) and used to evaluate the image quality for fluorescence from microspheres in green, red, and NIR with the aim of establishing the image quality in the optimal focus.


Ex vivo Human Lung Tissue Imaging with Labeled Cells and Bacteria

Ethics statement: All experiments using human samples in vitro were performed following approval of the appropriate regional ethics committee (REC), NHS Lothian (reference 15/ES/0094), and with informed consent of the patients.

Ex vivo human lung tissue, coincubated with mammalian and bacterial cells, was used as a model to evaluate cellular and bacterial imaging performance in multicolor scenarios. The 1×4  mm sections of excised human lung tissue were placed into a 96-well tissue culture plate (Corning) and incubated with 50  μL of prelabeled cells. The tissues were harvested from the periphery of specimens taken from the lung cancer resections. Overnight cultures of Staphylococcus aureus ATCC25923 grown in Luria-Bertani (LB) broth (37°C, 250 rpm) were adjusted to an optical density at 595 nm (OD595) of 1 and washed in phosphate-buffered saline (PBS, Gibco). The bacteria were labeled with Cellvue® Claret (Sigma-Aldrich), as per manufacturer’s instructions, and resuspended in a final volume of 1 mL PBS (1×109  CFUmL1). The murine macrophage cell-line RAW264.7 (ATCC) was grown in complete Dulbecco’s modified Eagle’s medium (Gibco) culture medium plus 1% pen/strep to 80% confluence in T75 tissue culture flasks. RAW264.7 cells were harvested and adjusted to 5×105  cells/mL. IR775 (IR-775 chloride, 544914, Sigma-Aldrich) was incubated with the cells at 10  μM, 37°C, 30 min followed by washing three times in PBS by centrifugation (5 min, 400×g). Primary human neutrophils were isolated, as previously described,25 with the number of retrieved neutrophils was determined with NucleoCounter NC-1000 (Chemo Metec). Neutrophils were resuspended at a concentration of 5×105 per mL in PBS and labeled with 1  μM calcein AM (Sigma-Aldrich), 30 min 37°C, followed by washing three times in PBS by centrifugation (5 min, 400×g). Following labeling, the cells were added to the ex vivo human lung tissue and imaged with Versicolour by placing the distal end of the imaging fiber (Alveoflex™, Mauna Kea Technologies, France) in direct contact with the tissue. Images were captured with a 25-ms exposure time.


Sheep Lung Imaging

Ex vivo whole ventilated ovine lung experiments were performed to evaluate performance in a human size-relevant model and in particular to determine proof of concept for: (i) bacterial imaging with a gram-negative specific bacterial imaging SmartProbe20 (three sheep lungs experiments) and (ii) blood flow imaging in NIR using ICG26 (one sheep lung experiment).

Following the same methodology as described before Ref. 27, ovine lungs were obtained from ewes destined for cull and ventilated. Gram-negative Pseudomonas aeruginosa 3284 were cultured overnight in LB, adjusted to an OD595 1 in PBS and labeled with Syto60 (1  μM, ThermoFisher Scientific), followed by washing three times in PBS providing a red counterstain. Delivery of bacteria and a gram-negative SmartProbe were performed as follows: bronchoscopy (Pentax EPM-1000) was used to guide to the distal lung location and a flexible APC probe (ERBE) catheter was then inserted into the working channel of the bronchoscope to instill 2 ml of bacteria OD595 2 (109  CFU/ml). The ERBE catheter was removed and an Alveoflex™ imaging fiber bundle was passed through the working channel of the bronchoscope and guided to the subsegments, where bacteria had been instilled. Imaging was performed in the distal lung on at least two locations for at least 10 s for each imaging session. The imaging fiber was removed and 2 ml of the gram-negative SmartProbe (80  μg/5  ml) was delivered into the same location. The imaging fiber was then reinserted and guided to the subsegments with imaging was performed as previously. Between each delivery, the bronchoscope was cleaned and disinfected using 8% H2O2. Clinical Alveoflex™ (Mauna Kea Technologies) was used as the imaging fiber bundle with Versicolour. For ICG imaging, ovine lungs were obtained and perfused as above. The 2 ml of 1 mM ICG were added to 2 L of perfusate and imaging was performed 2 min after ICG was added to the perfusate. The exposure time of the camera was set to 50 ms.


Human in vivo Imaging

Versicolour was deployed in a clinical study, clinicaltrials.gov identifier NCT02604862, following approval of the appropriate REC, NHS Lothian (reference 15/SS/0235), and with informed consent of the patients. In these cases, a clinical Alveoflex™ (Mauna Kea Technologies) was used as the imaging fiber bundle.

All patients were scheduled to undergo routine bronchoscopy in an outpatient procedure suite using local anesthetic and conscious sedation. Patients were recruited if they had an underlying diagnosis of diffuse parenchymal lung disease or suspected lung cancer. This allowed us to use our endomicroscopy system to characterize disease of both the airways and the alveolar space. As patients were already scheduled to undergo bronchoscopy, we were able to perform our imaging study in the same sitting as diagnostic bronchoscopic investigation, such as lavage, brushings, biopsy, and transbronchial ultrasound guided needle aspiration without complication.

Areas of interest were selected based on cross-sectional imaging (thoracic computerized tomography), which was used to determine which pulmonary segments were examined using endomicroscopy. For imaging of fibrotic and healthy alveolar regions, the bronchoscope was directed to the segment of interest before passing the Alveoflex™ down the working channel and past the point of vision. The endomicroscopy system enables real-time feedback as the fiber passes along bronchial structure before achieving a transbronchial push and emerging into the alveolar space with visual confirmation. For imaging of bronchial abnormalities such as lung cancer, the lesion was identified using the bronchoscope before the fiber was advanced to make contact under direct vision with the diseased mucosa. The total number of images acquired was 38,880 and the total number of spectra was 663. White light from the bronchoscope was briefly switched off during spectral acquisition for bronchus, but this was not required after transbronchial push into alveolar space. Bronchoscopy and endomicroscopy were completed in six patients (Table 1), and following the procedure, they were monitored for 4 h before discharge and all received a follow-up telephone call 24 h later. There were no serious adverse events recorded in this study.

Table 1

Six patients recruited to the clinical study. NSCLC, nonsmall cell lung cancer; SCLC, small cell lung cancer; IPF, idiopathic pulmonary fibrosis.

PatientGenderAgeDiagnosisBronchoscopy duration (min)aVersicolour imaging duration (min)


This included the gathering of other diagnostic investigations as required.


Autofluorescence Background Suppression Methods

To reduce the effects of autofluorescence background from lung tissue, the following two methods were evaluated. First, in human lung imaging in situ, the method of Muldoon et al.13 was adopted, where the fiber was gently pulled back once the distal lung tissue was reached, with reduced pressure on the tissue reducing autofluorescence.

In addition, we further explored droplet capture onto the endoscope’s tip for tissue background removal. For droplet analysis, the tip of the fiber captures liquid from wet tissue through surface tension adsorption. We utilized the captured liquid for separate imaging or spectroscopy analysis, because the tissue background is inherently removed. This is especially useful for molecular imaging of enzyme activity probes, where the spectral signature may overlap with tissue autofluorescence if the enzyme reporter has a similar fluorescent reporter. To evaluate the methodology, we placed excised human lung tissue in two microwells of a 96-well plate. The first microwell contained 150  μl of PBS, whereas the second contained a 3  μM solution of fluorescein amidite (FAM) in 150  μl of PBS. Spectroscopy was performed on tissue by placing the fiber tip on tissue in the microwell. After this, the tip was retrieved and a spectrum was acquired of the droplet captured on the tip of the imaging fiber bundle. Retrieving the fiber tip from the solution captured 400  nl droplet by surface tension. Five spectra were taken from each tissue in the microwell and the respective droplets. Exposure time for each spectroscopy acquisition was 2 s.


Imaging Settings

The imaging settings for Versicolour were as follows: images acquired were 960×600  pixels, camera exposure time was typically set to 25 ms (10 fps three-color imaging) unless otherwise stated. The exposure time needs to be adjusted so it acquires enough fluorescence from tissue and other optical signatures in all color channels. Our main clinical constraint has also been to achieve 10 fps in three-color imaging to allow instant viewing and reaction to endoscope positioning. After evaluation, we chose a 25-ms exposure time as it was adequate for most of our three-color imaging experiments. For each imaging experiment, a fluorescence background image was taken from a blackened Eppendorf filled with deionized water and subtracted from all acquired images. For displaying clinical images, we performed a division of acquired image with a selected lightfield image taken from the same sequence.





LoD analysis is shown in Fig. 4(a) (LoD here defined as the concentration of fluorophore, which generates a SNR of 21). The LoD was 5.7 nM for fluorescein, 2.4 nM for Cy5, and 8.4 nM for ICG for 100-ms exposure time (or 22.8 nM, 9.6 nM, and 33.6 nM, respectively, for 25-ms exposure time assuming read noise limited detection). As CMOS camera technologies are rapidly improving (as are increases in in LED optical power), reductions in camera noise will push LoDs to subnanomolar concentrations in dark background scenarios. In our previous implementation,7 LoDs for fluorescein and Cy5 were 10 and 50 nM, respectively, and thus, the LoD for fluorescein and Cy5 has improved by a factor of 2 and 20.8, respectively. The main reasons for these improvements are lower camera noise, improved collection efficiency due to monochrome detection, and increased LED optical power (4 mW on the output of the imaging fiber bundle for 470- and 625-nm LEDs and 1.9 mW for 780-nm LED). The effect is more dramatic in the red fluorescence channel, because previously in the color camera, the fiber fluorescence was detected in the red pixels of the Bayer filter, whereas now fiber fluorescence is detected during the 470-nm LED trigger (as shown before, the imaging fiber bundle has a long Stokes shift with 470-nm excitation7,24). We subtract the background (fiber fluorescence and camera dark noise) for each color band, but as mentioned before,7 the Poisson noise of this background remains the limiting factor.

Fig. 4

(a) SNR for 5, 10, 20, and 30 nM solutions of fluorescein, Cy5, and ICG. (b) Contrast for each color channel depended on proximal fiber end location. Optimal foci for green and NIR channels are separated by 25  μm.



Chromatic Shift Evaluation

Figure 4(b) shows the separation of optimal foci for green, red, and NIR channels. Green, red, and NIR channels were equally separated by 13  μm. Individual cores were visible across this range. Optimally, three-color imaging (green, red, and NIR) requires the focus to be set for the red channel [focus location at 25  μm in Fig. 4(b)]. For optimal two color imaging in green and red, the focus is set between green and red peaks [17  μm in Fig. 4(b)].

To illustrate that the optimal focus for three-color imaging is sufficient to image small objects, we show in Fig. 5 an image of Constellation™ microspheres dried on the tip of the imaging fiber bundle. Figure 5(a) has the three channels displayed separately to illustrate overall color colocalization, and Fig. 5(b) has them merged in single RGB image. The two line profiles were chosen to show colocation of green and red fluorescence (line profile 1) and separately, red and NIR fluorescence (line profile 2), both in Fig. 5(c). There are no colocating three-color microspheres, and NIR microspheres were always much larger than a single core, thus limiting our analysis for this band. Green and red fluorescence from the small microspheres generated signals in a single core in line profile 1 while red and NIR fluorescence in line profile 2 matched each other well despite the NIR having lower SNR. For saturating regions in green, we observed core-to-core coupling,28 which reduces contrast.

Fig. 5

(a) Constellation™ microspheres dried on the tip with each color channel displayed separately (green, red, and NIR) and merged in (b) with two line profiles shown in (c). Line profile 1 of a microsphere fluorescing shows green and red fluorescence. Line profile 2 of a larger microsphere shows red and NIR fluorescence.



Multicolor ex vivo Human Lung Tissue Imaging

Figure 6 demonstrates three-color imaging in ex vivo human lung tissue at 10 fps. Macrophages fluorescing in the NIR (displayed as blue) and bacteria fluorescing in red can be seen in the same field-of-view (FoV) [Figs. 6(a)6(c)]. Ex vivo human lung tissue autofluorescence is captured in the green fluorescence channel as well as calcein-labeled neutrophils [Fig. 6(c)]. Most of human lung tissue autofluorescence is derived from the elastin fiber network.2 Figure 6(b) has tissue out of focus due to motion of the probe, whereas the lung elastin fiber network is sharper in Figs. 6(a) and 6(c). Figure 6(c) shows colocating macrophages, neutrophils, and bacteria toward the center of the image. While depth information is not available to guarantee colocation in 3-D, we were encouraged to pursue imaging scenarios such as this in vivo in situ and add technical methodologies to improve optical sectioning. In summary, we have demonstrated three-color cellular and bacterial resolution imaging across the visible to NIR spectrum.

Fig. 6

(a) and (b) Ex vivo human lung (green fluorescence) with macrophages [NIR shown as blue from IR775, white arrows pointing to three in (a)] and bacteria (red fluorescence from Cellvue® Claret). Tissue in panel (b) is mostly out of focus (white arrows). (c) Ex vivo human lung (green fluorescence) with macrophages (NIR shown as blue from IR775), bacteria (red fluorescence from Cellvue® Claret), and neutrophils (strong green fluorescence from calcein labelled neutrophil—arrow).


In future applications that potentially involve exogenously delivered fluorophore SmartProbes20 to the distal lung, distinguishing signal from noise will be challenging if the fluorophores’ emission spectrum overlaps with elastin. Thus, background suppression techniques discussed below represent an important aspect for in vivo imaging.


Ventilated Ovine Lung Imaging

Moving from ex vivo imaging of small sections of patient-derived human lung tissue, the next stage of development was to demonstrate utility to derive optical signatures in a human size relevant model. In this regard, we pursued initial proof of concept in a ventilated and perfused ovine lung model in a dedicated setup [Fig. 7(a)]. The imaging bundle was navigated to the distal alveolar regions where imaging of bacteria is present [the bacteria were counterstained with Syto60 (red) prior to instilling into the lung and targeted with bacteria-specific SmartProbe (green) in situ]. Figure 7(b) shows example frames from one ex vivo ventilated sheep lung, each with representative green and corresponding red channels imaged simultaneously. No green punctate signals were observable in the green channel images prior to instilling gram-negative SmartProbe while red punctate Syto60 labeled bacteria were visible [top row in Fig. 7(b)]. After instilling gram-negative SmartProbe (with NBD green fluorophore), colocating red and green punctate signals were observable [bottom row in Fig. 7(b)]. The same results were obtained in the further two ovine lungs tested.

Fig. 7

(a) Perfusion set-up used in sheep lung experiments containing the following components: 1, organ chamber; 2, suction reservoir; 3, perfusate reservoir; 4, heater; 5, pump control; 6, gas membrane diffuser; 7, centrifugal pump head. (b) Ex vivo ventilated sheep lungs (each row) with green channel, corresponding red channel and combined image (three columns) prior to instilling gram-negative SmartProbe onto gram-negative bacteria (first row) and after instilling gram-negative SmartProbe onto gram-negative bacteria (second row). (c) Incoming wave of NIR fluorescence from ICG in the perfusate filling the distal lung of the ventilated and perfused ovine lungs.


Figure 7(c) shows sheep lung imaging snapshots with perfusate inflow. The ICG stained perfusate flow was observable in the distal lung for each pump cycle of the perfusate entering the FoV. The imaging fiber bundle was embedded in the distal lung to keep the FoV static. The NIR signal from ICG was diffused and blood vessels were not observable, probably due to trauma caused by the fiber insertion. After 20 min, the ICG started to stain the tissue.


Clinical Imaging in the Distal Human Lung

We embarked upon an exploratory clinical study with Versicolour with the primary aim to generate proof of concept data that Versicolour could delineate autofluorescent signatures of pulmonary elastin organization. We thus selected patients in whom the elastin network would be expected to be disordered. We used CT scans to help guide imaging to normal areas as control images.

Figure 8 demonstrates three in vivo imaging scenarios of normal bronchial epithelium (left column). Two example images with microvessels are shown in Fig. 8 (middle column). The alveolar space was reached (Fig. 8, right column) after pushing the imaging fiber bundle gently into acinar subunits. Alveolar septa are clearly delineated in normal alveolar space images (Fig. 8, right column). Broken epithelium in nonsmall cell lung cancer (NSCLC) is shown in Fig. 9, left column, whereas broken epithelium in small cell lung cancer (SCLC) is shown in Fig. 9, middle column. Fibrotic lung (alveolar space) is shown in Fig. 9, right column. Overall, the level of autofluorescence varied from patient to patient but also within the same patient. Some structures fluoresce strongly (Fig. 9, right column, fibrotic lung) while bronchial epithelium and some normal alveolar structures fluoresce weakly resulting in low SNR images (Fig. 8, top right and Fig. 9, middle column lower image). The varying autofluorescence is likely to be attributable to varying abundance of elastin and collagen in the native structures. Our comparison between normal and disrupted tissue was not blind in this clinical study. Clinically, prior information from CT and white-light bronchoscopy provides important input in judging endomicroscopy imaging.

Fig. 8

Human in vivo imaging with examples of bronchial epithelium (left column, arrow pointing in the direction of normal elastic fibred network), distal lung with blood microvessel (middle column, arrow pointing in the direction of the vessel), and normal alveolar space lung (right column, arrow pointing to alveolar septa). All images were scaled individually for intensity to maximize contrast.


Fig. 9

The disturbed bronchial epithelium in NSCLC (left column, arrow pointing at fibred networks crossing each other) and SCLC (middle column, arrow pointing at fibred networks crossing each other). Fibrotic (distal) lung with disturbed alveolar structure is shown in right column, no alveolar septa can be found.


Given the widefield capture of images and the lack of sectioning, background autofluorescence contributes to “noisy” images. In our clinical experience in the distal lung (alveolar regions), this can be significantly attenuated by a small retraction (200 to 400  μm) of the imaging bundle (see Fig. 10).

Fig. 10

Background suppression by retracting the fiber. (a) and (b) High background image in alveolar space can be improved if the clinician retracts the fiber in order to reduce the amount of elastin pressed beneath the imaging fiber bundle.


In addition, we explored background suppression by capturing droplets on the endoscope tip ex vivo. Figure 11 illustrates how the droplet suppresses tissue autofluorescence background. Spectra on tissue without FAM [Fig. 11(a)] and with FAM [Fig. 11(b)] are similar in intensity and shape, making it difficult to detect the presence of FAM. Droplet from the tissue without FAM does not fluoresce [Fig. 11(c)], whereas the droplet from tissue with FAM demonstrates strong spectra [Fig. 11(d)]. Droplet spectroscopy or imaging therefore allows for background suppression and could be useful for accurate enzyme activity analysis, where droplet capture is possible in vivo.

Fig. 11

While ex vivo human lung tissue (a) without FAM and (b) with FAM has similar spectra, (c) the droplet from tissue well without FAM does not fluoresce (d) while the droplet from tissue well with FAM has spectra corresponding to FAM.




As the only currently clinically approved endomicroscopy system for respiratory medicine is probe-based confocal laser endomicroscopy, clinicians rely on autofluorescence3 or intravenous fluorescein for alveolar imaging29 and microvessel localization.30 Other studies have used acriflavin hydrochloride topical staining for cellular imaging in vivo31 in green. Single color systems suffer from overlaps between green autofluorescence and green fluorescence staining, so there is an ongoing effort to verify red-shifted fluorescence cellular imaging in vivo such as with methylene blue.4 Our platform supports simultaneous imaging for numerous scenarios, such as methylene blue for cellular imaging in red and ICG for microvessel localization in NIR. We also observed weak red autofluorescence from alveoli during in vivo imaging in two out of six patients, but it is difficult to draw conclusions from such a small sample (see Fig. 12 for weak red fluorescence from alveolar septa).

Fig. 12

In two patients, alveolar space was visible in green and red channels. Green channel and matching red channel are combined into a single two color image. (a) Green channel, (b) red channel, and (c) combined.


With the development of optical imaging SmartProbes that are likely to be spread across the optical spectrum, the utility of modular cost effective solutions will be essential to realize the potential of OMI. In the case of Versicolour, the LEDs and filters can be simply interchanged to accommodate spectral windows.

The droplet method explored is very practical for single pass capture of surface liquids. While it exploits naturally occurring biofouling, it is also limited to single capture in vivo. In addition, it is not practical for the transbronchial pass. Despite the limitations, we feel it to be a very practical method for background suppression in all other situations. While we tested the method with spectroscopy, it is also applicable to imaging. For example, for imaging bacteria labeled in green where tissue background fluorescence masks most of them, we have observed more bacterial “dots” to reappear in the droplet on dark background thus improving detection.

In widefield imaging scenarios, optical sectioning remains an issue. Fluorescence emanates from up to 250  μm away and reaches the camera through the imaging fiber bundle.21 While this reduces contrast by increasing background, the system performs well for image guidance into distal lung tissue and bacterial imaging despite the fact that we do not perform any image processing. In addition, colocalization of objects in three channels will be a challenge in fast moving environments during in vivo imaging. We have found that a 25-ms exposure time is a good compromise in this respect. We have also found that exposure times above 100 ms cause significant blur during in vivo imaging. Our ambition in future work will be to address both 2-D and 3-D colocalization in order to bring endomicroscopy closer to the performance of bench top fluorescence microscopy.

Further technical developments include adding image processing,32 adopting structured illumination techniques,3334.35 and exploiting the acquisition speed of low noise CMOS cameras. These modular improvements offer a broad potential platform to exploit fluorescence-based endomicroscopy coupled with the emergence of low-cost disposable imaging fibers.36



We have demonstrated the imaging performance of a modular and highly economical endomicroscopy system in vivo and ex vivo. Three-color imaging across visible and NIR channels has the potential to provide multiple molecular and autofluorescent targets to be imaged concurrently at video rate, thus expanding the capability of endomicroscopy. Importantly, the low cost of the technology platform may enable the widespread dissemination of clinical endomicroscopy.


The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.


We would like to thank the Engineering and Physical Sciences Research Council (EPSRC, United Kingdom) Proteus Interdisciplinary Research Collaboration ( www.proteus.ac.uk): grant EP/K03197X/1 for funding this work. Clinical study was funded by the Medical Research Council (MRC, United Kingdom) grant number MR/J014702/1. Datasets for this paper can be found at  dx.doi.org/10.7488/ds/2355.


1. V. Ntziachristos, “Fluorescence molecular imaging,” Annu. Rev. Biomed. Eng. 8, 1–33 (2006).ARBEF71523-9829 https://doi.org/10.1146/annurev.bioeng.8.061505.095831 Google Scholar

2. L. Thiberville et al., “In vivo imaging of the bronchial wall microstructure using fibered confocal fluorescence microscopy,” Am. J. Respir. Crit. Care Med. 175, 22–31 (2007).AJCMED1073-449X https://doi.org/10.1164/rccm.200605-684OC Google Scholar

3. L. Thiberville et al., “Human in vivo fluorescence microimaging of the alveolar ducts and sacs during bronchoscopy,” Eur. Respir. J. 33, 974–985 (2009). https://doi.org/10.1183/09031936.00083708 Google Scholar

4. L. Thiberville et al., “Confocal fluorescence endomicroscopy of the human airways,” Proc. Am. Thorac. Soc. 6, 444–449 (2009). https://doi.org/10.1513/pats.200902-009AW Google Scholar

5. M. van Oosten et al., “Targeted imaging of bacterial infections: advances, hurdles and hopes,” FEMS Microbiol. Rev. 39, 892–916 (2015).FMREE40168-6445 https://doi.org/10.1093/femsre/fuv029 Google Scholar

6. B. Mills, M. Bradley and K. Dhaliwal, “Optical imaging of bacterial infections,” Clin. Transl. Imaging 4, 163–174 (2016). https://doi.org/10.1007/s40336-016-0180-0 Google Scholar

7. N. Krstajic et al., “Two-color widefield fluorescence microendoscopy enables multiplexed molecular imaging in the alveolar space of human lung tissue,” J. Biomed. Opt. 21, 046009 (2016).JBOPFO1083-3668 https://doi.org/10.1117/1.JBO.21.4.046009 Google Scholar

8. M. Goetz, N. P. Malek and R. Kiesslich, “Microscopic imaging in endoscopy: endomicroscopy and endocytoscopy,” Nat. Rev. Gastroenterol. Hepatol. 11, 11–18 (2014). https://doi.org/10.1038/nrgastro.2013.134 Google Scholar

9. R. Kiesslich et al., “Diagnosing helicobacter pylori in vivo by confocal laser endoscopy,” Gastroenterology 128, 2119–2123 (2005).GASTAB0016-5085 https://doi.org/10.1053/j.gastro.2004.12.035 Google Scholar

10. C. M. Lee et al., “Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging,” J. Biophotonics 3, 385–407 (2010). https://doi.org/10.1002/jbio.v3:5/6 Google Scholar

11. C. Yang et al., “Mitigating fluorescence spectral overlap in wide-field endoscopic imaging,” J. Biomed. Opt. 18, 086012 (2013).JBOPFO1083-3668 https://doi.org/10.1117/1.JBO.18.8.086012 Google Scholar

12. L. E. Savastano et al., “Multimodal laser-based angioscopy for structural, chemical and biological imaging of atherosclerosis,” Nat. Biomed. Eng. 1, 0023 (2017). https://doi.org/10.1038/s41551-016-0023 Google Scholar

13. T. J. Muldoon et al., “Subcellular-resolution molecular imaging within living tissue by fiber microendoscopy,” Opt. Express 15, 16413–16423 (2007).OPEXFF1094-4087 https://doi.org/10.1364/OE.15.016413 Google Scholar

14. M. Pierce, D. Yu and R. Richards-Kortum, “High-resolution fiber-optic microendoscopy for in situ cellular imaging,” J. Visualized Exp. 47, e2306 (2011). https://doi.org/10.3791/2306 Google Scholar

15. M. H. Koucky and M. C. Pierce, “Axial response of high-resolution microendoscopy in scattering media,” Biomed. Opt. Express 4, 2247–2256 (2013).BOEICL2156-7085 https://doi.org/10.1364/BOE.4.002247 Google Scholar

16. T. J. Muldoon et al., “High-resolution imaging in Barrett’s esophagus: a novel, low-cost endoscopic microscope,” Gastrointest. Endosc. 68, 737–744 (2008). https://doi.org/10.1016/j.gie.2008.05.018 Google Scholar

17. T. J. Muldoon et al., “Noninvasive imaging of oral neoplasia with a high-resolution fiber-optic microendoscope,” Head Neck 34, 305–312 (2011). https://doi.org/10.1002/hed.21735 Google Scholar

18. M. C. Pierce et al., “Low-cost endomicroscopy in the esophagus and colon,” Am. J. Gastroenterol. 106, 1722–1724 (2011). https://doi.org/10.1038/ajg.2011.140 Google Scholar

19. B. Obstoy et al., “Safety and performance analysis of acriflavine and methylene blue for in vivo imaging of precancerous lesions using fibered confocal fluorescence microscopy (FCFM): an experimental study,” BMC Pulm. Med. 15, 30 (2015). https://doi.org/10.1186/s12890-015-0020-4 Google Scholar

20. A. R. Akram et al., “T4 optically detectable antimicrobial peptides enable the immediate detection of bacteria and fungi in the lung,” Thorax 70, A2–A3 (2015).THORA70040-6376 https://doi.org/10.1136/thoraxjnl-2015-207770.4 Google Scholar

21. A. Megia-Fernandez et al., “Highly selective and rapidly activatable fluorogenic Thrombin sensors and application in human lung tissue,” Org. Biomol. Chem. 15(20), 4344–4350 (2017). Google Scholar

22. “Cellvizio confocal endomicroscopy system for characterising pancreatic cysts,” Guidance and guidelines NICE,  https://www.nice.org.uk/advice/mib69/chapter/Technology-overview (2018). Google Scholar

23. P. B. Garcia-Allende et al., “Towards clinically translatable NIR fluorescence molecular guidance for colonoscopy,” Biomed. Opt. Express 5, 78–92 (2014).BOEICL2156-7085 https://doi.org/10.1364/BOE.5.000078 Google Scholar

24. J. A. Udovich et al., “Spectral background and transmission characteristics of fiber optic imaging bundles,” Appl. Opt. 47, 4560–4568 (2008).APOPAI0003-6935 https://doi.org/10.1364/AO.47.004560 Google Scholar

25. A. G. Rossi et al., “Regulation of macrophage phagocytosis of apoptotic cells by cAMP,” J. Immunol. 160, 3562–3568 (1998).JOIMA30022-1767 Google Scholar

26. J. T. Alander et al., “A review of indocyanine green fluorescent imaging in surgery,” Int. J. Biomed. Imaging 2012, 1–26 (2012). https://doi.org/10.1155/2012/940585 Google Scholar

27. D. Choudhury et al., “Endoscopic sensing of alveolar pH,” Biomed. Opt. Express 8, 243–259 (2017).BOEICL2156-7085 https://doi.org/10.1364/BOE.8.000243 Google Scholar

28. A. Perperidis et al., “Characterization and modelling of inter-core coupling in coherent fiber bundles,” Opt. Express 25, 11932–11953 (2017).OPEXFF1094-4087 https://doi.org/10.1364/OE.25.011932 Google Scholar

29. F. S. Fuchs et al., “Fluorescein-aided confocal laser endomicroscopy of the lung,” Respiration 81, 32–38 (2011).RESPBD0025-7931 https://doi.org/10.1159/000320365 Google Scholar

30. R. C. Newton et al., “Imaging parenchymal lung diseases with confocal endomicroscopy,” Respir. Med. 106, 127–137 (2012).RMEDEY0954-6111 https://doi.org/10.1016/j.rmed.2011.09.009 Google Scholar

31. F. S. Fuchs et al., “Confocal laser endomicroscopy for diagnosing lung cancer in vivo,” Eur. Respir. J. 41, 1401–1408 (2013). https://doi.org/10.1183/09031936.00062512 Google Scholar

32. T. Vercauteren et al., “Processing of in vivo fibered confocal microscopy video sequences,” in Microscopic Image Analysis for Life Science Applications, , J. Rittscher, R. Machiraju and S. T. C. Wong, Eds., pp. 441–463, Artech House, Boston, Massachusetts (2008). Google Scholar

33. M. A. A. Neil, R. Juskaitis and T. Wilson, “Method of obtaining optical sectioning by using structured light in a conventional microscope,” Opt. Lett. 22, 1905–1907 (1997).OPLEDP0146-9592 https://doi.org/10.1364/OL.22.001905 Google Scholar

34. T. N. Ford, D. Lim and J. Mertz, “Fast optically sectioned fluorescence HiLo endomicroscopy,” J. Biomed. Opt. 17, 021105 (2012).JBOPFO1083-3668 https://doi.org/10.1117/1.JBO.17.2.021105 Google Scholar

35. P. A. Keahey et al., “Optimizing modulation frequency for structured illumination in a fiber-optic microendoscope to image nuclear morphometry in columnar epithelium,” Biomed. Opt. Express 6, 870–880 (2015).BOEICL2156-7085 https://doi.org/10.1364/BOE.6.000870 Google Scholar

36. J. M. Stone et al., “Low index contrast imaging fibers,” Opt. Lett. 42, 1484–1487 (2017).OPLEDP0146-9592 https://doi.org/10.1364/OL.42.001484 Google Scholar


Nikola Krstajić is a lecturer in biomedical engineering at the University of Dundee, United Kingdom. His interests include both photonics system design and component design (detectors and sources) with strong focus on microscopy and clinical translation. After industrial work in medical and scientific instrumentation, he received his PhD in physics from the University of Surrey in 2007 and has worked since in biomedical optics.

Bethany Mills completed her PhD in molecular microbiology at the University of Nottingham in 2014, where she developed methods to image bacterial infections in vivo by optical and nuclear techniques and was awarded the prestigious European Association of Nuclear Medicine Eckert & Ziegler Abstract Award in 2014. She joined Proteus in January 2015, she was awarded a Women in Molecular Imaging Scholar Award prize by the World Molecular Imaging Congress in 2016.

Ian Murray is a software developer with over 12 years experience working in medical device research and industry on a wide range of novel applications. He has expertise in image processing and algorithm development as well as front-end software, usability and a detailed understanding of the industry regulatory framework regarding medical devices and their software.

Adam Marshall is a clinical PhD student at the University of Edinburgh. He graduated from University of Edinburgh Medical School in 2007, and trained in Respiratory Medicine in Aberdeen, Glasgow and Edinburgh. His research focus is in clinical interventional studies with chemical smartprobes, and ex vivo Lung Perfusion.

Dominic Norberg is currently a PhD student at the University of Edinburgh within the Proteus EPSRC-interdisciplinary Research Group. He received his BSc and MSc degrees in electrical engineering, information technology and technical informatics from RWTH Aachen University. He has worked in research and development of OLEDs, feedback control of HiL systems, and embedded programming of upper limb prosthetic control. In his PhD project he works on improving image processing and illumination techniques for optical endomicroscopy.

Thomas H. Craven is a clinical lecturer at the University of Edinburgh. He graduated from Cambridge University in 2006 and moved to Edinburgh in 2008 to train in Intensive Care Medicine. He completed his PhD in biophotonics and inflammation in 2017. He won the prestigious Intensive Care Foundation Research Gold Medal in 2018. His research includes clinical applications for fibre based biophotonic technology, especially targeted at ventilator associated pneumonia and acute respiratory distress syndrome.

Philip Emanuel is currently a PhD student at the University of Edinburgh within the Proteus EPSRC-Interdisciplinary Research Group. He received his undergraduate degree in biomedical sciences from the University of Northumbria, Philip has over 5 years technical laboratory experience. His current PhD project focuses on developing optical fibre-based strategies to interrogate the alveolar epithelium within the lung, and the impact of cellular senescence on tissue repair in this region.

Tushar R. Choudhary is an EPSRC IRC Proteus postdoctoral research associate. He received his PhD at Heriot–Watt University and the University of Glasgow in human retinal oximetry using multispectral imaging. His research interests include optical and spectral imaging, optical sensing and fluorescence lifetime in research and clinical applications.

Gareth O. S. Williams is working at the University of Edinburgh as part of the interdisciplinary Proteus Project. He is focused on the commercialization of scientific advances and translating ideas into diverse new applications. Highly skilled in optical spectroscopy and fibre optics, specializing in the design of opto-mechanical and microfluidic systems. Experience includes incorporating these technologies in areas ranging from fluorescence imaging techniques and protein detection to high pressure fluorescence, laser crystallization, and multiphoton systems.

Emma Scholefield studied environmental biology and has 15 years’ biological research experience at the University of Edinburgh. She has spent the last six years within the Pulmonary Optical Molecular Imaging Group developing optical imaging probes to detect infection and inflammation in the distal lung. Currently, she is the quality and operations manager enabling the transition from bench to bedside to allow first-in-man studies of optical imaging agents.

Ahsan R. Akram is a Cancer Research UK Clinician Scientist working in the field of pulmonary optical molecular imaging. He received his medical degree from the University of Edinburgh in 2005 and his PhD in translational optical imaging in 2015. His current research interest is utilizing imaging modalities for understanding the dynamic biological activity of lung cancer in patients to help stratify therapies.

Andrew Davie is a clinical scientist and chartered engineer and works for NHS Lothian, United Kingdom. His role includes providing specialist advice on regulatory, scientific and safety matters relating to medical devices and technology. He is involved in research and development projects in a number of ways: a design and support engineer for the development of new medical devices, a collaborator in clinical research involving medical technology, and in the assessment of novel devices for first-in-man use.

Nik Hirani qualified from Nottingham University, clinically trained in pulmonology and acquired Wellcome Trust and GSK training fellowships. Currently, he is a PI in the MRC Centre for Inflammation Research and Clinical Director for Respiratory Medicine in Edinburgh. His research interests include macrophage biology in lung inflammation and repair, early phase trials in lung fibrosis and he leads the Edinburgh Lung Fibrosis clinic.

Annya Bruce completed her PhD in molecular neuroscience in 2012 and has since been delivering impact driven research grants focusing on translating novel technologies for the assessment and diagnosis of lung disease. She has previously set up and managed UK wide clinical studies investigating therapies for patients with severe mental health conditions and has significant experience in interpreting complex imaging clinical datasets and regulatory requirements and in the development of commercialization cases for Healthcare Technology.

Anne Moore studied mathematics at Cambridge before receiving her PhD in signal processing from the University of Edinburgh. She then worked for 16 years in industry on experimental phased array radar projects. In 2007, she returned to academia completing her PhD in molecular plant science at the University of Edinburgh. Currently, she manages the EPSRC IRC Proteus project.

Mark Bradley received his doctorate from the University of Oxford in 1989, followed by postdoctoral studies at Harvard and returning to the UK as a Royal Society University Research Fellow. In 1997, he was made a professor of combinatorial chemistry setting up the UK’s Centre for Combinatorial Chemistry. In 2005, he moved to the University of Edinburgh, where he is now director of a £15M Interdisciplinary Research Collaboration in the area of Optical Molecular Imaging (Proteus 2014-2023).

Kevin Dhaliwal is a clinician scientist. He leads the IRC Proteus Interdisciplinary hub in the MRC Centre for Inflammation Research at the University of Edinburgh. He trained in medicine in Edinburgh and completed a PhD as a MRC clinical training fellow. He was appointed as professor of Molecular Imaging and Healthcare Technology at the University of Edinburgh in 2017. His work focuses on the clinical translation of interventional approaches to diagnose, treat and monitor disease. His clinical subspecialist interests are interventional medicine, infection, and tuberculosis.

© The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
Nikola Krstajić, Nikola Krstajić, Bethany Mills, Bethany Mills, Ian Murray, Ian Murray, Adam Marshall, Adam Marshall, Dominic Norberg, Dominic Norberg, Thomas H. Craven, Thomas H. Craven, Philip Emanuel, Philip Emanuel, Tushar R. Choudhary, Tushar R. Choudhary, Gareth O. S. Williams, Gareth O. S. Williams, Emma Scholefield, Emma Scholefield, Ahsan R. Akram, Ahsan R. Akram, Andrew Davie, Andrew Davie, Nik Hirani, Nik Hirani, Annya Bruce, Annya Bruce, Anne Moore, Anne Moore, Mark Bradley, Mark Bradley, Kevin Dhaliwal, Kevin Dhaliwal, "Low-cost high sensitivity pulsed endomicroscopy to visualize tricolor optical signatures," Journal of Biomedical Optics 23(7), 076005 (10 July 2018). https://doi.org/10.1117/1.JBO.23.7.076005 . Submission: Received: 23 October 2017; Accepted: 16 May 2018
Received: 23 October 2017; Accepted: 16 May 2018; Published: 10 July 2018

Back to Top