Benchmarking of fluorescence cameras through the use of a composite phantom

Dimitris Gorpas; Maximilian Koch; Maria Anastasopoulou; Uwe Klemm; Vasilis Ntziachristos

doi:10.1117/1.JBO.22.1.016009

12 January 2017 Benchmarking of fluorescence cameras through the use of a composite phantom

Dimitris Gorpas, Maximilian Koch, Maria Anastasopoulou, Uwe Klemm, Vasilis Ntziachristos

Author Affiliations +

Journal of Biomedical Optics, Vol. 22, Issue 1, 016009 (January 2017). https://doi.org/10.1117/1.JBO.22.1.016009

Abstract

Fluorescence molecular imaging (FMI) has shown potential to detect and delineate cancer during surgery or diagnostic endoscopy. Recent progress on imaging systems has allowed sensitive detection of fluorescent agents even in video rate mode. However, lack of standardization in fluorescence imaging challenges the clinical application of FMI, since the use of different systems may lead to different results from a given study, even when using the same fluorescent agent. In this work, we investigate the use of a composite fluorescence phantom, employed as an FMI standard, to offer a comprehensive method for validation and standardization of the performance of different imaging systems. To exclude user interaction, all phantom features are automatically extracted from the acquired epi-illumination color and fluorescence images, using appropriately constructed templates. These features are then employed to characterize the performance and compare different cameras to each other. The proposed method could serve as a framework toward the calibration and benchmarking of FMI systems, to facilitate their clinical translation.

1. Introduction

An important goal of surgical oncology is the complete removal of malignant tissue. However, limitations in human vision and tactile feedback lead to insufficient intraoperative inspection of tumor margins. Typically, postsurgical assessment of excised specimen with histopathology is required for verification if a complete tumor resection was achieved. Histological analysis of biopsies and surgical specimen is nevertheless time consuming, labor intensive, and subject to sampling errors as it involves specialized personnel and subjective interpretation of readouts. Importantly, postsurgical findings of positive tumor margins may require secondary procedures. Thus, new technologies for real-time guidance are needed to address limitations that may lead to incomplete surgical treatment of disease. Indeed, statistics with breast conserving surgeries, for example, show that the incomplete resection rates range between 20% and 50%,¹^–³ entailing recurrent surgical intervention with associated risks.⁴

Fluorescence imaging has the potential to improve surgical and endoscopic guidance and positively impact the clinical management and prognosis of numerous diseases. Even though indocyanine green and other fluorescent agents have been considered for surgical guidance,⁵^–⁷ including robot assisted interventions,⁸^,⁹ their ability for tumor delineation is limited due to lack of specificity.¹⁰ Conversely, fluorescent agents that target specific biochemical and molecular features can improve demarcation of malignant tissue and have received approvals for experimental clinical use.¹⁰^–¹²

A critical issue associated with the clinical translation of fluorescence molecular imaging relates to the reproducibility of the collected measurements. In particular, images acquired from the same target using different fluorescence cameras may vary considerably when the employed systems have markedly different specifications. Methods that standardize fluorescence imaging are therefore becoming necessary for assessing the performance of fluorescence systems and agents and for providing a reference to the recorded data. Consequently, a number of phantoms have been suggested for comparing fluorescence imaging systems.¹³^–¹⁶ Polyurethane-based phantoms were considered recently for assessing the sensitivity of fluorescence cameras or for quantifying the excitation light leakage into the acquired fluorescence images.¹³^,¹⁴ Tissue-mimicking phantoms simulating optical properties and/or geometry of sample tissues have also been applied for training surgeons or for validating new hardware configurations and software methodologies.¹⁵^,¹⁷^,¹⁸ Nevertheless, most of these phantoms resolve one or a few parameters and do not allow for comprehensive characterization of all variables associated with fluorescence imaging performance.

In this work, we sought to identify a methodology that could offer seamless benchmarking of fluorescence cameras. We have recently proposed a composite FMI phantom, i.e., a phantom capable of integrating multiple targets within the field of view of a fluorescence camera, so that multiple camera characteristics can be examined with a single or a few image acquisitions.¹⁹ Using this composite phantom, we develop an approach to process the fluorescence phantom image and quantify different camera performance parameters automatically. We show, for the first time, how the use of a composite phantom can be employed for the comprehensive calibration of a camera system. Furthermore, we demonstrate how composite phantoms can be employed for comparing systems of different specifications. The described benchmarking method may become critical for standardization of imaging systems with broader applications for clinical translation of fluorescence molecular imaging.

2. Materials and Methods

2.1.

Standardization Phantom

We have previously proposed a rigid phantom for interrogating different aspects of fluorescence and optical imaging performance.¹⁹ The phantom contains a number of imaging targets and resolves different fluorescence features (Fig. 1). In particular, as shown in Fig. 1(b), each quadrant of the phantom tests different performance parameters, i.e., (1) sensitivity as a function of the optical properties (red color); (2) sensitivity as a function of the depth from the top surface (blue color); (3) resolution of the fluorescence and optical imaging (purple color); and (4) cross-talk from the excitation light (pink color). The five wells at the corners and center of the phantom (green color) have been added to assess the field illumination (i.e., illumination for enabling reflectance color imaging) homogeneity when optical measurements are performed through a color camera.

Fig. 1

The standardization phantom employed in this study. (a) The designed phantom and its dimensions. (b) The different compartments per element and/or group of elements of the phantom. The base material is transparent rigid polyurethane. In (b), arrowheads indicate that a group of elements (per row, column, or color code) have the same constituents, while the dotheads indicate the composition of a single element of the phantom. Color codes: red—sensitivity versus optical properties (three sets of optical properties, A, B, and C); blue—sensitivity versus depth (index in the $3 \times 3$ matrix defines the depth); purple—resolution; pink—cross-talk; green—field illumination homogeneity; cyan—phantom body; Di—depth from the phantom’s top surface.

The main constituent of the phantom is transparent polyurethane (WC-783 A/B, BJB Enterprises, Tustin, United States). The fluorescence compounds are organic quantum dots (Qdot^® 800 ITK^™, Q21771MP, Thermofisher Scientific Waltham, United States). Scattering is imposed by anatase ${TiO}_{2}$ nanoparticles (Titanium IV Oxide, Sigma Aldrich, St. Louis, United States); absorption is mimicked by alcohol soluble nigrosin (Sigma Aldrich) in the phantom main body and by Hemin (Sigma Aldrich, from bovine $\geq 90 %$ ) in the different wells. Selection of the specific phantom materials was based on the stability of their optical properties over time and their ability to create homogeneous mixtures.¹⁴^,¹⁹ The specific mixture content of all phantom elements is given in Fig. 1(b). The procedure of the phantom’s preparation has been explicitly described elsewhere.¹⁹

2.2.

Imaging Systems

To develop a methodology that employs composite phantoms for comparing different systems, we used two imaging systems (Fig. 2). The system in Fig. 2(a) (camera I) is a modified version of the one that has been developed, characterized, and reported by our group elsewhere.²⁰ Briefly, a 750-nm CW laser diode (BWF2-750-0, B&W Tek, Newark, Delaware, United States) with a maximum power of 300 mW is used to excite the fluorescence compounds of the phantom, while field illumination is enabled by a 250-W halogen lamp (KL-2500 LCD, Schott AG, Mainz, Germany). The laser power incident to the phantom at a working distance of 15 cm was measured at $85 mW / {cm}^{2}$ , which is lower than the maximum permissive exposure according to American National Standards Institute (ANSI) standard for eye exposure. A short-pass filter (E700SP, Chroma Technology, Rockingham, Vermont, United States) is used to remove the near infrared (NIR) component of the field illumination and thus to eliminate the cross-talk between fluorescence detection and field illumination optical paths [ $F_{1}$ in Fig. 2(a)]. Ground glass diffusers (DG10-220, Thorlabs, Newton, New Jersey, United States) are used to achieve uniform illumination of the field of view from both light sources [D in Fig. 2(a)]. The optical signal is collected by a motorized zoom/focus lens (CVO GAZ11569M, Goyo Optical Inc., Asaka, Saitma, Japan) and spectrally resolved in two channels by a dichroic mirror (700DCXXR, AHF analysentechnik AG, Tubingen, Germany) [DM in Fig. 2(a)]. The first channel, which is within the spectral range from 720 to 850 nm, is relayed through a NIR achromatic doublet pair (MAP10100100-B, Thorlabs) [ ${RL}_{1}$ in Fig. 2(a)], filtered by a NIR emission filter (ET810/90, Chroma Technology) [ $F_{2}$ in Fig. 2(a)] and recorded by an iXon electron multiplying charge-coupled device (EMCCD, DV897DCS-BV, Andor Technology, Belfast, Northern Ireland). The second channel, which is within the spectral range 450 to 700 nm, is relayed through a visible achromatic doublet pair (MAP10100100-A, Thorlabs) [ ${RL}_{2}$ in Fig. 2(a)], filtered by a short-pass filter (ET700SP-2P, Chroma Technology) [ $F_{3}$ in Fig. 2(a)] and recorded by a 12-bit color charge-coupled device (CCD) camera (pixelfly qe, PCO AG, Kelheim, Germany).

Fig. 2

Schematics of the two imaging systems employed in this study. (a) The fluorescence/color camera (camera I) and (b) the fluorescence camera (camera II) were validated and benchmarked through imaging the composite phantom. D: diffuser; F: filter; DM: dichroic mirror; RL: relay lens.

Camera I can operate under two configurations: (1) with both cameras (EMCCD and CCD) enabled for simultaneous acquisition of fluorescence and color images (camera I-FC) or (2) using only the EMCCD camera, thus capturing only fluorescence images (camera I-F).

The second system [camera II; Fig. 2(b)] is also based on EMCCD detection (Luca R, Andor Technology). Camera II has four major differences compared to camera I: (1) it lacks the color imaging channel (450- to 700-nm spectral band), (2) it has different operational characteristics (see Table 1), (3) it uses a different fluorescence filter (D850/40 m, Chroma Technology) [ $F_{4}$ in Fig. 2(b)], and (4) it employs a different lens (Zoom 7000 Macro Lens, Navitar, New York, United States). The differences between the two fluorescence imaging systems are summarized in Table 1.

Table 1

The main differences between camera I and camera II.

	Camera I	Camera II
Resolution (pixels)	$512 \times 512$	$1004 \times 1002$
Pixel size ( $μ m$ )	$16 \times 16$	$8 \times 8$
Quantum efficiency (at 800 nm)	$\sim 70 %$	$\sim 40 %$
Dynamic range (bit)	16	14
Detection wavelength (nm)	810/90	850/40
Color reflectance imaging	Yes (450 to 700 nm)	N/A

Data acquisition and control of cameras were enabled via the Solis software (Solis I, Andor Technology) and a GPU-based C++ software developed by our group.²⁰ All data processing was implemented in MATLAB (Mathworks Inc., Massachusetts, United States).

2.3.

Data Acquisition Protocols

We explored the use of the composite phantom in association with two acquisition parameters: (1) the level of pixel-binning of the camera sensor and (2) the working distance, i.e., the distance from the camera lens to the surface of the target object. For all experiments performed, fluorescence was enabled by the same excitation source, that is, the 750-nm laser diode (Fig. 2), and images were acquired with room lights turned off. The integration time was set at 0.1 s, to resemble real-time measurements as they are performed in vivo. To ensure minimization of boundary effects, the phantom was placed on top of a highly absorbing material.

To examine the effects of pixel-binning, both cameras were positioned at the same 320-mm working distance from the phantom surface. This distance is a representative working distance for a wide range of intraoperative applications. Fluorescence images were then acquired with different gains, cooling temperatures, and pixel binning settings, as listed in Table 2.

Table 2

The acquisition settings for all the investigated cases.

Experimental configuration	Working distance (mm)	Binning	Gain	Temperature (°C)
Camera I-F Camera I-FC	320	$1 \times$	4000	$- 70$
Camera II-a	320	$1 \times$	200	$- 20$
Camera $II - 2 \times a$	320	$2 \times$	200	$- 20$
Camera $II -4 \times a$	320	$4 \times$	100	$- 20$
Camera II-b	200	$1 \times$	200	$- 20$
Camera $II - 2 \times b$	200	$2 \times$	200	$- 20$
Camera $II - 4 \times b$	200	$4 \times$	60	$- 20$

The influence of the working distance was also examined by operating camera I at 320 mm distance from the target while changing the working distance of camera II to 200 mm. Fluorescence images were acquired from camera II without pixel-binning (camera II-b), with $2 \times$ (camera $II - 2 \times b$ ) and with $4 \times$ (camera $II - 4 \times b$ ) pixel-binning. The gain of camera $II - 4 \times b$ was reduced compared to the other configurations of camera II in order to avoid saturation. Table 2 summarizes all acquisition settings for both tests implemented.

For every fluorescence image acquired, a corresponding dark image, i.e., image with the excitation light disabled and maintaining constant all other acquisition settings, was also captured. This image was used for estimating the effects of ambient illumination, that is stray light emitted from sources different from the ones of the two systems (i.e., computer monitors, optical mice, indication light emitting diodes on electronic devices, to name a few). Furthermore, subtraction between the fluorescence and dark images compensated for possible parasitic illumination (i.e., ambient illumination at the detection wavelengths) or dark current influence on the validation process.

In addition, reflectance images were acquired for every experimental configuration, as shown in Table 2. These images were used to estimate the optical parameters of each camera (i.e., magnification and optical resolution, see Sec. 2.4 below). The reflectance images, for all configurations of camera I-C and camera II, were acquired with room lights turned on. On the other hand, for the configuration of camera I-FC, the reflectance images were acquired by the CCD camera with field illumination enabled and room lights turned off.

2.4.

Camera Performance Assessment

To calibrate and compare different imaging systems with minimal user intervention, we developed an automated method for the detection of all the composite phantom elements. This method was based on the application of the speeded-up robust features (SURF) algorithm²¹ applied on the acquired images and specially designed templates. The distance between the two sets of features was then computed and thresholded based on an efficient approximate nearest neighbor search.²² The outcome of this process was two sets of image points: one set corresponding to the template and one to the acquired image. These points were then used to estimate the geometric transformation between the two images. Two phantom templates were designed: (1) one that was used for the fluorescence images [Fig. 3(a)] and (2) one that was used for the reflectance images [Fig. 3(b)].

Fig. 3

The templates used for the automated detection of all phantom elements. (a) The template for the fluorescence images includes only the elements of the phantom containing the QDots. (b) The template for the reflectance images, on the other hand, includes all elements of the phantom. (c) The diffused fluorescence resolution is defined as the smallest line perpendicular to the bisector of the L-shaped element (purple line) that can resolve the two edges of the concave vertex (blue dot). The scanning direction of the purple line is indicated by the white arrow and following the template matching can be applied to the fluorescence images.

Employing the geometric transformation derived from the abovementioned process, predefined points of interest were projected from the templates onto the acquired images. These points include (1) the four corners of the phantom, (2) the center and one perimeter point of all the circular phantom elements, (3) the six corners of the L-shaped phantom element, and (4) the four corners of the USAF-1951 target, as well as the four corners of all the target’s line elements. These points are adequate to extract all phantom components and consequently to quantify camera performance metrics, i.e., magnification, optical resolution, diffused fluorescence resolution, excitation light leakage and parasitic illumination, sensitivity and field illumination homogeneity as described in the following:

2.4.1.

Magnification

By detecting the boundaries of the phantom in the acquired images, the magnification, $M$ , is approximated by the following formula:

Eq. (1)

M = \frac{D_{phantom}}{\sqrt{D_{pw}^{2} \cdot {(u_{1} - u_{2})}^{2} + D_{ph}^{2} \cdot {(v_{1} - v_{2})}^{2}}},

where

D_{phantom} = 100 mm

is the phantom’s width, and the denominator represents the Euclidean distance in Cartesian coordinates between two adjacent corners of the phantom on the imaging plane. In Eq. (1),

(u_{1}, v_{1})

and

(u_{2}, v_{2})

are the pixel coordinates of the two phantom corners,

D_{pw}

is the pixel width, and

D_{ph}

is the pixel height. This metric is then expressed as the well-known magnification ratio

1 : M

. The magnification can be approximated either from the acquired fluorescence images or from the reflectance ones.

2.4.2.

Optical resolution

The optical resolution is approximated through the USAF-1951 target, which consists of a series of elements (having two sets of three lines separated by spaces equal to one line’s width) at right angles, organized in groups. This metric is quantified as cycles per mm ( $c / mm$ ) through the general formula

Eq. (2)

F = 2^{group + (element - 1) / 6},

which provides the frequency,

F

, of any target element in any target group. In addition, the line or space width of each element can be estimated from the frequency as

L_{O} = {(2 \cdot F)}^{- 1}

in mm.

For each one of the bounding boxes of the USAF-1951 target elements, as defined by the four corners of the corresponding line, the contrast transfer function (CTF) is quantified by the Michelson’s formula²³

Eq. (3)

{CTF}_{i} = \frac{\max (I_{i}) - \min (I_{i})}{\max (I_{i}) + \min (I_{i})},

where

I_{i}

denotes the intensity values of the pixels inside the

i

’th bounding box. Adopting the Rayleigh criterion for optical imaging, the limit where CTF is

\sim 26.4 %

²⁰ defines the USAF-1951 target’s elements that can be fully resolved by the system. The frequency of these elements is then estimated by Eq. (2), and thus, the resolution of the system for the specific working distance and field of view is quantified.

Optical resolution can be validated only on reflectance images, acquired either with room lights turned on (i.e., for all camera II configurations and camera I-F) or under field illumination (i.e., for camera I-FC configuration), as described in Sec. 2.3.

2.4.3.

Diffused fluorescence resolution

The diffused fluorescence resolution is quantified by the L-shaped element of the phantom. The determination of diffused resolution of a fluorescence imaging system is achieved for a specific set of optical properties of the fluorescent target and the surrounding medium and changes if the optical properties of either one change. Thus, the purpose of the L-shaped element in the phantom is the quantification of resolution improvements that may be offered by a system and/or method.

As in the optical resolution, the CTF is employed for the approximation of the diffused fluorescence resolution. Specifically, the CTF is calculated over every line segment that is perpendicular to the bisector of the L-shaped element [Fig. 3(c)]. In order to exclude any possible bias by the USAF-1951 target, all lines crossing it are not considered. In addition, to remove high frequency noise from the CTF measurements, the CTF distribution across the lines is fitted to a second-order polynomial. Limiting the CTF to $\sim 26.4 %$ , the length of the last line, $L_{F}$ , over this threshold is transformed from image to scene (phantom) coordinates, using the pixel dimensions, $D_{pw} \times D_{ph}$ , and the magnification, $M$ :

Eq. (4)

L_{F} = M \cdot \sqrt{D_{pw}^{2} \cdot {(u_{1} - u_{2})}^{2} + D_{ph}^{2} \cdot {(v_{1} - v_{2})}^{2}},

where

(u_{1}, v_{1})

and

(u_{2}, v_{2})

are the two edges of the line segment. This distance is reported as the diffused fluorescence resolution of the investigated camera.

2.4.4.

Excitation light leakage and parasitic illumination

The presence of excitation light leakage through the filters is assessed by the highly scattering and the highly absorbing wells in the upper left quadrant of the phantom [pink color in Fig. 1(b)]. Specifically, the transmission ratio, $R_{exc} = S / N$ , is estimated by assuming the average pixel counts (i.e., digitized pixel intensity) from the highly scattering element as the signal component, $S$ , and the average pixel counts from the highly absorbing element as noise, $N$ . If this ratio is equal to or smaller than 1, it implies reduced excitation light leakage, whereas higher values indicate the presence of excitation light in the acquired data.

Parasitic illumination is assessed following the exact same process as for light leakage, but on the dark images. With the excitation source disabled, the highly scattering and absorbing elements of the phantom can only be visualized under the presence of parasitic illumination. The transmission ratio, $R_{par}$ , will provide a quantified indicator on the influence and significance of parasitic illumination. This quantity may be related to filter parameters and environmental conditions, such as ambient light, and can be further employed to optimize the system or the lighting conditions.

2.4.5.

Sensitivity versus optical properties and depth

The sensitivity of a fluorescence camera for various optical properties (i.e., different absorption and scattering) or depth has been validated based on readouts from the 18 corresponding phantom wells [red and blue colors in Fig. 1(b)].

The signal-to-noise ratio (SNR) for each of these wells is approximated through the formula

Eq. (5)

{SNR}_{dB} = 20 \cdot \log (\frac{S}{RMSN}),

where

S

is the average intensity within each well of the raw fluorescence image and

RMSN

corresponds to the root mean square noise approximated from the phantoms’ main body (i.e., excluding all elements). In this way, the SNR is normalized for variations that are introduced by the base constituents of the phantom and excitation light leakage. Moreover, considering normal signal distributions, a measurement is assumed to present 95% confidence if the signal is twofold the noise level,²⁰ which in the dB scale corresponds to 6 dB.

The capability of the system to image the fluorescence wells is further assessed through the contrast as expressed by the Weber’s fraction, $C$ :²⁴

Eq. (6)

C = \frac{S - N}{N},

where

N

is the average pixel counts from the highly scattering element of the phantom [right pink color in Fig. 1(b)]. The higher this metric for a fluorescence element of the phantom, the better a fluorescence camera can discriminate it from light leakage. Similar to SNR, 95% confidence is achieved when the signal is twofold the noise level, which corresponds to

C = 1

. As expected from the definitions of SNR and contrast, these quantities are figures of merit for benchmarking different systems as a function of the specific phantom, rather than absolute values to characterize a single system.

2.4.6.

Field illumination homogeneity

The homogeneity of the field illumination can be approximated using the five highly scattering elements located at the four corners and the center of the phantom [green color in Fig. 1(b)]. A normalized average intensity equal to 1 for all five elements indicates a homogenous illumination profile, whereas the higher the deviation among the five values, the higher is the illumination heterogeneity. We note that while this phantom offers a crude spatial sampling of the illumination field, integration of more reflecting elements could be considered for more accurate determination of field illumination.

Flat-fielding can be achieved by applying bicubic splines interpolation of the average intensity from each of the five elements and dividing the acquired reflectance images with the resulting profile.¹⁹

2.5.

Image Registration

In addition to the performance assessment of fluorescence cameras, the phantom enables interrogation of the registration between fluorescence and color images, as it applies to fluorescence/color imaging systems, such as camera I. The transformation from the image coordinates of the color camera, $(u_{c}, v_{c})$ , to the coordinates of the fluorescence camera, $(u_{f}, v_{f})$ , is performed through the linear system

Eq. (7)

[\begin{matrix} u_{c} \\ v_{c} \\ 1 \end{matrix}] = A \cdot [\begin{matrix} u_{f} \\ v_{f} \\ 1 \end{matrix}] .

The $3 \times 3$ affine transformation $A$ can be estimated by Eq. (7) after solving the correspondence problem, i.e., the extraction of points of interest visible by both imaging modalities. Nevertheless, the fiduciary markers of the phantom (see Sec. 2.4) can serve as points of interest and thus can be used to solve the correspondence problem and through that approximate the affine transformation $A$ .

3. Results

Figure 4 shows the fluorescence and reflectance image pairs as acquired by three experimental configurations of the two cameras employed in this study. Specifically, Fig. 4(a) shows the fluorescence image acquired by configuration camera I-F and Fig. 4(b) depicts the corresponding reflectance image acquired with the room lights turned on. The equivalent image pair acquired by configuration camera II-a is shown in Figs. 4(c) and 4(d). Finally, for the experimental configuration camera I-FC, the fluorescence image of the phantom is shown in Fig. 4(e) and the reflectance image acquired by the color camera with enabled field illumination and the room lights turned off is shown in Fig. 4(f). The different sensor size and pixel resolution between the EMCCD and CCD of camera I lead to the different aspect ratios between panels (e) and (f). The fluorescence images shown in Fig. 4 are employed for the determination of the fluorescence performance parameters of the two cameras for the various experimental configurations (i.e., fluorescence resolution, excitation light leakage, parasitic illumination, and sensitivity), while the reflectance images are used for the determination of the optical parameters (i.e., optical resolution, magnification).

Fig. 4

The fluorescence and reflectance image pairs acquired by three experimental configurations of the two cameras employed in this study, with the phantom’s elements highlighted by color-coded edges. (a) The fluorescence image acquired by camera I-F and (b) the corresponding reflectance image acquired with room lights turned on. (c) The fluorescence image acquired by camera II-a and (d) the corresponding reflectance image acquired with room lights turned on. (e) The fluorescence image acquired by camera I-FC and (f) the reflectance image simultaneously acquired by the color camera of camera I with the field illumination enabled and the room lights turned off. The color code of the phantom elements correspond to the one of Fig. 1(b).

The boundaries of the phantom elements shown in all panels of Fig. 4 were determined by the projection of the points of interest describing each element from the templates to the phantom images via the application of the SURF algorithm (see Sec. 2.4). These elements are employed for the automated quantification of the camera parameters without any user intervention.

3.1.

Assessment of Fluorescence Imaging Sensitivity

Figures 5 and 6 demonstrate the fluorescence imaging performance of the first and third configurations in Table 2. Figure 5 depicts the fluorescence measurements from configuration camera I-F. Specifically, Fig. 5(a) shows the pixel counts of the nine wells with different optical properties and Figs. 5(b) and 5(c) depict the corresponding SNRs and contrast ( $C$ ) achieved as estimated by Eqs. (5) and (6), respectively. The visualization scheme adopted for the SNR and contrast is based on the corresponding thresholds (i.e., 6 dB for SNR and 1 for $C$ ). This provides a rapid visual assessment whether these thresholds are exceeded in any of the wells. Furthermore, the height of the saturated area, that is the red part of the cylinder, provides information regarding the order of magnitude that the achieved metric is higher than the corresponding threshold. The same metrics as derived from the nine wells with different depths from the phantom’s top surface are shown in Figs. 5(d)–5(f). Figure 6 shows the corresponding measurements for configuration camera II-a in Table 2, and in the same order as in Fig. 5.

Fig. 5

The fluorescence measurements from experimental configuration of camera I-F. (a) The pixel counts of the nine wells with different optical properties. (b) The corresponding SNR and (c) contrast achieved. (d) The pixel counts of the nine wells with different depth from the phantom’s top surface. (e) The corresponding SNR and (c) contrast achieved. SNR and contrast were quantified through Eqs. (5) and (6), respectively. Colorbars correspond to the $z$ -axis of each panel and for SNR and contrast metrics define the threshold levels.

Fig. 6

The fluorescence measurements from experimental configuration of camera II-a. (a) The pixel counts of the nine wells with different optical properties. (b) The corresponding SNR and (c) contrast achieved. (d) The pixel counts of the nine wells with different depth from the phantom’s top surface. (e) The corresponding SNR and (c) contrast achieved. SNR and contrast were quantified through Eqs. (5) and (6), respectively. Colorbars correspond to the $z$ -axis of each panel and for SNR and contrast metrics define the threshold levels.

Two systems can be considered of similar sensitivity if (1) they succeed to pass the SNR and $C$ thresholds in the same number of wells and (2) the quantified SNR and $C$ values are of equivalent distances from the corresponding thresholds. For the two systems considered in this study, it is evident from Figs. 5 and 6 that camera I-F outperforms camera II-a in all sensitivity metrics: (1) the pixel counts of camera I-F exceed 10-fold those from camera II-a; (2) camera I-F passes the 6 dB SNR threshold at eight out of the nine wells with different optical properties and at all depths, while camera II-a passes the SNR threshold at six of the wells with different optical properties and at seven wells of different depth; (iii) although $C$ for both camera configurations is above the threshold for all wells, camera I-F presents twice as stronger contrast than camera II-a.

Figure 7 depicts the summary of quantitative comparisons between camera I-F and all camera II experimental configurations (see Table 2). Figure 7(a) depicts the pixel counts from the nine wells with the different optical properties, while Figs. 7(b) and 7(c) show the corresponding achieved SNRs and $C$ . Similarly, panels (d)–(f) show the pixel counts, SNRs, and $C$ from the nine wells with the different depths from the phantom’s top surface.

Fig. 7

Quantitative comparisons between camera I-F and all configurations of camera II. (a) Pixel counts, (b) SNR, and (c) contrast from the nine wells with different optical properties. (d) Pixel counts, (e) SNR, and (f) contrast from the nine wells with different depths from the phantom’s top surface. In all panels, $x$ -axis labeling corresponds to the labeling of phantom elements shown in Fig. 1(b).

Panels (a), (b), (d), and (e) of Fig. 7 show that binning or reduction of the working distance for camera II improves both pixel counts and SNRs for all wells. For example, in Fig. 7(a), there is more than 10-fold increase in the pixel counts between camera II-a and camera $II - 4 \times b$ for the well containing $20 μ g / g$ Hemin, $0.66 mg / g$ ${TiO}_{2}$ , and 10 nM QDots [the right well of the B group in Fig. 1(b)], which is translated to almost 50% SNR increase as seen in Fig. 7(b). In the last two columns of Table 3, the achieved sensitivity of all cameras is given by means of minimum (i.e., the minimum value above the 6 dB threshold) and maximum SNR for the nine wells with different optical properties, ${SNR}_{OP, \min}$ and ${SNR}_{OP, \max}$ correspondingly, and the ones with different depth from the top surface of the phantom, ${SNR}_{D, \min}$ and ${SNR}_{D, \max}$ , correspondingly. However, the observable improvement in sensitivity comes with contrast reduction, as seen in panels (c) and (f) of Fig. 7. This is due to the increase of the excitation light leakage and the parasitic illumination (Table 3). Although camera II-a presents inferior performance compared to camera I-F, the measurements shown in Fig. 7 reveal that modification of its acquisition parameters (i.e., working distance and/or binning) can eventually lead to comparability to camera I-F performance.

Table 3

Quantification of various validation metrics from both cameras under all experimental configurations.

Camera	M	LR (mm)a	LF (mm)	Rexc	Rpar	SNROP,minSNRD,min (dB)b	SNROP,maxSNRD,max (dB)b
Camera I-F	1:13.3	$0.13 \times 0.10$	1.3	1.9	1.3	${6.5}_{C 1}$ ${9.4}_{D 9}$	${18.0}_{B 10}$ ${17.0}_{D 1}$
Camera I-FC	1:15.9c 1:17.5d	$0.04 \times 0.04$ d	1.9	1.8	1.9	${6.5}_{B 1}$ ${6.8}_{D 6}$	${14.0}_{B 10}$ ${13.4}_{D 1}$
Camera II-a	1:14.2	$0.05 \times 0.05$	1.4	1.3	1.1	${7.4}_{A 10}$ ${6.2}_{D 6}$	${10.9}_{B 10}$ ${11.4}_{D 1}$
Camera $II - 2 \times a$	1:14.2	$0.11 \times 0.10$	3.0	1.4	1.3	${6.5}_{C 1}$ ${7.1}_{D 9}$	${14.6}_{B 10}$ ${15.4}_{D 1}$
Camera $II - 4 \times a$	1:14.2	$0.25 \times 0.20$	3.7	1.5	1.6	${9.0}_{C 1}$ ${8.8}_{D 9}$	${16.2}_{B 10}$ ${17.0}_{D 1}$
Camera II-b	1:13.8	$0.06 \times 0.04$	2.2	1.7	1.1	${7.4}_{A 10}$ ${8.2}_{D 6}$	${12.9}_{B 10}$ ${14.2}_{D 1}$
Camera $II - 2 \times b$	1:13.6	$0.14 \times 0.10$	3.6	1.8	1.2	${6.4}_{B 1}$ ${7.3}_{D 8}$	${15.0}_{B 10}$ ${16.2}_{D 1}$
Camera $II - 4 \times b$	1:13.6	$0.25 \times 0.28$	3.7	1.9	1.5	${8.0}_{C 1}$ ${6.0}_{D 9}$	${15.0}_{B 10}$ ${17.0}_{D 1}$

^aHorizontal × vertical resolution.

^bIndices represent the wells labeling according to Fig. 1(b).

^cFluorescence.

^dColor.

3.2.

Assessment of Optical/Fluorescence Parameters

Table 3 tabulates the magnification ( $M$ ), the optical ( $L_{R}$ ) and diffused ( $L_{F}$ ) resolution, and the excitation light leakage ( $R_{exc}$ ) from both cameras and all experimental configurations as quantified by the images of Fig. 4. Parasitic illumination ( $R_{par}$ ) quantified by the transmission ratio from the corresponding dark images is also included. The magnification does not deviate much from one measurement to another since the phantom was covering most of the field of view for all configurations.

The optical resolution depends on the pixel size of each camera and the magnification (i.e., $512 \times 512 pixels$ for the EMCCD of camera I and $1002 \times 1004 pixels$ for camera II at $16 \times 16 μ m$ and $8 \times 8 μ m$ , respectively). Camera I-F, having the largest pixel dimensions, presents lower resolution compared to camera II-a and camera II-b but has similar resolution with camera $II - 2 \times a$ and camera $II - 2 \times b$ . The diffused fluorescence resolution depends on the specific optical properties of the fluorescence target and is introduced to interrogate devices that may account for photon diffusion, a function that is not enabled in the experimental study herein.

A limited amount of excitation light leakage in both cameras was observed, as shown in Table 3. At the same working distance, camera I-F displays higher light leakage than camera II-a, possibly due to differences of the filters employed. Nevertheless, the light leakage becomes comparable between the two systems when the working distance of camera II is reduced to 200 mm. The latter is expected and is due to the reduced distance between the imaged surface, the camera lens, and the excitation source. When comparing camera I-F and camera II-a, it becomes apparent that the level of parasitic illumination is slightly higher in camera I-F, also possibly due to the different band-pass filters employed in the two cameras. Under binning, however, camera II presents equal or even higher (i.e., binning $4 \times$ ) parasitic light contamination than camera I-F. Finally, the sensitivity measurements shown in the last two columns of Table 3 summarize the findings discussed in the previous Sec. 2.1 when comparing camera I with camera II. Although magnification is substantially different from all other camera configurations, camera I-FC shows higher sensitivity to many camera II acquisition settings.

To identify which of the acquisition settings of camera II investigated in this study brings the performance of that camera closer to the performance of camera I, we adopted a least squares method between all metrics quantified through the phantom. This analysis identified camera II-2×b as the one that approaches better the performance of camera I.

3.3.

Correction of Field Illumination Homogeneity

Figure 8(a) depicts the normalized intensity inside the five highly scattering wells [green color in Fig. 1(b)] imaged by the color camera of the camera I-FC. These values are used to approximate the field illumination profile onto the phantom surface. For the specific measurement, as seen in Fig. 8(b), this profile is strongly inhomogeneous. This observation is further confirmed by Fig. 8(c), where the phantom image is affected by vignetting. The applied flat-fielding, however, corrected for this inhomogeneity, while preserving all the color information in the acquired image, as it can be seen in Fig. 8(d).

Fig. 8

Field illumination correction through reflectance measurements. (a) The normalized intensity of the five highly scattering wells of the phantom. (b) The field illumination profile over the entire field of view. (c) An acquired image from the color camera of camera I affected by vignetting. (d) Application of flat-fielding to correct for inhomogeneous field illumination.

3.4.

Registration of Color and Fluorescence Images

Figure 9 shows two versions of the same color image, after flat-fielding, augmented with the corresponding fluorescence image. In panel (a), the fluorescence image has been transformed to the coordinates system of the color image, while panel (b) depicts the inverse transformation (i.e., the color image transformed to the coordinates system of the fluorescence image). These transformations do not require increased computational cost, as all the fiducial markers, required for the estimation of the affine transformation between the two imaging planes, are available through the SURF algorithm. The derived transformation matrices can be employed to provide highly accurate coregistration between the color and fluorescence images.

Fig. 9

A color image of the phantom augmented with fluorescence data as acquired by camera I-FC. (a) The augmented image in its original coordinates system. (b) The augmented image transformed to the coordinates system of the fluorescence image.

4. Discussion

The absence of robust methods for benchmarking fluorescence molecular imaging platforms and standardizing fluorescence medical imaging may impose hurdles on its diagnostic and clinical translation. The study herein introduced a framework for the use of composite fluorescence phantoms for the (a) automatic characterization of the performance of different imaging cameras and (b) calibration of two imaging platforms. The procedure can also employ the calibration parameters extracted to configure different systems so that they offer comparable performance, which may be of relevance to multicenter clinical trials or for producing measurements using different systems. Equilibrating images could be also performed in retrospect by image processing using the calibration parameters extracted by the different systems.

To benchmark two fluorescence imaging systems of markedly different specifications, we employed a composite phantom, designed to exhibit a variety of optical and fluorescence characteristics.¹⁹ Fluorescence images from this phantom were acquired using two cameras and different operational modes. The methodology developed provides the means for nonbiased validation of systems by automatically extracting imaging features and thus excluding human interference, using previously developed composite phantoms.¹⁹ Compared to simple phantoms,¹³^–¹⁶ composite phantoms can provide a comprehensive characterization of fluorescence imaging system performance using a single or a few images, possibly even at a single snapshot.

As seen in Fig. 4, there is always a relatively large number of phantom elements visible in the acquired images, making the application of the SURF-based registration algorithm quite robust and accurate. Nevertheless, other registration algorithms can also be applied with potentially equivalent results, as the novelty of the proposed approach is the use of templates, which even for the case of the fluorescence images, where not all wells are visible, ensures successful extraction of all phantom elements.

The results presented in this study allowed the comparison of the two exemplary cameras employed herein for demonstration purposes and clearly indicated a superiority of camera I compared to camera II when operating at equivalent settings (i.e., working distance, pixel-binning, or gain, to name a few). However, modification of one or more of these acquisition parameters impacts imaging performance. Indeed, our results clearly demonstrate that appropriate adjustment of camera II acquisition parameters can modify the performance and optimize certain performance features to resemble camera I performance.

Automatic extraction of camera specifications using composite phantoms can be employed in basic research and clinical studies to validate a camera’s performance prior to surgery or endoscopy (e.g., under miniaturization of the phantom design) and to confirm selected configurations and proper operation. Automatic feature extraction visualized in composite phantoms can therefore serve a trouble-shooting role over time and before each experimental process.²⁰ Another possible use is in quality control during a camera manufacturing or selection process. Finally, an intended application for the methodology developed is the generation of “equivalency” of data acquired during multicenter clinical trials, where the camera systems employed can be calibrated on the same composite phantoms for referencing purposes in association with the data collected.

Future work includes the development of composite phantoms that can capture a larger number of camera parameters, such as characterizing the dynamic range and spectral response, and to offer a more accurate correction for inhomogeneous illumination. The base material of the phantom described herein presents a relatively high absorbance. This minimizes any possible cross-talk between neighboring wells and photon diffusion is relatively limited. Nevertheless, such implementation is not ideal for validating the performance of systems designed to encounter for photon diffusion. In order to enable such a feature, we plan to develop phantoms that will incorporate a number of wells within highly absorbing base material and thus, cross-talks between neighboring elements will be eliminated, whereas other wells will be within highly scattering base material, and thus allow for validation of the aforementioned systems. Functions for the automatic extraction of these additional features can then be developed to streamline the detection and analysis of a larger set of calibration parameters. Although there exists evidence in literature regarding the stability of the employed materials,¹⁴^,¹⁹ we further plan to perform systematic constancy and mechanical integrity tests to validate the stability of the optical properties of the various phantom elements at different environments.

The proposed herein methodology represents an early attempt of standardizing imaging measurements or systems for fluorescence molecular imaging. Overall, we expect that the field of standardization will play a major role in the growth of fluorescence molecular imaging.

Disclosures

All authors declare no conflict of interest.

Acknowledgments

The authors would like to thank Dr. Pilar Beatriz Garcia-Allende for her contribution during the preparation of the phantom and Dr. Ovsepyan Saak for his insightful comments during the preparation of this manuscript. The research leading to these results has received funding by the Deutsche Forschungsgemeinschaft (DFG), Sonderforschungsbereich-824 (SFB-824), subproject A1 and from the European Union’s Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No. 644373 (PRISAR).

References

1.

L. E. Mc Cahill et al., “Variability in reexcision following breast conservation surgery,” J. Am. Med. Assoc., 307 (5), 467 –475 (2012). http://dx.doi.org/10.1001/jama.2012.43 Google Scholar

2.

J. F. Waljee et al., “Predictors of re-excision among women undergoing breast-conserving surgery for cancer,” Ann. Surg. Oncol., 15 (5), 1297 –1303 (2008). http://dx.doi.org/10.1245/s10434-007-9777-x Google Scholar

3.

C. Cellini et al., “Factors associated with residual breast cancer after re-excision for close or positive margins,” Ann. Surg. Oncol., 11 (10), 915 –920 (2004). http://dx.doi.org/10.1245/ASO.2004.12.037 Google Scholar

4.

C. Chiappa et al., “Surgical margins in breast conservation,” Int. J. Surg., 11 (Suppl. 1), S69 –S72 (2013). http://dx.doi.org/10.1016/S1743-9191(13)60021-7 Google Scholar

5.

L. Boni et al., “Clinical applications of indocyanine green (ICG) enhanced fluorescence in laparoscopic surgery,” Surg. Endosc., 29 (7), 2046 –2055 (2015). http://dx.doi.org/10.1007/s00464-014-3895-x Google Scholar

6.

M. Plante et al., “Sentinel node mapping with indocyanine green and endoscopic near-infrared fluorescence imaging in endometrial cancer. A pilot study and review of the literature,” Gynecol. Oncol., 137 (3), 443 –447 (2015). http://dx.doi.org/10.1016/j.ygyno.2015.03.004 GYNOA3 Google Scholar

7.

T. Yamamichi et al., “Clinical application of indocyanine green (ICG) fluorescent imaging of hepatoblastoma,” J. Pediatr. Surg., 50 (5), 833 –836 (2015). http://dx.doi.org/10.1016/j.jpedsurg.2015.01.014 JPDSA3 0022-3468 Google Scholar

8.

M. A. Bjurlin, T. R. McClintock and M. D. Stifelman, “Near-infrared fluorescence imaging with intraoperative administration of indocyanine green for robotic partial nephrectomy,” Curr. Urol. Rep., 16 (4), 1 –7 (2015). http://dx.doi.org/10.1007/s11934-015-0495-9 Google Scholar

9.

D. Daskalaki et al., “Fluorescence in robotic surgery,” J. Surg. Oncol., 112 (3), 250 –256 (2015). http://dx.doi.org/10.1002/jso.v112.3 JSONAU 0022-4790 Google Scholar

10.

M. V. Marshall et al., “Near-infrared fluorescence imaging in humans with indocyanine green: a review and update,” Open Surg. Oncol. J., 2 (2), 12 –25 (2010). http://dx.doi.org/10.2174/1876504101002020012 Google Scholar

11.

G. M. van Dam et al., “Intraoperative tumor-specific fluorescence imaging in ovarian cancer by folate receptor-[alpha] targeting: first in-human results,” Nat. Med., 17 (10), 1315 –1319 (2011). http://dx.doi.org/10.1038/nm.2472 1078-8956 Google Scholar

12.

M. J. Whitley et al., “A mouse-human phase 1 co-clinical trial of a protease-activated fluorescent probe for imaging cancer,” Sci. Transl. Med., 8 (320), 320ra324 (2016). http://dx.doi.org/10.1126/scitranslmed.aad0293 STMCBQ 1946-6234 Google Scholar

13.

B. Zhu, J. C. Rasmussen and E. M. Sevick-Muraca, “A matter of collection and detection for intraoperative and noninvasive near-infrared fluorescence molecular imaging: to see or not to see?,” Med. Phys., 41 (2), 022105 (2014). http://dx.doi.org/10.1118/1.4862514 MPHYA6 0094-2405 Google Scholar

14.

B. Zhu et al., “Validating the sensitivity and performance of near-infrared fluorescence imaging and tomography devices using a novel solid phantom and measurement approach,” Technol. Cancer Res. Treat., 11 (1), 95 –104 (2012). http://dx.doi.org/10.7785/tcrt.2012.500238 Google Scholar

15.

B. Leh et al., “Optical phantoms with variable properties and geometries for diffuse and fluorescence optical spectroscopy,” J. Biomed. Opt., 17 (10), 108001 (2012). http://dx.doi.org/10.1117/1.JBO.17.10.108001 JBOPFO 1083-3668 Google Scholar

16.

B. W. Pogue and M. S. Patterson, “Review of tissue simulating phantoms for optical spectroscopy, imaging and dosimetry,” J. Biomed. Opt., 11 (4), 041102 (2006). http://dx.doi.org/10.1117/1.2335429 JBOPFO 1083-3668 Google Scholar

17.

Ł. Szyc et al., “Development of a handheld fluorescence imaging camera for intraoperative sentinel lymph node mapping,” J. Biomed. Opt., 20 (5), 051025 (2015). http://dx.doi.org/10.1117/1.JBO.20.5.051025 JBOPFO 1083-3668 Google Scholar

18.

R. G. Pleijhuis et al., “Near-infrared fluorescence (NIRF) imaging in breast-conserving surgery: assessing intraoperative techniques in tissue-simulating breast phantoms,” Eur. J. Surg. Oncol., 37 (1), 32 –39 (2011). http://dx.doi.org/10.1016/j.ejso.2010.10.006 Google Scholar

19.

M. Anastasopoulou et al., “Comprehensive phantom for interventional fluorescence molecular imaging,” J. Biomed. Opt., 21 (9), 091309 (2016). http://dx.doi.org/10.1117/1.JBO.21.9.091309 JBOPFO 1083-3668 Google Scholar

20.

J. Glatz et al., “Concurrent video-rate color and near-infrared fluorescence laparoscopy,” J. Biomed. Opt., 18 (10), 101302 (2013). http://dx.doi.org/10.1117/1.JBO.18.10.101302 JBOPFO 1083-3668 Google Scholar

21.

H. Bay et al., “Speeded-up robust features (SURF),” Comput. Vision Image Understanding, 110 (3), 346 –359 (2008). http://dx.doi.org/10.1016/j.cviu.2007.09.014 Google Scholar

22.

M. Muja and D. G. Lowe, “Fast approximate nearest neighbors with automatic algorithm configuration,” in Int. Conf. on Computer Vision Theory and Applications, 331 –340 (2009). Google Scholar

23.

A. A. Michelson, Studies in Optics, Dover Publications, Inc., Mineola, New York (1995). Google Scholar

24.

E. Peli, “Contrast in complex images,” J. Opt. Soc. Am. A, 7 (10), 2032 –2040 (1990). http://dx.doi.org/10.1364/JOSAA.7.002032 JOAOD6 0740-3232 Google Scholar

Biographies for the authors are not available.

Citation Download Citation

Dimitris Gorpas, Maximilian Koch, Maria Anastasopoulou, Uwe Klemm, and Vasilis Ntziachristos "Benchmarking of fluorescence cameras through the use of a composite phantom," Journal of Biomedical Optics 22(1), 016009 (12 January 2017). https://doi.org/10.1117/1.JBO.22.1.016009

Received: 27 September 2016; Accepted: 23 December 2016; Published: 12 January 2017

Access the abstract

JOURNAL ARTICLE
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 24 scholarly publications and 2 patents.

Explore citations on Lens.org

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Cameras

Luminescence

Imaging systems

Signal to noise ratio

Composites

Optical properties

Reflectivity

1.

Introduction

2.

Materials and Methods

2.1.

Standardization Phantom

Fig. 1

2.2.

Imaging Systems

Fig. 2

Table 1

2.3.

Data Acquisition Protocols

Table 2

2.4.

Camera Performance Assessment

Fig. 3

2.4.1.

Magnification

Eq. (1)

2.4.2.

Optical resolution

Eq. (2)

Eq. (3)

2.4.3.

Diffused fluorescence resolution

Eq. (4)

2.4.4.

Excitation light leakage and parasitic illumination

2.4.5.

Sensitivity versus optical properties and depth

Eq. (5)

Eq. (6)

2.4.6.

Field illumination homogeneity

2.5.

Image Registration

Eq. (7)

3.

Results

Fig. 4

3.1.

Assessment of Fluorescence Imaging Sensitivity

Fig. 5

Fig. 6

Fig. 7

Table 3

3.2.

Assessment of Optical/Fluorescence Parameters

3.3.

Correction of Field Illumination Homogeneity

Fig. 8

3.4.

Registration of Color and Fluorescence Images

Fig. 9

4.

Discussion

Disclosures

Acknowledgments

References

Show All Keywords

Keywords/Phrases

Search In:

Publication Years