2.1.

Component Design and Selection

The primary components needed for an SFDI system are a source of pattern projection and an image detector to capture the projected patterns on a sample of interest. We chose to use an optical fiber array as the source of projection patterns and a micro camera as the detector.

To create an ultra-miniature fringe projector without using DMD elements, we designed a customized 2D pitch-reducing optical fiber array (PROFA™, Chiral Photonics, New Jersey) to create fringes interferometrically, shown in Fig. 1. The fiber array was designed to produce interference patterns within a widely used spatial frequency range (0.1 to $0.3{\mathrm{mm}}^{-1}$^28,29) at an initial test working distance (WD) of 50 mm when two adjacent channels are illuminated by the same laser source. To compute the required fiber spacings, we used a double slit equation:

Fig. 1

Proposed ultra-miniature SFDI system. (a) Schematic of fiber array in ultra-miniature SFDI system showing dual wavelength illumination simultaneously. Light passes from the two lasers into the fiber array via a selection of seven single-mode fiber input ports. At the tip of the fused taper, the fibers are spaced in a hexagonal array, providing three possible spacings. Crossed polarizers are placed in front of the fiber tip and the micro camera to reduce specular reflections from the imaging sample. (b) Photograph of the experimental setup. (c) Prototype device package of 3 mm diameter with inset showing zoomed in view of fiber tip and camera.

Fig. 2

Determination of the desired fiber spacing to produce spatial frequencies within our range of interest of 0.1 to $0.3{\mathrm{mm}}^{-1}$. (a) Addressable spatial frequency projection at WD of 50 mm (solid lines) and 30 mm (dashed lines). The dotted lines represent the three possible fiber tip spacings of 5, 8.66, and $10\mu \mathrm{m}$. (b) Proposed design of the spatial frequency projection at various WDs for fiber tip spacing ($d$) of $5\mu \mathrm{m}$ (solid lines) and $2.5\mu \mathrm{m}$ (dashed lines), useful for smaller WDs.

The camera chosen is a $1\times 1\mathrm{mm}$ micro camera module (Osiris M module, OptaSensor, Germany). The camera has a resolution of $320\times 320\text{pixels}$, with an individual pixel size of $2.4\mu \mathrm{m}$. An in-built lens placed in front of the sensor provides horizontal and diagonal fields of view of 68 deg and 90 deg, respectively, accompanied by a depth of focus of 5 to 50 mm. The camera module produces a 12 bit RGB raw image output. The camera is accompanied by software to control camera parameters, such as exposure, gamma correction, and black level correction. The automatic exposure correction was disabled so that all image frames contain the same optical power ranges. The micro camera has a frame rate of up to 50 fps, but here we typically operate it at 10 fps due to capture software limitations. However, 10 fps is the minimum rate required for proper endoscopic visualization.³¹

To minimize specular reflections present on the imaging sample, adhesive-backed polymer polarizer sheets are placed in front of the camera and fiber tip to create cross-polarization. The camera is also placed at a small angle of 4 deg to the fiber to further limit specular reflections on the imaging sample. This angle is smaller than conventional SFDI systems,⁶ but it is more amenable to miniaturization. Previous work has shown that this angle can still produce high quality optical property maps³²

2.2.

Phase-Tracking Algorithm

An inherent property of an interferometer such as our fiber array is that the sinusoidal pattern produced shifts over time due to mechanical drifts, vibrations, temperature, and intensity variations.³³ Conventional wisdom may suggest using a complex setup consisting of a phase-shifting control system and a piezoelectric transducer driver to stabilize and control this phase shift.²⁷ However, we exploit the natural phase drift to our advantage via a phase-tracking algorithm.

A video, typically 10 to 20 s, is first recorded of the shifting sinusoidal pattern on a sample of interest. This timeframe is selected purely for experimental convenience, but in practice, only the first second or so of the recorded frames is required. To determine which frames to use for demodulation, we first take an average of all frames within the video and subtract this from each individual frame. This allows us to visualize the spatial frequency pattern with reduced noise [see Fig. 3(a)].

Fig. 3

Characterization of fringes and phase tracking. (a) Image of selected zeroth frame and average of all frames, where $N$ is the total number of all frames within the video capture, and corresponding cross sections. The single frame-averaged frames had a smoothing filter applied. (b) Image of zeroth frame, 120 deg shifted frame, and 240 deg shifted frame and corresponding cross sections, all with smoothing filters applied.

We then take an average across several rows within the frame, applying a smoothing filter, and plot the sinusoidal pattern. We select a zeroth frame and designate the phase of the extracted sinusoid to be 0 deg. Next, we calculate the average distance between the adjacent maxima of this sinusoid in pixels. This value gives the period of the pattern, in pixel units. Custom Python code then cycles through all frames in the captured video and selects frames of equal intensity variation with sinusoidal projections that have relative phases of $(120\pm 10)\mathrm{deg}$ and $(240\pm 10)\mathrm{deg}$ from the selected zeroth frame [see Fig. 3(b)]. Frames in which the sine wave is non-discernible or the intensity variation between peak and trough is low relative to the zeroth frame are disregarded. This eliminates frames in which the coherence is temporarily disturbed while perturbations are still in progress. We then select these frame numbers from the initial video and demodulate the images using Eqs. (1) and (2).

2.3.

Imaging Homogeneous Tissue-Mimicking Phantoms

To perform initial validation of the system, we fabricated tissue mimicking co-polymer in oil phantoms with tunable optical properties by controlling concentrations of ${\mathrm{TiO}}_2$ and Nigrosin dye.³⁴ The fabricated phantoms had a thickness of 30 mm and were ensured to be non-transparent to meet the semi-infinite thickness requirement of SFDI.³⁵ We fabricated two phantom batches: one with increasing amounts of dye stock solution from 0.5 to 1 g (0.0015 to 0.0030 w/v%) corresponding to an absorption coefficient range of 0.006 to $0.017{\mathrm{mm}}^{-1}$ at 660 nm and the second with increasing amounts of ${\mathrm{TiO}}_2$ from 0.07 to 0.13 g (0.07 to 0.13 w/v%) corresponding to a reduced scattering coefficient range of 0.52 to $0.99{\mathrm{mm}}^{-1}$ at 660 nm. The batches with an increasing dye stock solution each had 0.1 g of ${\mathrm{TiO}}_2$ and the batches with an increasing ${\mathrm{TiO}}_2$ each had 0.5 g of the dye stock solution to ensure the semi-infinite material requirement was met. We chose these optical property ranges as they are within the optical properties of interest of typical gastrointestinal tissue samples¹⁷ and they had previously been calibrated in the literature using a double integrating sphere (DIS).³⁴

For the purposes of comparison, we imaged phantoms in both our ultra-miniature system and a standard bench-top SFDI system. The bench-top system was built in-house using a commercial projector (LG Minibeam PH150g HD ready mini projector), a Raspberry Pi camera, crossed polarizers placed in front of the camera and projector, and a 635 nm filter placed in front of the camera to ensure that only red light was captured. The system was validated against phantoms with optical properties measured in a DIS system and found to be in agreement with a margin of error around 20%, which is within the typical expected range.³⁶

The phantoms were placed in the bench-top system such that the projector is placed at a 6 deg angle to the plane of the camera, the projector–camera distance is fixed at 35 mm, and the distance from the projector–camera plane to the base breadboard is 220 mm. The diffusion approximation was used to generate a look-up table to recover the optical properties of the phantoms. Because our ultra-miniature system uses a laser at 660 nm and the bench-top system uses white-light source with a camera filter at 635 nm, a small adjustment for wavelength is required. First, separate LUTs are computed for each of the wavelengths. Second, an adjustment based on multiwavelength measurements from a DIS system is applied.³⁴ For these particular phantoms, the measured difference via DIS between absorption and reduced scattering coefficients from 635 to 660 nm is 14% and 3%, respectively. Therefore, we adjust reference optical properties for optical property calculation depending on whether we were using the bench-top or ultra-miniature system. When imaging phantoms at 515 nm in the ultra-miniature system, the LUT and reference optical properties were also adjusted accordingly.

We placed the phantoms such that the top of the phantom was 50 mm from the distal end of the imaging probe and the projection pattern was in the center of the sample. We took videos of the shifting projection pattern on the phantom for 10 to 20 s. The video was then input to Python phase-tracking code for processing, described in Sec. 2.2, to find the exact frames needed to calculate the optical properties. We imaged each phantom at three different spatial frequencies by illuminating three different fiber channel combinations. We calculated the optical properties using a look-up table generated from the diffusion approximation. For each phantom, we calculated the optical property maps a total of 18 times, using every other phantom as a reference in turn for each spatial frequency. This approach helps to average out errors arising from mismatches in expected optical properties of phantoms, which arises in turn due to discrepancies between DIS and SFDI measurements, which can be up to 20%.³⁶ Finally, the mean of all 18 optical property maps is used to determine the absorption and reduced scattering coefficients. A 2D Gaussian filter with a standard deviation of 5 pixels was applied to the resultant optical property maps using scipy.ndimage.gaussian_filter.

2.4.

Dual Wavelength Imaging

Multi-wavelength imaging is possible with this system as the fiber array consists of seven channels and only two are needed per wavelength to produce an interference pattern. Therefore, this system has the potential to explore up to tri-wavelength simultaneous illumination. We imaged three phantoms with 660 nm projection only, then 515 nm projection only, and finally with 660 and 515 nm projected simultaneously. We performed dual-wavelength imaging by illuminating channels 1 and 7 with 660 nm and channels 2 and 5 with 515 nm, producing spatial frequency patterns of 0.3 and $0.2{\mathrm{mm}}^{-1}$, respectively, at a 50 mm WD. A video was captured of both illumination patterns simultaneously, and the analysis was carried out by extracting the red and green channels from the video capture. Following the same process in Sec. 2.2, the fringes of interest were selected and optical properties were calculated. Expanding the existing system to three wavelengths (tri-color) would be possible by connecting an additional blue laser (e.g. 450 nm) to two unused illumination channels and extracting the blue channel of the captured video. Multi-wavelength imaging would probe different depths and could be used to image optical properties of layered material.

2.5.

Extrapolating Performance to Clinical Applications

The ultra-small size of our prototype device may incur performance penalties compared to standard bench-top SFDI. Qualitatively, this appears as noise in measured images. However, the ultimate goal of the tool is to provide sufficient contrast for identifying diseased tissue compared to healthy tissue. To assess this, we post-process our optical property measurements to estimate the sensitivity and specificity that could be achieved in a clinical device. This is then compared against American Society of Gastrointestinal Endoscopy (ASGE) guidelines, which require a new device to demonstrate 90% sensitivity and 80% specificity to be used in lieu of biopsy sampling.³⁷

To extrapolate our device’s potential clinical performance, we simulate random measurements based on previously measured absorption and scattering statistics of healthy esophageal tissue, Barrett’s esophagus, and SCC.¹⁷ Because of the non-negativity and underlying multiplicative nature of absorption and scattering processes, a log-normal distribution is used to generate samples.³⁸ A baseline model that estimates that performance under the assumption that intra-sample variation is purely biological in origin is first created. To adapt this into the model used here, we add an additional term to the variance based on error measurements observed from our ultra-miniature system.

500 random training samples are generated for the healthy, Barrett’s, and SCC tissue types, and two binary linear support vector machine (SVM) classifiers are trained (healthy versus Barrett’s and healthy versus SCC). The SVMs are implemented using the MATLAB software package. A further 500 validation samples are then generated for each of the three classes and passed through the trained classifiers. Based on the classification error rates, sensitivity and specificity for the baseline and ultra-miniature system are estimated.

Another important clinically relevant parameter is the minimum size lesion that can be detected: lesions smaller than 6 mm are not typically considered high risk and thus are not removed. There are two aspects to this: the first is the effect of the image elements and apertures on the MTF, which can be estimated by imaging a resolution test chart. The second aspect is the impact of noise in the recovered optical properties. One way of considering this is that increased noise increases the uncertainty when identifying the “edge” of a step function. This in turn translates to a larger range of spot sizes, reducing the resolution. To quantify this, we first image two phantoms with different optical properties placed side by side so they are touching, which should ideally produce a “step” function. This step function manifests in our images as a sigmoid due to the system MTF. We then use a bootstrap sampling approach to generate random sigmoid functions and compute their probability of fit to step at the boundary between the phantoms using the measured noise statistics as the parameters of Gaussian probability density functions. We then reject samples according to their probability (i.e., their likelihood of being correct) to obtain a representative sample of likely sigmoid functions that could adequately fit the boundary between phantoms. Next, these sigmoids are differentiated to give their corresponding impulse function (i.e., spot function for imaging). By taking the mean across these sampled spots and computing the full-width half maximum, we get an estimate of how the expected ability to resolve structures changes as a result of noise.

3.1.

Projector Performance

The expected spatial frequency of the projected illumination pattern is comparable to the desired spatial frequency with 12% and 7% error for 660 and 515 nm, respectively. Some channels produce clearer interference patterns than others due to cross talk between fibers. This also results in the interference pattern from some channels being more stable than others in time, with interference patterns being stable for $<1\mathrm{s}$ under typical operating conditions, but $>10\mathrm{s}$ if the fibers are kept still.

Through imaging a resolution target (R3L3S1N - Negative 1951 USAF Test Target, 3” x 3”, Thorlabs, United Kingdom), we determined the resolution of the imaging system to be $0.793\mathrm{lp}/\mathrm{mm}$ at a WD of 50 mm, shown in Fig. 4(a). Figure 4(b) shows the raw performance of the projection fiber with green and red wavelengths onto an absorbing and scattering phantom. This represents a typical raw image required from the camera. Finally, Fig. 4(c) shows a direct capture on a CMOS image sensor of the projected pattern from the fiber. This shows that clear fringes are produced within an envelope, but that there is also some noise arising from aperture and cross-talk effects forming a ring around the target area. We, therefore, restrict our analysis to regions within this ring where fringes are projected with high quality.

Fig. 4

Raw performance of the camera and projector. (a) Image of USAF target captured with a mini camera module taken with room lights on; (b) image captured with mini camera module of dual wavelength projection from fiber tip showing extracted red and green channels, respectively; and (c) direct capture on image sensor of projected fringe pattern, showing good fringe contrast within a region of circular interference.

3.2.

Phase-Tracking Stability

We analyzed the fringe phase shift and contrast for 15 recorded videos, during which we continually perturbed the fiber by hand to simulate realistic usage. An example trace of fringe phase versus time is shown in Fig. 5(a), which indicates that the required three phases can be obtained under 1 s. Figure 5(b) shows the calculated difference from maximum to minimum of the interference fringes as a function of time, i.e., the fringe contrast. The contrast appears relatively stable over the 3 s time interval. To further analyze the combined effect, we used Poisson statistics across these 15 captures to estimate the expected time at the maximum frame rate required to obtain three usable frames (defined as having sufficiently high contrast across three phases) with 99% probability. We found that this varied from 0.42 to 2.34 s with a mean of 1.17 s. This may be sufficient for practical imaging provided the probe can be held within the same field of view for 1 to 2 s.

Fig. 5

Fiber array performance. (a) Phase of frames versus time and (b) fringe contrast, defined as the difference from maxima to minima of interference fringes detected versus time.

To examine the stability of the fringe envelope, we averaged all frames together for individual video captures and plotted the cross-sectional envelope profile. By repeating this across five separate video captures using a range of different phantoms, we observed how the fringe envelope varies. Given that the envelope shape is largely determined by the fiber exit aperture, we would expect the shape to be relatively uniform across different samples, although effects such as cross-talk between fibers may cause a variation. In addition, fluctuations in laser power, laser polarization state, or small transients in relative power in different fibers due to manual perturbations may influence the envelope.

From our results, shown in Fig. 6, we observe that the shape of the envelope is fairly consistent across different captures, though the absolute scale does appear to change within a range of $\pm 10\%$. In future systems, this could be partially compensated by having a power meter recording laser output that can be used later for normalization.

We therefore expect that there may be some variation between the reference phantom envelope and the measurement: the envelope will be approximately correct but the amplitudes may vary, which could be a contributing source to the overall system error, in the region of 10%.

Fig. 6

Plot of average fringe envelope for five different captures using phantoms with different optical properties. Though the amplitude of the envelop varies in intensity, the position and shape stay relatively constant.

3.3.

Comparing Ultra-Miniature SFDI System to Bench-Top SFDI System

We then compared optical property measurements from our conventional bench-top SFDI system to our ultra-miniature SFDI system. The results are shown in Figs. 7(a) and 7(b). We found that the average standard error in absorption and reduced scattering coefficients between the ultra-miniature system and the bench-top system were 15% and 6%, respectively. However, we note that the slope of the absorption graph is much less than unity, reaching as high as 60% in the high absorption range. This may be due to a range of factors including stray background light, non-ideal sinusoid shapes, and the resulting mismatch in the LUT or varying envelope amplitudes. Because the absorption is consistently underestimated, this may be rectified using additional calibration steps, for example, phantom-based LUT generation³⁹ or realistic simulation of the fiber imaging setup.³²

Fig. 7

Comparison of the bench-top SFDI system and the ultra-miniature system: (a) absorption coefficient and (b) reduced scattering coefficient measured from bench-top system ($x$ axis) and ultra-miniature system ($y$ axis). Error bars represent the standard deviation across the image.

3.4.

Imaging Typical Gastrointestinal Conditions with Ultra-Miniature SFDI System

We fabricated two phantoms to simulate gastrointestinal tissue states: one with optical properties mimicking SCC and the second with optical properties mimicking healthy esophageal tissue, and we placed them side by side. SFDI imaging of this sample was then performed at 660 nm, with the resulting optical property maps shown in Figs. 8(c) and 8(f), and resultant optical property maps with filtering applied shown in Figs. 8(d) and 8(g).

Fig. 8

Imaging a phantom simulating esophageal tissue at 660 nm: (a) white light image of two phantoms with different optical properties side by side; (b) expected and (c) measured absorption coefficient of phantoms; (d) measured absorption coefficient with smoothing filter applied; (e) expected and (f) measured reduced scattering coefficient of phantoms; (g) measured reduced scattering coefficient with smoothing filter applied. Expected optical properties are computed from the mean measured values of the individual phantoms measured in the bench-top SFDI system.

To illustrate the effect of speckle averaging, we compared the optical properties of the same phantoms measured with just a single spatial frequency to three spatial frequencies averaged together. The results are shown in Fig. 9: Figs. 9(a) and 9(d) represent the expected optical properties of the phantoms, Figs. 9(b) and 9(e) show the optical properties measured from a single spatial frequency projection, and Figs. 9(c) and 9(f) show the optical properties measured by taking the average over three different spatial frequency projections. The standard deviation of pixels in the absorption coefficient maps reduces from 0.0049 to 0.0044 in Figs. 9(b) and 9(c). The standard deviation of pixels in the reduced scattering coefficient maps reduces from 0.32 to 0.15 in Figs. 9(e) and 9(f).

Fig. 9

Visual representation of averaging over several spatial frequencies to reduce speckle noise: (a) expected absorption map; (b) recovered absorption map using a single spatial frequency; (c) recovered absorption map by averaging three different spatial frequencies (i.e., different fibre pairs); (d) expected scattering map; (e) recovered scattering map using a single spatial frequency; and (f) recovered scattering map by averaging three different spatial frequencies.

Using our post-processing simulation with SVM classifiers, we found that, for detecting Barrett’s esophagus versus healthy, we achieved a sensitivity of 76.2% and a specificity of 82.8%. This falls just short of the ASGE guidelines.³⁷ However, for SCC versus healthy, we find $>99\%$ sensitivity and 89.6% specificity. This meets the ASGE guidelines and is in agreement with experimental pre-clinical SFDI studies that typically achieve values $>90\%$ for both sensitivity and specificity when comparing healthy and cancer tissue.^40,41

Next, using our method to analyze the impact of noise on resolution, we find that for absorption the FWHM of our instrument spot size is 4.1 mm versus an estimated 3.7 mm for bench-top systems presented in the literature.¹⁷ Similarly, for scattering, we find that our spot size is 0.74 versus 0.39 mm for bench-top systems in the literature.¹⁷ This discrepancy in resolution between scattering and absorption is due to the higher observed noise in absorption. We can therefore conclude that our system should feasibly be able to detect lesions of 4 mm size at the current WD used.

3.5.

Dual-Wavelength Imaging

Finally, we characterized the performance across the two wavelengths. We found that the recovered optical properties varied by $\le 10\%$ when the two wavelengths were measured simultaneously rather than sequentially. This demonstrates the capability of the system to image optical properties at two wavelengths simultaneously with relatively low cross-coupling.

We then imaged two phantoms with different optical properties placed adjacent to one another, one mimicking the optical properties of SCC and the other mimicking the optical properties of healthy esophageal tissue. The results are shown in Figs. 10(a)–10(l). The difference in optical properties is visible from both the red and green channels. The optical properties measured from the red and green channels are not expected to be the same as the phantom properties shift with wavelength.³⁴ We expect the phantom optical properties measured from the green channel to be higher than phantom optical properties measured from the red channel.

Fig. 10

Optical properties measured from dual-wavelength imaging experiment: (a) expected absorption coefficient from red channel; (b) measured absorption coefficient from red channel with (c) filtering applied. (d) Expected absorption coefficient from green channel; (e) measured absorption coefficient from green channel with (f) filtering applied. (g) Expected reduced scattering coefficient from red channel; (h) measured reduced scattering coefficient from red channel with (i) filtering applied. (j) Expected reduced scattering coefficient from green channel; (k) measured reduced scattering coefficient from green channel with (l) filtering applied.