1.

Introduction

Concentration of tissue constituents, such as hemoglobin, water, or lipid, provides vital functional information about tissue health, which can assist healthcare practitioners in making important decisions. For instance, quantitative information about tissue oxygenation and blood volume fraction can provide visual assistance during surgery and monitoring tissue condition during and after treatment.¹ Even though devices that provide this vital information at a single point already exist, there is a constant push and need toward attaining noncontact, real-time, wide-field and quantitative mapping of tissue functional properties. To address this need, spatial frequency domain imaging (SFDI) is a promising candidate within the field of diffuse optical imaging that has already found commercial application in aiding medical treatment.²

In essence, SFDI exploits the properties of interaction of spatially modulated light (i.e., structured illumination) with the tissue by measuring the “blurring” of the projected fringes and using light propagation modeling to extract optical properties.³ As depicted in Fig. 1(a), a typical imaging system consists of a projection module capable of projecting patterned light at a single wavelength, and a camera for recording images of the field-of-view. Polarizers, arranged in a cross configuration, are commonly used to reduce the effect of specular reflectance and record only diffuse light out of the tissue.

Fig. 1

(a) Schematics of a SFDI acquisition system: a laser diode source is fiber-coupled to a digital micromirror device (DMD) projector. Intensity sinewaves are projected and collected using a CCD camera. (b) SFDI processing workflow: raw images are collected at two spatial frequencies and three phases, then demodulated and calibrated to obtain the diffuse reflectance at these spatial frequencies $(R_0,R_n)$. Finally, an inversion method is used to extract the optical properties maps, namely absorption (${\mu }_{\mathrm{a}}$) and reduced scattering (${\mu }_{\mathrm{s}}^{\prime }$).

The modulation amplitudes $[A_0,A_1,\dots ,A_n]$ of the sample of interest are obtained at each projected spatial frequency $[f_0,f_1,\dots ,f_n]$ by illuminating, acquiring raw data, and demodulating images using multiple phases³ (using SFDI) or a single phase⁴ [using single snapshot of optical properties (SSOP)]. A calibration measurement using a reference phantom with known optical properties is also imaged to obtain modulation amplitudes at the same spatial frequencies $[A_{0,\mathrm{ref}},A_{1,\mathrm{ref}},\dots ,A_{n,\mathrm{ref}}]$. The diffuse reflectance $R_n$ of the tissue at spatial frequency $f_n$ is then given by $R_n=\frac{A_n}{A_{n,\mathrm{ref}}}*R_{n,\mathrm{ref}}$, where $R_{n,\mathrm{ref}}$ is the simulated diffuse reflectance of the calibration phantom at spatial frequency $f_n$. Finally, a light propagation model is used to extract optical properties from the measured diffuse reflectance at several spatial frequencies. This workflow is illustrated in Fig. 1(b).

Because entire images are processed at once in the frequency domain, SFDI has shown the potential to provide very rapid maps of optical properties. The speed of the method can be typically optimized by (1) using a fast acquisition and demodulation method, such as SSOP, and (2) using a fast inversion method to recover the optical properties from the measured data.⁵ It should be noted that both are necessary for SFDI to perform rapidly.

To accelerate the entire process, it has been shown that using only two spatial frequencies can provide accurate estimation of optical properties with errors under 7%.^3,5,6 In a two-frequency SFDI scheme, the amplitudes $[A_0,A_n]$ at low frequency (typically $0{\mathrm{mm}}^{-1}$) and a high frequency (e.g., $0.2{\mathrm{mm}}^{-1}$) are recorded, and the corresponding reflectance maps $[R_0,R_n]$ are obtained using the same calibration step. The diffuse reflectance at these two spatial frequencies and the optical properties have a unique, nonlinear, one-to-one mapping relationship.^3,5,6 This nonlinear mapping relationship is shown in Fig. 2 for a wide range of values of the optical properties.

Fig. 2

Color-coded mapping between the optical properties and the reflectance $R_0$ and $R_n$ (corresponding to spatial frequencies $f_0=0{\mathrm{mm}}^{-1}$ and $f_n=0.2{\mathrm{mm}}^{-1}$).

Given the diffuse reflectance images obtained at two spatial frequencies, the task is to solve an inverse problem of obtaining the optical properties, namely the absorption coefficient (${\mu }_{\mathrm{a}}$) and the reduced scattering coefficient (${\mu }_{\mathrm{s}}^{\prime }$). The inversion is possible by fitting the diffuse reflectance to either diffusion approximation of photon transport or by using a Monte-Carlo (MC) based simulation of photon migration in tissues.³ Using either of these two light propagation models, two basic approaches have been used so far to perform this inversion, namely, least-square fit and look-up table (LUT) method.^3,5,6 On the one hand, precise and robust fitting usually requires diffuse reflectance at multiple frequencies, but it is computationally slow to fit a large number of pixels using either the analytical function provided by the standard diffusion approximation or, especially, with the MC-based simulation. On the other hand, the LUT method uses a pregenerated table of diffuse reflectance for a wide range of optical properties, generally using interpolation to estimate the optical properties.⁵ The LUT method is relatively fast but the computational complexity increases with the density of the grid used to generate the LUT. For a low-resolution LUT, the accuracy of estimation is hampered. A compromise between computational time and accuracy has to be made to use the LUT method. Recent methods that improve the computational time many folds have been proposed.⁵ However, inversion at multiple spatial frequencies and scaling the LUT method to include more parameters other than reflectance has significant computational time and/or memory costs.

In this study, we propose an alternative method for the inversion using a publicly available machine learning technique to estimate the optical properties maps from diffuse reflectance images applied to SFDI. A large set of simulated data using MC was used to train a random forest regression (RFR) algorithm to attain the inversion, allowing in return to obtain directly optical properties maps from diffuse reflectance images in the frequency domain with similar errors than state-of-the-art LUT methods. In the next sections, the method is detailed and its performances evaluated against standard models (diffusion and MC-LUT) on simulated and real experimental data.

2.

Material and Methods

3.

Results

3.1.

Simulation Experiments

Results from simulation experiments are shown in Fig. 3. The plot shows expected optical properties versus optical properties predicted by the diffusion approximation, MC LUT, and the RFR method, which are shown in cyan, red, and blue, respectively. Overall, the MC LUT inversion shows 0.035% and 0.003% mean relative error in estimation of absorption and reduced scattering coefficients, respectively. However, such large size LUT can take up to 8.8 s to compute the optical properties for a set with ${10}^6$ data points. The RFR method computes the predictions in 0.45 s (nonparallelized) and produces a mean relative of error of 0.556% (in ${\mu }_{\mathrm{a}}$) and 0.126% (in ${\mu }_{\mathrm{s}}^{\prime }$). To obtain, a similar run time on MC LUT, we compare the prediction of the MC LUT Low obtained with $100\times 100$ data points. The run time was improved to 0.43 s, but the accuracy of prediction was deteriorated to 1.86% (in ${\mu }_{\mathrm{a}}$) and 0.097% (in ${\mu }_{\mathrm{s}}^{\prime }$).

Fig. 3

Comparison of inversion methods on randomly generated test set of optical properties and reflectance. The symbols in cyan represent the optical properties predicted by the diffusion approximation, red is the prediction by MC-based LUT, and blue symbols represents the RFR method predictions.

3.2.

Phantom Experiments

Figure 4 shows the comparison of the estimated optical properties on six tissue-simulating phantoms by plotting the mean and standard deviation values obtained from each method against each other. The optical properties estimated from RFR method and the dense LUT method are shown to be similar with a maximum difference of 0.46% in absorption and 0.026% in reduced scattering.

Fig. 4

Optical properties of tissue phantoms estimated by the MC-based LUT method ($x$ axis) are compared with the corresponding estimations using RFR method. The values are averaged over $100\times 100\text{pixels}$ of a central pixel in each phantom.

3.3.

In Vivo Experiments

The optical properties maps of an in vivo human hand obtained from the MC LUT, diffusion approximation, MC MUT low, and RFR methods are shown in Fig. 5. As expected, the MC LUT and the RFR methods perform similarly while the diffusion method visually exhibits deviations in both absorption and scattering. To further quantify these results, the percentage error map with respect to the MC LUT was computed. The diffusion approximation shows a mean percentage error of $-16\%$ and $-9.2\%$ in estimation of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, respectively, where the mean is taken over a region indicated by the black rectangle shown in Fig. 5. The mean error in a low resolution MC-LUT is 0.8% and $-0.14\%$ in estimation of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, respectively, while the corresponding error for the RFR method is 0.16% and 0.01% in estimation of ${\mu }_{\mathrm{a}}$ and ${\mu }_{\mathrm{s}}^{\prime }$, respectively.

Fig. 5

(a) Absorption and (b) reduced scattering results of an in vivo hand obtained with all inversion methods (MC LUT, diffusion, MC LUT low, and RFR). For each, the top line shows the optical properties maps and the bottom line shows the percentage error maps in estimation of optical properties compared to the MC-LUT. The black rectangle shows the RoI used for analysis.

4.

Discussion

In this study, we presented a machine learning approach for estimating optical property maps from diffuse reflectance images in the spatial frequency domain. A training set was generated from MC simulations and used to train a RFR algorithm. The RFR method was then tested and validated onto simulations, phantoms, and in vivo onto a human hand, against a dense ($1000\times 1000$) MC LUT. The accuracy of the RFR method was shown to be slightly degraded compared with the MC LUT in all cases, with differences on average less than 0.16% in absorption and 0.01% in reduced scattering in vivo. With very little difference in the accuracy of the recovered optical properties compared to LUTs, the main interest of this work is to propose an alternative approach to estimating optical properties for SFDI without the need of a photon propagation model.

Other machine learning algorithms have been proposed in diffuse optics and in the spatial frequency domain.^7–10 The originality of this work is to propose to apply machine learning to process megapixel images rapidly (450 ms) for SFDI while providing accuracies similar to state-of-the-art, dense LUT methods. While our implementation is not faster than previously published work using optimized LUTs,⁵ this study lays the foundation for using machine learning methods for SFDI with the purpose to replace model-based inversion methods to provide images in real-time during surgery. Our machine learning inversion method can certainly be optimized and improved greatly in processing time by using GPU processing, making such approaches potentially as fast as or even faster than LUT-based methods. Coupled with acquisition methods,^4,13,14 both acquisition and processing can be performed in real-time, a requirement to operate in time-constrained environments such as the operating room.

While limited to simple inversion from diffuse reflectance images at two spatial frequencies to obtain optical properties (similar to LUTs), machine learning is particularly interesting for its versatility as it can be easily scaled to more complex data environments for surgical guidance, such as multispatial frequencies analysis, where the instantaneous spatial frequency at each pixel may differ such as during endoscopic SFDI,¹⁵ multispectral imaging to quantitatively estimate the concentrations of oxyhemoglobin, deoxyhemoglobin, and oxygenation¹⁶ or fluorescence imaging to quantitatively estimate local the fluorescence concentration.¹⁷ The method can also be extended to depth-sensitive measurements,⁷ tomography,¹⁸ 3-D profile correction,¹⁹ metabolic imaging,²⁰ and chromophores mapping²¹ and their combination²² in real-time. These measurement modalities make use of a series of models that are usually nonlinear. Given the access to large amounts of simulated or experimental data, these processes can be learned by machine learning models and provide significant improvements in computation time while preserving accuracies of more computationally costly methods. In addition, GPU implementations of machine learning models can help in speeding up the processing of such complex imaging modalities.

5.

Conclusion

Machine learning methods provide a versatile means for replacing nonlinear models to estimate optical properties from diffuse reflectance images in SFDI without compromising accuracy of estimation. In this study, we presented an RFR algorithm that we trained to perform the nonlinear mapping between diffuse reflectance images and optical properties maps in the spatial frequency domain. The model we presented is capable of computing megapixel maps of optical properties with similar accuracies compared with a standard MC-based LUT. Along with recent developments in real-time acquisition methods in the spatial frequency domain, this work lays the foundation for an integrated imaging system performing acquisition and processing of wide-field images in real-time for surgical applications.