2.1.

Measurement Speed

The acquisition speed of OCT, typically measured in A-scans per second or voxels per second, has been increasing since its inception. This steady increase in speed has supported the continuous expansion of OCT capabilities and applications. Initially, OCT’s predecessors low-coherence interferometry and optical time domain reflectometry had been used to measure distances along the sample beam or along an optical fiber.^19,20 Early time domain OCT systems for ophthalmic diagnostics then recorded B-scans with an A-scan rate of several hundred hertz.^21,22 The introduction of Fourier domain OCT (FD-OCT), however, had the biggest impact on speed as well as clinical usability. It increased the A-scan rate to tens of thousands of A-scans per second. With such rates, volume capture scans and high definition scans, i.e., an averaged B-scan calculated from up to 100 B-scans, replaced individual B-scans in ophthalmic diagnostics. More recently, technological improvements of line scan cameras, tunable light sources, digitizers, and data transfer interfaces pushed the speed further to hundreds of kHz and even MHz A-scan rates.^23–26 Such speeds now permit the observation of dynamic processes with live volumetric OCT scans, called 4D OCT, in which the fourth dimension is time.²⁷

In theory, both FD-OCT implementations, SD-OCT and swept-source OCT (SS-OCT), have similar sensitivity and should thereby reach similar acquisition speeds. However, in practice, SS-OCT has lower losses in the detection and does not suffer from a depth dependent sensitivity roll-off in the case of long coherence length sources such as vertical-cavity surface-emitting lasers (VCSELs) or akinetic swept sources, introduced by the limited modulation transfer function of the spectrometer. At imaging depths of several millimeters, SS-OCT may therefore exhibit $>10\mathrm{dB}$ higher sensitivity than its spectrometer-based counterpart. In consequence, the highest speed confocal point scanning OCT systems so far have been SS-OCT systems. The fastest reported SSOCT systems so far use dispersive stretching of ultrashort pulses propagating through a fiber.^28–30 These OCT systems are currently limited by missing fast enough data acquisition boards to sample the stretched ultrashort pulses running at up to 100 MHz. The highest voxel rates are reported for SS OCT systems employing a so-called circular ranging technique, which uses a frequency comb spectrum leading after Fourier transform to degenerate depth ranges sampled in parallel.³¹ The most convincing performance for in vivo imaging so far has been demonstrated with systems powered by Fourier domain mode locked lasers,²⁶ although vertically VCSEL-based systems are gradually closing in concerning speed and bandwidth performance.³²

Such high speeds, however, come at the price of decreased sensitivity, at least when imaging light sensitive samples in which the illumination power cannot be arbitrarily increased, such as the human eye. Ophthalmic OCT systems therefore rely on fast safety circuits, which monitor the motion of the scanning beam and the illumination power to quickly shut off the laser in case of a failure. This permits treating the scanned beam as an extended source and thereby higher permissible light exposure than with a stationary beam. However, because it takes time to detect an unintended slowdown of the scanners, the maximum applicable power with confocal point scanning systems will soon be reached.

Another factor limiting the speed of point scanning devices is the requirement for fast scanners. To minimize strain of the scanners and maximize scan speed, sinusoidal or spiral scan patterns may be applied; however, this introduces the need for a resampling step in postprocessing.³³

To continue the trend of ever-increasing imaging speed, we see a shift from single-beam confocal point scanning systems toward parallel systems.^34–36 These include systems that illuminate the sample with multiple confocal beams in parallel, often referred to as multibeam OCT systems, and systems that illuminate a line or an area on the sample, called line field and full-field OCT (FFOCT) systems, respectively (Fig. 1).

Fig. 1

MHz OCT systems: (a) equivalent A-scan rates; for parallel systems, this is the number of parallel channels times the A-scan rate; shown are published systems with rates $\ge 1\mathrm{MHz}$. (b) Data rates in gigasamples/s; only systems with sufficient data available have been included; fastest A-scan rates are achieved by stretched pulse laser OCT; fastest data rates were demonstrated by circular ranging OCT systems. In both categories, full field swept source OCT is a key player (PS, point scanning; TD, time domain; SS, swept source; and SD, spectral domain).

2.1.2.

Line-field OCT

Line-field OCT (LFOCT) comes at all flavors of OCT: TD-OCT, spectral domain OCT (SD-OCT) and swept source OCT (SS-OCT) implementations have been demonstrated.^35,37–40 Its line illumination has a major advantage: because an entire B-scan is captured at once, only one scanner is required to acquire a volume scan. In most cases, this scanner can even have a lower performance than typical galvanometric scanners used in point scanning systems as the scan in the orthogonal direction relative to the B-scan is typically relatively slow. Further, a line illumination maintains confocality in one dimension and never focuses to a spot. This is beneficial for the suppression of multiply scattered photons and permits high illumination powers when imaging the human retina. TD LFOCT particularly allows for keeping the confocal gate aligned with the coherence gate. Using a high-resolution system with tight coherence gating, such confocal LFOCT, has enabled the production of impressive images of human skin that efficiently suppress scattering⁴¹ [Fig. 2(b)].

Fig. 2

Examples for FF and LF OCT. (a) High-resolution large FOV retinal FFOCT. Panels 1 to 4 visualize the cone photoreceptor mosaic at different eccentricities as shown in panel S2 (scale bar $100\mu \mathrm{m}$). Reproduced from Ref. 42., © 2020 Optical Society of America (OSA). (b) TD LF OCT of human skin, using dynamically aligned confocal and coherence gating (K, keratinocytes; DEJ, dermal–epidermal junction; SD, sweat duct; H, hair; CF, collagen fibers; BV, blood vessels. Scale bars: $200\mu \mathrm{m}$). Reproduced from Ref. 41, © 2018 OSA. (c) Retinal imaging with LF SS OCT at 600 kHz equivalent A-scan rate. Reproduced from Ref. 39, © 2015 OSA. (d) Computational adaptive optics for in vivo high-resolution retinal imaging (left: uncorrected enface slice from photoreceptor layer; center: reconstructed wavefront error; and right: corrected enface slice by phase conjugation of the wavefront error to the pupil plane phase; error bars $100\mu \mathrm{m}$). Adapted from Ref. 43. (e) Advantage of FFOCT phase stability for functional OCT assessment of photoreceptor response (upper row: enface plane at OS photoreceptor layer of recorded volume together with tomogram; lower row: response measured as phase change over time after light stimulus onset; inlay shows the actual stimulus mask). Adapted from Ref. 44.

In SD-OCT configurations, B-scan rates of several kHz can be achieved by employing standard off-the-shelf 2D cameras.^43,45 Such high B-scan rates make LFOCT attractive for functional extensions that require fast repeated scans, such as elastography, OCTA, or Doppler OCT (see Sec. 4). However, high acquisition speed in general is beneficial for achieving high phase stability, making LFOCT also a candidate for phase sensitive extensions, such as computational adaptive optics^43,46 [Fig. 2(d)], dynamic OCT, or functional OCT⁴⁷ (see Sec. 4).

SS-OCT implementations require very fast line scan cameras. The fastest suitable off-the-shelf line scan cameras reach line rates of $\sim 300\mathrm{kHz}$. Assuming 1000 spectral sampling points, this results in a B-scan rate of 300 Hz. The number of sampling points can be reduced using an holographic off-axis approach, resulting in full range imaging with up to a 1 kHz B-scan rate³⁹ [Fig. 2(c)]. Yet, the lower axial sampling poses limitations on its practical use for clinical imaging. Although the B-scan rate is comparable to state-of-the-art commercial point scanning OCT systems, the low spectral sampling rate may pose problems in the case of fast sample motion. It introduces a phase shift across the spectral interferogram, which results in a point spread function broadening, similar to a dispersion mismatch between reference and sample arm. For example, for retinal line-field imaging, using B-scan rates of several kHz⁴³ has been recommended to avoid axial blurring. Recently, a high-speed CMOS camera has been applied for fast line field sensing to overcome the motion limitation, but at increased costs for the sensor.^47,48

The advantages of LFOCT and its technical feasibility using readily available components make it the most promising candidate for parallel OCT embodiments and put it closest to market translation.

2.2.

Lateral Resolution in OCT

The lateral resolution of OCT systems is given by the numerical aperture (NA) of the imaging optics. In contrast to, for example, confocal microscopy, it is decoupled from its axial resolution (see Sec. 2.3).⁵⁹ Most FD-OCT systems are therefore designed with a comparably low NA to create a large depth of focus and thereby enable an instantaneous imaging depth of several millimeters.

Diffraction limited resolution can be achieved when the sample itself does not introduce wavefront aberrations. However, especially when imaging the human retina, the imperfect optics of the eye introduces large wavefront errors that prevent diffraction limited resolution when making use of their full NA. It has been shown that the diffraction limited resolution can be recovered by the use of adaptive optics,⁶⁰ using deformable mirrors, or computationally by using the interferometric phase that different approaches have reported, ranging from algorithms from diffraction tomography⁶¹ to iterative⁶² and non-iterative approaches⁶³ to extract and manipulate the wavefront shape. This enables the resolution of individual cells in some retinal layers, i.e., photoreceptors and retinal pigment epithelium (RPE) cells. Although other retinal cells, in theory, are large enough to be resolved even by standard OCT systems, such as ganglion cells, they often remain hidden due to limited contrast. However, it has been shown that averaging a large number of acquisitions or applying dynamic contrast methods helps to reveal them even in the living human eye.⁶⁴

In confocal systems, hardware-based adaptive optics has the advantage of correcting the wavefront optically before the light is coupled back into a single-mode fiber, thereby maximizing the signal. Deformable mirrors, especially ones with a large number of elements, are costly, which so far has prevented their use in commercial OCT systems and makes computational approaches more attractive. To overcome the prohibitive cost of hardware-based adaptive optics OCT on the one hand and the limited collection efficiency of confocal computational adaptive optics OCT systems on the other hand, more basic deformable mirrors may be used to correct low-order aberrations optically and higher orders numerically.

The detection of an aberrated wavefront is not a concern in FFOCT, as long as the detector has sufficient resolution to resolve the wavefront’s phase slope. Because computational wavefront correction algorithms can propagate the focal plane to any depth, they can realize a depth invariant lateral resolution, whereas high NA hardware-based adaptive optics OCT systems are limited to a depth of focus of only a few micrometers.

Cellular and subcellular resolution is the field of optical coherence microscopy (OCM). TD OCT is a natural candidate for OCM since the tight confocal gate can be dynamically aligned with the coherence gate during depth scanning.⁶⁵ For FD OCT configurations with a fixed reference arm, other means are needed to extend the FOV beyond the tight confocal gate. Again, computational methods have shown their merit for OCM by extending the focus depth.^66,67 These include hardware-based methods used depth fusion approaches,⁶⁸ Bessel beams,⁶⁹ structured illumination,⁷⁰ or employed metalenses.⁷¹

2.3.

Axial Resolution in OCT

The axial resolution of OCT is given by the center wavelength and spectral bandwidth of the illumination.⁵⁹ The most prominent wavelength band of OCT systems lies at 840 nm. This is a result of OCT’s success in ophthalmology. The near-infrared light permits good penetration through the water-like filled eye to the retina, while not blinding the patient and still maintaining good detection efficiency with silicon cameras. Most OCT in this wavelength band are SD-OCT systems using super-luminescence diodes or Titanium:sapphire lasers. With full-width half-maximum bandwidths of up to 180 nm, an axial resolution down to $1\mu \mathrm{m}$ in tissue can be achieved.⁷² However, commercial ophthalmic SD-OCT systems typically employ sources with a much narrower spectrum, realizing an axial resolution of $\sim 5\mu \mathrm{m}$ in tissue. If an axial resolution below $1\mu \mathrm{m}$ in tissue is desired, super-continuum sources with spectral bandwidths of several hundred nanometers that stretch far into the visible spectrum can be used (see Sec. 4.3).

Most SS-OCT systems operate at longer center wavelengths, 1060 nm for imaging the posterior segment of the human eye, and 1310 and 1550 nm for imaging more strongly scattering samples, such as skin or brain tissue. The achievable resolution at 1060 nm is typically set by the water absorption window, which limits the useful bandwidth to $\sim 100\mathrm{nm}$, resulting in a typical axial resolution of $\sim 7\mu \mathrm{m}$ in tissue. At 1310 and 1550 nm swept-sources with a broader tuning range are available; however, the achievable axial resolution is similar due to the higher central wavelength.

3.1.

Non-linear Optical Microscopy and Spectroscopy: Subcellular, Biochemical, and Metabolic

Detection of structural information at the cellular level expands the understanding of tissue environment for in vivo studies, but the lack of molecular specificity hampers differentiation between pathologic and healthy tissues with comparable scattering or structural properties. Structural alterations in tissues and cells usually take place only after carcinogenic biochemical changes. OCT contrast can be improved by various implementations. However, molecular specificity including metabolic information at the cellular level is still not easily accessible. Therefore, despite the power of OCT, the sensitivity and specificity to detect pathologic tissue are limited.

This weakness limits the ongoing success of this technology. One solution to overcome this limitation and to address current needs in the life sciences and in the clinical practice is to combine OCT with other non-invasive molecular specific modalities. Label-free spectroscopic and microscopic optical imaging technologies such as Raman spectroscopy (RS)⁷⁵ and MPM^76,77 that complement OCT¹ have been established and extensively applied over the past years. These modalities present comparable contrast to standard histopathological methods,⁷⁸ but no single modality can play the role alone.

Indeed, integrated multimodal imaging provides the possibility of fusing morphological information with metabolic-molecular information in a label-free way, complementing basic observation with multiple specific contrast mechanisms to gain a complete picture of disease, but it is still challenging due to different optics and hardware requirements.⁷⁹ MPM has become an essential instrument for biological and medical research with inherent 3D sectioning capability, subcellular resolution, high sensitivity, molecular and metabolic specificity, and deep tissue penetration compared with confocal microscopy, but compared with OCT achievable FOVs, penetration depths and acquisition speeds are restricted. This weakness can be overcome with novel high-speed MPM approaches achieving kilohertz frame rates by implementing pulse-modulated, rapidly wavelength-swept lasers, and inertia-free beam steering via angular dispersion.⁸⁰ In the future, this could help to match the different needs of OCT and MPM, thus facilitating the combination of these complementary techniques in a single co-registered platform.

Usually, the key technologies to add molecular sensitivity to OCT are RS⁸¹ and coherent Raman spectroscopy. RS allows for full molecular fingerprinting of tissue offers high specificity. Several variants including non-linear vibrational imaging⁸² with different sophisticated laser sources^83,84 and single-laser source approaches^85–87 have been proposed, but the main deficit is that real-world applications are often missing due to special treatment of the sample—thin slices of transparent samples with high Raman cross sections to be investigated in transmission or reflection mode. Coherent anti-Stokes Raman scattering efficiently provides images with label-free molecular information from DNA, lipids, proteins, and collagen. Recently, stimulated Raman scattering-spectroscopic OCT was introduced to leverage the spatial and spectral multiplexing capabilities of OCT with the molecular specificity and sensitivity of SRS for label-free fast 3D molecular imaging with a single laser on a single platform.⁸⁸ Two-photon excitation fluorescence (TPEF) microscopy^89–94 is another powerful OCT add-on that can intrinsically be merged with SHG^91,95–97 or fluorescence lifetime imaging microscopy (FLIM),^98–100 showing augmented contrasts with the same FOV and resolution for all modalities. TPEF microscopy can bring unique additional insight into the mechanisms underlying immune system dynamics and function as cellular motility within the native environment in vivo. TPEF microscopy and FLIM achieve ultrahigh isotropic subcellular resolution, enhance chromophore contrast via excitation of fluorescent biomolecules, improve sectioning compared with conventional one photon fluorescence, and metabolic information by imaging endogenous metabolites such as nicotinamide adenine dinucleotide and hydrogen/flavin adenine dinucleotide. Pleomorphism (cell nuclei versus cell size ratio) can be detected with subcellular resolution imaging, and grade classification can be performed. Co-registered OCT and TPEF microscopy has the ability to link specific cellular phenotypes and functions as revealed by TPEF to tissue morphology, thus paving the way between basic research knowledge and clinical observations.¹⁰¹

SHG provides contrast from non-centrosymmetric molecules such as collagen, which mainly appears in the extracellular matrix as distinct morphological feature. The supramolecular organization can be revealed. In vivo skin imaging on a cellular level proves the potential for dermatology.^102–104 THG microscopy provides cellular morphological information in real-time with acceptable penetration depth and ultrahigh isotropic subcellular resolution, but reduced penetration depth and remarkable smaller FOV compared with OCT (Fig. 3).

Fig. 3

(a) Integrated approach using cross-polarization OCT, multiphoton tomography, and FLIM in skin equivalent preclinical research. Adapted from Ref. 99. (b) 3D reconstructed image in the popliteal lymph node of a CX3CR1:eGFP mouse: OCT (magenta) in the whole lymph node, SHG (blue) of collagen in the lymph node capsule, GFP fluorescence of CX3CR1+ cells (green), and rhodamine dextran signal in blood vessels in the whole lymph node and phagocytes, especially in the subcapsular sinus (red). Adapted from Ref. 101. $\text{Scale}200\mu \mathrm{m}$.

The problems with these multimodal optical imaging approaches with MPM are the need for very high-photon concentration in space and time and subsequent requirement of extremely high near infrared (NIR) laser intensities. Ultra-short laser pulses provide high-peak powers necessary to achieve appropriate excitation powers for MPM signal generation with moderate time-averaged illumination doses. Ultra-broadband Titanium:sapphire lasers, which are proven and well-established light sources for MPM, are a challenge in the commercialization of such combined platform since they introduced high costs and complexity.^105–109 Alternative stable and cost-effective light sources are required.¹¹⁰ Direct diode-pumping of mode-locked Titanium:sapphire lasers¹¹¹ and scaling up of the achievable output power¹¹² could pave the way toward more widespread application of this technology beyond scientific research.¹¹³ Recently, advancements in soliton-based photonic integrated frequency comb soliton microcombs have accelerated the development of broadband and low-noise chip scale frequency comb sources with the potential for high-resolution OCT deep tissue imaging at 1300 nm.¹¹⁴ Combining OCT with MPM at these longer wavelengths or even at 1700 nm can enhance tissue penetration. The key enabling elements to improve the imaging penetration depth, depth of focus, and spatial resolution simultaneously are the laser wavelengths and pulse energy, beam shaping concepts, detection schemes, and real-time AI algorithms. First, deploying longer wavelengths, e.g., 1700 nm for OCT, TPEF, SHG, and THG, the penetration depth for imaging is significantly improved, facilitated by the novel ultrafast laser wavelength agility. Second, novel beam shaping allows for propagating invariant light fields that achieve increased depth, thereby improving penetration and retaining high resolution. Third, to compensate for attenuation intrinsically present in tissue, absorption correction is applied, and, more importantly, dynamic changes from the strong optical scattering are compensated for by so-called computational adaptive optics. The latter relies on OCT motility contrast as a measure of the wavefront for correcting the input and real-time data processing. Penetration depth could be even further improved using light sources at longer wavelengths. Recently mid-infrared OCT could be demonstrated with an axial resolution of $8.6\mu \mathrm{m}$.¹¹⁵ Detection could be performed with a standard CCD camera upon implementation of an upconversion module.

One major consideration for future development of the multimodal morpho-molecular metabolic imaging platforms lies in the capability of performing fast OCT and successive simultaneous MPM in the backward propagation direction without affecting one modality to interfere with the others, thus compromising image quality. Hence, one solution in multimodal diagnostic imaging is the combination of a wide-field high-resolution imaging platform including OCT and advanced microspectroscopic imaging techniques and an automated software to extract and classify the morpho-molecular metabolic patterns with the aim to see what can be seen in immunohistochemistry from the macroscopic level to the highly specific microscopic level. With deep learning, automatized full tissue characterization on a multiparametric level can reveal the early onset of disease and improve understanding.

However, deep optical imaging, such as deep brain imaging in vivo, at high resolution still poses a great challenge beyond the light source development. Adaptive optics based on direct wavefront sensing can recover diffraction-limited resolution corrupted when achieving deep optical imaging. Successful implementation of adaptive optics in any optical system is always an engineering challenge and often makes the overall system complex and economically expensive. The subaperture correlation-based computational adaptive optics method, which is the computational equivalent of the Shack–Hartmann wavefront sensor (SHWS), can achieve near diffraction-limited performance in FF-SS OCT.¹¹⁶ The advantage of this method is that, unlike other optimization-based techniques, it is non-iterative in nature and it does not require a priori knowledge of any system parameters such as wavelength, focal length, NA, or detector pixel size. This method can be also extended to region of interest (ROI)-based aberration correction to achieve diffraction-limited lateral resolution beyond the isoplanatic patch in high-resolution point scanning OCT with high NA. A computational sensorless adaptive optics strategy including OCT could play a critical role in correcting aberrations over large volumes and enabling rapid random-access multiplane imaging without highly sophisticated precompensation, SHWS, or electron-multiplying charge-coupled device.

AI-supported denoising and deconvolution of images to increase sharpness, resolution, or brightness are currently implemented at some microscopy platforms. However, it is often limited to a single modality and leaves the user patching different AI systems together for various use scenarios. At the moment, each imaging method needs a specific deep learning architecture with little similarities between each architecture. Moreover, to achieve a good level of validity, one needs a high number of training data to train an AI. One main problem for achieving the needed quality for detection and pixel-based quantification through deep learning lies in the lack of quality in ground truth data and the lack of a ground truth management system. By invoking a novel ground truth management system, an efficient AI system can be trained using only a few training data sets. Once the AI has been trained for one modality, the gained “experience” can be used to significantly speed up the training for other modalities through new transfer learning techniques, i.e., passing on experience. Active learning and knowledge transfer can improve learning speed and accuracy. This will apply to resolution enhancement as well as automatic annotations for ROIs. Therefore, in a unified approach in which the AI system supports OCT and MPM, radically improved learning speed and accuracy can be provided to establish novel multiparametric biomarkers. By developing and deploying novel beam shaping concepts, supported by advanced AI algorithms in combination with real-time data processing, critical parameters for the imaging performance in terms of penetration depth, acquisition speed, and spatial resolution are vastly improved, which could pave the way toward a compact multimodal biophotonics platform for advanced real-time 3D intrasurgical morphological, metabolic, and molecular imaging platform with increased sensitivity and specificity.

3.2.

Photoacoustic Microscopy and Tomography: Absorption, Molecular Contrast, and Seeing Deeper

OCT as an optical imaging modality that relies on the optical scattering properties of samples has its inherent limitations in terms of contrast generation. The strong scattering of many biological tissues further restricts the penetration depth of OCT due to its dependence on ballistic photon detection. To alleviate these problems, combining OCT with PAI has been explored by various groups over the past decade. Our previous review paper covered this topic for scientific works up to 2014.¹¹⁷ This perspective focuses on an update on dual modality OCT/PAI system configurations published recently as well as an outlook focusing on their application in molecular imaging.

3.2.3.

Outlook of OCT-PAI

A major technological limitation of current OCT-PAI systems is the imaging speed mismatch between the two modalities.^1,148 Although modern OCT technology permits video rate imaging, PAI systems, especially when PAT is applied, are much slower. Recent advances in high-speed PAI have pointed the way for novel real-time OCT-PAT systems. First, as the speed of PAI has been largely confined by the limited repetition rate of current pulsed laser sources, high repetition rate light sources such as pulsed laser diodes¹⁸¹ and light emitting diodes^182,183 have been applied for in vivo PAI. However, the low fluence from these alternative sources has resulted in imaging results with a poor contrast, which could be potentially improved by emerging methods such as deep learning^184,185 (Fig. 4).

Fig. 4

Real time PAI: (a) cross-sectional images of the lower thoracic cavity, (b) two lobes of the liver, (c) upper abdominal cavity, and (d) lower abdominal cavity of a nude mouse at a frame rate of 50 Hz. AA, abdominal aorta; BM, backbone muscles; CM, caecum; IN, intestines; IVC, inferior vena cava; LK, left kidney; LL, left lung; LLV, left lobe of liver; LV, liver; PV, portal vein; RK, right kidney; RLV, right lobe of liver; SC, spinal cord; SP, spleen; SV, splenic vein; TA, thoracic aorta; and VE, vertebra. Reproduced from Ref. 154 with permission from Springer Nature. Time-lapse 3D images of a freely swimming zebrafish recorded at an imaging rate of $100\text{volumes}/\mathrm{s}$. The shown sequences correspond to two portions of the movie having (e) smooth and (f) abrupt movements. Scale bar is 1 mm. Adapted from Ref. 188 with permission. Wide-field imaging of mouse brain hemoglobin responses to front paw stimulations at 2 kHz frame rate using a single transducer and an ergodic acoustic relay. (g) Calibration image of mouse brain vasculature through an intact skull. (h) Wide-field image. Fractional changes in signal amplitude (shown in red) in response to (i) right paw stimulation and (j) left paw stimulation superimposed on the calibration image. Norm., normalized and r.m.s., root mean square. Adapted from Ref. 186 with permission from Springer Nature.

Another way toward real-time OCT-PAI systems is applying multiple transducers and parallel data acquisition for PAI, which has demonstrated 2D PAI for large objects^154,187 and 3D PAI for a smaller FOV.¹⁸⁸ However, currently most photoacoustic detectors are conventional piezoelectric transducers, which are opaque to OCT detection beams. Therefore, all optical US detectors, due to their optical transparency, are especially interesting for OCT-PAI systems. Using a planar Fabry–Perot etalon, a multibeam scanner for simultaneous interrogation of multiple points was demonstrated.¹⁸⁹ This parallel detection reduces the 3D PAI time to a few seconds. Increasing photoacoustic interrogation beams will further increase imaging speed.

However, the parallel detection also greatly increases system costs. A new trend in high-speed PAI is to reduce acoustic detection points and adopt novel algorithms for image reconstruction using the obtained sparse data. These methods can be classified as iterative reconstruction algorithms^190–193 and deep learning methods.¹⁹⁴ For the former, prior knowledge of photoacoustic images, such as smoothness, sparsity, or total variation constraints, are added in the iterative algorithms. For the latter, a priori training is necessary to reconstruct high-quality images from sparse data.

Furthermore, in the latest breakthrough to speed up PAI, the number of required transducers for a large area 3D imaging was reduced to one single detector.¹⁸⁶ To do this, an acoustic relay cavity is placed between the imaged object and the detector. The propagation of photoacoustic waves in this cavity creates unique stretched acoustic pulse signatures for each point in the FOV. Therefore, upon full field illumination, the signal of the single detector is the combination of stretched pulses from the whole FOV; thus the optical absorption at each point can be unmixed based on the acoustic pulse signature. Although more research is needed for this method, together with all progress in achieving high-speed PAI, these breakthroughs have paved the way to future real-time OCT-PAI technologies.

3.3.

Multimodal Endoscopic OCT: More Comprehensive Access to Internal Body Organs

In addition to its success story in ophthalmology, OCT can also provide exquisite cross-sectional morphological information of organs that are not easily accessible, such as coronary arteries, intestines, or the brain. OCT imaging penetration suffers from light attenuation in tissue, especially due to high scattering at the near-infrared wavelengths (800 to 1300 nm). Penetration depths of $\sim 0.5$ to 2 mm prevent OCT from acting as a full-body imaging modality but enable tissue information up to comparable depths of those accomplished with conventional biopsies. In the past 30 years, research and industry have focused on the development of optical probes to endoscopically access internal organs with OCT. Two major approaches have been realized: sideward and forward imaging devices.¹⁹⁵ Luminal organs, such as vessels, airways, or the esophagus, can be imaged by sideward viewing probes, realized with scanning mechanisms based on micromotor-based distal rotation of a reflector¹⁹⁶ or proximal scanning of rotary joints.¹⁹⁷ Larger hollow organs, such as the urinary bladder, stomach, or cervix, are accessible via a forward viewing probe placed in front of the ROI. Beam scanning is achieved using microelectromechanical systems (MEMS), such as piezoelectric actuators^198,199 or mirrors.^200,201 Also other forward scanning schemes have been reported on paired GRIN lenses²⁰² or optical fiber bundles,²⁰³ although fiber bundles have not yet shown sufficient OCT performance and reduce probe bending flexibility for proper clinical applications.

During the development process of such probes, general optical and mechanical requirements have to be met: overall mechanical diameter, taking safety measures into account for electrical isolation, electromagnetic shielding, bending protection, and sealing; the necessary rigid distal length of the probe to fit clinical instrumentation, which has often insertion angles on the proximal end; or a careful micro-optical design to reach best optical performance. For the lateral resolution, approaches have been proposed to access development parameters already in the biomarker identification phase using microscope setups.²⁰⁴ Additional obligatory clinical requirements regarding sterilizability or biocompatibility must be taken into account. Furthermore, orientation is key for performing biopsies at a location identified with endoscopic OCT for proofing diagnosis or resecting identified malignant lesions. A proposed approach in esophageal endoscopic OCT is using laser landmarks.^205,206

Not only is the change toward a combined diagnostic and therapeutic tool of high interest, but also new approaches in probe design and development have been fostered recently. Concepts based on diffractive lenses were reported by Xi et al.²⁰⁷ Although the construction size is still considerably large, diffractive lenses are overcoming the compromised optical performance if OCT is implemented in endoscopic probes. Improved image performance was demonstrated by Pahlevaninezhad et al. in 2018 using metalenses for developing nano-optic endoscopes. These specially designed metalenses were shown to precisely control the light phase, thus reducing spherical aberrations and astigmatism. Therefore, increased depth of focus in parallel to high resolving power is achieved. This technique could furthermore be beneficial for other endoscopic imaging modalities.⁷¹ As 3D printing currently is available for metal and glass material, the development and research are going toward 3D printed glass surfaces with optical quality. This enables freeform optics manufacturing of optical components and even multilens objectives.¹⁵ Implementing this technology in optical endoscopic probes allows for direct printing on optical fiber facets with freeform optical elements, such as a freeform total internal reflection mirror for sideward imaging endoscopic OCT.²⁰⁸ In addition, microstructuring concepts will most likely lead to implemented anti-reflection behavior of the optical elements and tailored optical properties using different photoresists to customize optical properties at a micrometer scale (Fig. 5).

Fig. 5

Representative steps toward the development of fast data acquisition, 3D printed optical free-forms and representation easy to follow in clinical routine processes. (I) Zhang et al.³² reported on sideward imaging e-OCT and OCTA of a swine esophagus at an A-scan rate of 2.4 MHz. The distance covered by pullback was 14 mm, and a B-scan rate of 600 Hz was achieved. Arrows in (a) indicate blood vessels in OCT enface image, dashed lines in (b) indicate the depth of (c)–(f). This remarkable performance of OCT and OCTA at MHz speed is paving the way toward multi-megahertz endoscopically acquired data of OCT and OCTA. Adapted from Ref. 32. (II) Li et al.²⁰⁸ have realized 3D-printed free-form optic directly spliced on a light delivering fiber for improving optical performance (TIR, total internal reflection). Not only are very thin optical assemblies possible with this technique, but also the combination of various complementary techniques might be enabled with tailored optical properties to achieve outstanding imaging quality. Adapted from Ref. 208. (III) Alfonso-Garcia et al.²⁰⁹ introduced an augmented visualization of FLIM delineating brain tissue from prenecrotic and necrotic tissue. Left: Magnetic resonance image with indications of planes 1 and 2 of the FLIM maps on the right. Extrapolating this approach toward information feedback to clinicians, the color-coded maps, based on one single or multiple imaging techniques, could be easily interpreted and used by clinicians for immediate medical intervention. Adapted from Ref. 209.

Endoscopic OCT on its own has great potential in providing cross-sectional morphological information. Nevertheless, it lacks molecular or metabolic tissue contrast. Intramodal multimodal imaging using Doppler OCT, polarization sensitive OCT (PS-OCT), OCE, or spectroscopic OCT may provide additional contrast, but its clinical impact is still unclear.²¹⁰ As an additional extension, endoscopic OCTA for perfusion and angiogenetic contrast is of increasing interest as well. First realized in sideward viewing probes,²¹¹ forward endoscopic OCTA has recently been successfully applied,^212,213 demonstrating promising advantages compared with narrow band imaging with respect to 3D visualization and increased depth.²¹⁴ Increasing clinical endoscopic OCTA usability will require even further stabilization, robust acquisition modes, and image co-registration.^215–217 Imaging speed is certainly an important factor for its easy adaption. MHz OCT was reported for intravascular imaging to overcome heartbeat artifacts during imaging.²¹⁸ Multi-MHz OCT/OCTA was reported recently and enables video rate OCT/OCTA at impressive imaging performance.^32,218 In the near future, multimodal endoscopic OCT will demonstrate significant potential to unleash the full capability of accessing complementary morpho-functional and/or morpho-molecular tissue information needed for improved clinical diagnosis and therapy monitoring. For example, reports on a probe combining OCT and fluorescence imaging²¹⁹ providing additional molecular contrast and on a probe combining OCT, PAI, and US²²⁰ have been published. There are still challenges to be solved for a multimodal endoscopic OCT combination: the optic design needs to combine and find the best solution to fulfill the various optical requirements coming from the different image technologies in the scope of limited physical space within an endoscope. The acquisition times for the different modalities differ and are limited by biological constraints such as peristaltic movement, heartbeat, or breathing. Co-registered information is critical to retrieving the correct clinical information, especially if FOVs of the combined techniques are different. Research has been conducted toward the full cross-link of OCT and RS, despite remaining challenges.²²¹ To investigate clinical validity, large clinical studies, preferably multicentral studies, are absolutely necessary. Toward real-time displaying of the relevant information, the complementary techniques should be processed and visualized in a way to permit in situ clinical understanding and finally diagnostic decision. Therefore, each single modality needs to be optimized for detecting/characterizing the disease.²²² Data analysis speed and robustness have to be optimized, enabled by the full capacity of classification algorithms dealing with multivariant analysis. In the near future, computer science involvement will be the major topic for bringing endoscopic OCT (with or without other imaging technologies) to the patient bedside and into daily clinical practice. Techniques such as unsupervised classification for biomarker identification may be promising approaches. Morpho-molecular augmented painting, as presented by Alfonso-Garcia et al.,²⁰⁹ with autofluorescence lifetime imaging during neurosurgery, would be an intuitive real-time information display for guiding clinicians to detect the malignant tissue. Finally, this technique would lead to an augmented map for resecting malignant tissue inside human organs based on morphological, molecular, and functional contrast.

From a clinical point of view, the use of the described imaging devices—capable of multimodal endoscopic OCT—should preferably be applicable in an outpatient department setting, where typically no stationary stay of the patient is required, no general anesthesia is necessary, and immediate diagnostic information is needed. To achieve this, further miniaturization and increased usability with minimum patient discomfort are key. A quantum leap in diagnostic imaging of the gastrointestinal (GI) tract, for instance, may be swallowable low-cost imaging units transmitting reports of the GI tract to the patient’s cell phone or even to a centralized data analysis facility. Further development may go toward personalized medicine available at home to everybody for various internal organ diagnostics.

4.1.

Optical Coherence Angiography

OCTA is a label-free non-invasive OCT extension that uses blood cell motion contrast for high-resolution imaging of volumetric blood flow information generating angiographic images—hence providing both structural and functional (i.e., blood flow/perfusion) tissue information. Such angiographic maps in 3D have already been demonstrated by FD OCT-based Doppler OCT by several groups.²³⁴ However, the visibility of small capillary vessels was critically improved by comparing signals of adjacent B-scans rather than A-scans.²³⁸ Instead of quantifying the correlation between signals, OCTA compares the decorrelation signal between sequential OCT B-scans taken at the same cross-sectional location to construct a map of blood flow. Emerging from Doppler OCT, between 2004 and 2012, at least 10 different research groups published different versions of OCTA, the majority of them claiming its invention and producing new acronyms for OCTA.^237,239,240 The majority of the 4000 publications are in the field of ophthalmic diagnosis.^241–243 However, OCTA has also been successfully demonstrated for detecting angiogenetic biomarkers in cancer diagnosis and therapy monitoring as well as in endoscopic applications.^244,245 The success in ophthalmic applications and in clinical translation of this technique lies in its technological simplicity, moderate additional engineering as compared with conventional OCT systems, and extremely significant clinical impact—slowly replacing fluorescein angiography and indocyanine green angiography in clinical routine. To eliminate patient or organ movement induced artifacts, OCTA requires higher imaging speeds than most currently available OCT systems. It is noteworthy that OCTA provides 3D qualitative flow information at a fixed point in time. Therefore, vessel leakage is not detectable by OCTA. Furthermore, exact automated segmentation of all diagnostically important intraretinal layers is of essence to avoid artifacts in the OCT angiograms of the respective layers. Consequently, exact segmentation necessitates sufficient system sensitivity, axial resolution, and contrast. Retinal blood flow on OCTA can be obscured by hemorrhage as this decreases the ability of light to penetrate into the deeper layers of the eye. Despite the rapid, tremendous commercial and clinical success of OCTA, some (at least relative) blood flow quantification will be needed in the near future.^246–249 Improved and reproducible quantitative OCTA is definitely also of significant clinical interest as is the correct visualization and quantification of choriocapillaris^247,250,251 (Fig. 6).

Fig. 6

(a) Wide-field OCTA of the human retina: posterior pole montage of $12\times 12\mathrm{mm}$ (about 40 deg) swept source OCTA. Adapted from Ref. 252. (b) Averaged OCTA image of the choriocapillaris. Adapted from Ref. 253. (c) OCTA VISTA image: red indicates faster blood flow speeds and blue indicates slower speeds. Enlargements to the right, with two different mean projection times of 1.5 and 3 ms as well as the OCTA VISTA version, respectively. Notice that the capillary loops, which likely correspond to microaneurysms, are associated with slower blood flow speeds. Adapted from Ref. 254.

Another important future clinical role enabled by ultrahigh speed swept sources will be wide-field OCTA for the detection of neovascularization of the disc and elsewhere, microaneurysms, changes of the foveal avascular zone, intraretinal microvascular abnormalities, and capillary non-perfusion.^255,256 This advancement of OCTA technology in clinical research will ultimately lead to enhancement of individualized management of diabetic retinopathy and prevention of visual impairment in patients with diabetes.

Using the eye and especially the retina as a part of the central nervous system diagnostically as a window to the brain started in the late 1970s, and about 400 papers since then have covered diagnostic methods in the posterior pole of the eye for early diagnosis of brain diseases. An important prerequisite for successful OCTA-based diagnosis in neurodegenerative diseases and other clinical applications will be accurate and reproducible quantitative OCTA. Quantitative analysis of OCTA is essential to standardize objective interpretations of clinical outcomes.²⁵⁷ Indeed, a concerted effort has been put forth to understand how Alzheimer’s disease (AD) pathology may manifest in the retina as a means to assess the state of the AD brain.^258–260 OCTA has also been successfully evaluated as a tool to assess retinal changes in Parkinson’s disease²⁶¹ and both schizophrenia and bipolar disorder.²⁶²

4.2.

Optophysiology/Optoretinography: Non-Invasive Detection of Intrinsic Optical Signals

Modern medical diagnosis significantly benefits from extracting functional tissue information from structural imaging data (“structure–function correlation”). This is especially important in organs that cannot be biopsied, like the human retina. Retinal function has long been studied with psychophysical methods in humans, e.g., with electrophysiology and electroretinograms.²⁶³ Non-contact, depth-resolved, optical probing of retinal response to visual stimulation was introduced as optophysiology—an optical analog to electrophysiology. This method takes advantage of the fact that physiological changes in dark-adapted retinas caused by light stimulation can result in local variation of the tissue reflectivity.²⁶⁴ At that time, optophysiology could only be demonstrated in isolated rabbit retinas. Ophthalmic OCT technology back then was not sufficiently fast at longer wavelengths performing at sufficiently high sensitivity and resolution to be successfully applied in living animals or humans.²⁶⁵ A decade later, light-driven signals of photoreceptors in vivo could be measured. Visible light stimulation over a 200-fold intensity range caused correlated rod outer segment (OS) elongation and increased light scattering in wild-type mice, but not in mice lacking the rod G-protein alpha subunit, transducin (Gα(t)), revealing these responses to be triggered by phototransduction.²⁶⁶ The diurnal variation in rod OS length in mice was measured using optophysiology, being consistent with prior histological investigations demonstrating that rodent rod discs are phagocytosed by the RPE maximally over several hours around the time of normal light onset. The rate of recovery of rod OSs to baseline length before normal light onset was consistent with the hypothesis that disc membrane synthesis is fairly constant over the diurnal cycle²⁶⁷ (Fig. 7).

Fig. 7

(a) Illumination pattern (three bars) drawn to scale over the LSO image. The spatial map of OPL change between the ISOS and COST (b) before and (c) after stimulus measured at 20 Hz volume rate. (d) Rectangles over an LSO image represent the areas over which averages were obtained to plot the ORGs: 0.27°2 (yellow), 0.20°2 (gray), 0.14°2 (red), and 0.07°2 encompassing $\approx 10$ cones (violet). (e) Repeatability of the response: single ORGs (gray dashed) and their mean (solid black) for six repeat trials, in which phase responses were averaged over 0.27°2 for 17.9% bleach. (f) Spatial averaging: ORGs over different areas color-coded according to the rectangles in (d). (g) Maximum intensity projection at COST layer with AO-OCT reveals individual cone photoreceptors. (h) ORGs for a subset of single cones in (g) demonstrating the response in each cone for 0.3% S-cone bleach and 29.7% average L- and M-cone bleach. The magnified view near stimulus onset shows a negligible early response in putative S-cones (blue) compared with L/M cones (orange). (i) ORG early and late response amplitudes for each cone in (g). (j) Histogram of the ORG early and late response magnitude, computed as the Euclidean distance from origin of each data point in (i). The two-component Gaussian mixture model (black dotted line) and its component Gaussians are used to distinguish S-cones (blue fit) from L/M cones (orange fit). The vertical dotted line marks $t=0$ in (e), (f), and (h) indicating the rising edge of stimulus onset. (a)–(f) are obtained without AO, with 4-mm imaging pupil, at 120-Hz volume rate. (g), (h) are obtained with AO, for 6-mm imaging pupil, at 162-Hz volume rate. The stimulus wavelength for all plots is $528\pm 20\mathrm{nm}$. (a)–(j) Adapted from Ref. 268.

Fast intrinsic optical signal (IOS), which arises before light-evoked pupillary response, promises to be a unique biomarker for photoreceptor physiology for objective optoretinography with high resolution. In another study, depth-resolved optophysiology verified OS as the anatomic origin of fast photoreceptor-IOS. Dynamic IOS changes were primarily confined at OS boundaries connected with inner segment and RPE, supporting transient OS shrinkage due to phototransduction process as the mechanism of the fast photoreceptor-IOS response.²⁶⁹ Non-invasive, objective measurement of light-evoked, functional responses of human rods and cones, measured non-invasively using a synchronized adaptive optics OCT and scanning light ophthalmoscopy system have also been reported recently.²⁷⁰ Another recent study revealed that the onset of phototransduction is accompanied by a rapid ($<5\mathrm{ms}$), nanometer-scale electromechanical deformation in individual human cone photoreceptors. Characterizing this biophysical phenomenon associated with phototransduction in vivo was enabled by high-speed phase-resolved optical LFOCT that allowed for sufficient spatiotemporal resolution to visualize the nanometer/millisecond-scale light-induced shape change in photoreceptors.²⁶⁸

4.3.

Visible Light OCT: Unprecedented Axial Resolution and Enhanced OCT Access to Absorption

OCT in the visible wavelength range with unprecedented submicrometer axial resolution achieved by employing a photonic crystal fiber in combination with a sub-15 fs Titanium:sapphire laser was first demonstrated in the beginning of this millennium.^271,272 Visible light OCT theoretically provides higher axial resolution than NIR OCT for a given wavelength and bandwidth. To realize this potential in the human retina in vivo, the unique technical challenges of visible light OCT must be addressed: incorporating a grating light valve spatial light modulator spectral shaping stage to modify the source spectrum; developing a novel, Fourier transform-free, software axial motion tracking algorithm with fast, magnetically actuated stage to maintain near-optimal axial resolution and sensitivity in the presence of eye motion; and implementing spatially dependent numerical dispersion compensation for the first time in the human eye in vivo. Wavelength-dependent images of the outer retina suggest that, beyond merely improving the axial resolution, shorter wavelength visible light may also provide unique advantages for visualizing Bruch’s membrane (BM).²⁷³ Furthermore, it seems that shorter visible wavelengths improve the visualization of BM in pigmented eyes, where it is located behind a highly scattering layer of melanosomes in the RPE. Monte Carlo simulations of radiative transport suggest that, while absorption and scattering are higher at shorter wavelengths, detected multiply scattered light from the RPE is preferentially attenuated relative to detected backscattered light from BM.²⁷⁴

Using visible light OCT, accurate and robust non-invasive measurement of retinal oxygen metabolic rate (${\mathrm{rMRO}}_2$) in rat eyes was demonstrated. Both oxygen delivery and ${\mathrm{rMRO}}_2$ increased from the highly regulated retinal circulation under hypoxia. The increased oxygen extraction compensated for the deficient oxygen supply from the poorly regulated choroidal circulation. These results have the potential to reveal the fundamental role of oxygen metabolism in various retinal diseases such as age-related macular degeneration (AMD), diabetic retinopathy, and glaucoma.²⁷⁵ Oximetry saturation ${\mathrm{sO}}_2$ measurements were also recently extended to capillaries and investigated in all three retinal vascular plexuses by amplifying and extracting the spectroscopic signal from each capillary segment under the guidance of OCTA. Using this approach, capillary ${\mathrm{sO}}_2$ in the retinal circulation in rats was measured.²⁷⁶

In vivo depth resolved human imaging of oxygenation in retinal capillaries was recently demonstrated.²⁷⁷ In addition, first visible light OCTA with a 100-kHz A-line rate for human retinal imaging was also demonstrated recently, enabling accurate localization of microvasculature down to the capillary level and thus enabling oximetry at vessels $<100\mu \mathrm{m}$ in diameter. Microvascular hemoglobin oxygen saturation (${\mathrm{sO}}_2$) at the feeding and draining vessels at the perifoveal region was demonstrated, allowing for future studies on the role of microvascular oxygen in various retinal pathologies²⁷⁸ (Fig. 8).

Fig. 8

(a)–(b) In vivo human retinal visible light OCT; (b) inset from (a). Adapted from Ref. 274. (c) B-scan image of a brown Norway rat retina using visible-light OCT. NFL, nerve fibre layer; GCL, ganglion cell layer; IPL, inner plexiform layer; INL, inner nuclear layer; OPL, outer plexiform layer; and BM, Bruch’s membrane. (d)–(f) En face images of vascular/capillary plexuses. SVP projected in the NFL and GCL slabs. ICP projected in the slab containing the inner border of the INL. DCP projected in the slab containing the outer border of the INL. (g) En face structural image projected from the ILM to BM, overlaid with measured oxygen saturation (${\mathrm{sO}}_2$) values in major vessels to differentiate arteries from veins in an animal breathing 100% ${\mathrm{O}}_2$. Interplexus capillaries (white arrows) appear as dark spots due to greater light absorption than neighbouring capillaries. (h) Overlaid en face angiograms of three vascular/capillary plexuses to demonstrate the detailed organization of the retinal circulation. Examples of interplexus capillaries (indicated by white arrows in the enlarged images) were validated by observing their presence in corresponding locations. (i) En face projection of the NFL slab. The SVP was found to run anterior to the nerve fibre bundles (bright radial striations), which appear posterior to the vessels. The interplexus capillaries (black arrows) penetrate between NFL bundles and connect the SVP to the ICP and DCP. (c)–(i) Adapted from Ref. 276.

Another technique enabling OCT access to absorption is photothermal OCT, which monitors changes in the optical path length caused by the photothermal effect. Like in PAI in the photothermal effect, photons are absorbed by chromophores within a sample, causing a localized rise in temperature. Thermoelastic expansion caused by the photothermal effect generates isolated variations in the refractive index, which in turn generates a variation in the optical path length. The optical path length variations typically generated by the photothermal effect, often nanometer in scale, can be resolved with phase-sensitive OCT.²⁷⁹ This approach might be extremely attractive for non-destructive 3D molecular imaging deep ($\sim 2\mathrm{mm}$) within biological samples with a sensitivity of 14 parts per million (weight/weight) nanoparticles in the sample.²⁸⁰ In addition, photothermal OCT could quantify changes in pigmentation that occur in retinal diseases.^281,282 In vivo photothermal OCT was also demonstrated for cross-sectional in human skin measurement with endogenous absorption agents.²⁸³ Recently, a swept source dual-wavelength photothermal OCT system was demonstrated for quantitative imaging of microvasculature oxygen saturation and measuring of microvasculature ${\mathrm{sO}}_2$ levels in phantom blood vessels with a range of blood flow speeds (0 to $17\mathrm{mm}/\mathrm{s}$).²⁸⁴

4.4.

Dynamic Contrast OCT

Recently, an extremely powerful new contrast enhancing OCT extension was introduced. It employs a new endogenous approach to reveal subcellular metabolic contrast in fresh ex vivo tissues, taking advantage of the time dependence of FF OCT interferometric signals. This method reveals signals linked with local activity of the endogenous scattering elements that can reveal cells where other OCT-based techniques fail or need exogenous contrast agents. Using high-transverse resolution to image intracellular features, the time dependence to identify different dynamics at the millisecond scale on a wide range of organs in normal or pathological conditions is used.²⁸⁵ Dynamic contrast OCT has also been applied in freshly excised human esophageal and cervical biopsy samples. Depth-resolved dynamic contrast OCT images of intact tissue show that intracellular dynamics provides a new contrast mechanism that highlights subcellular morphology and activity in epithelial surface maturation patterns²⁸⁶ (Fig. 9).

Fig. 9

(a) LightCT en face OCT image of murine liver. (b) LightCT dynamic OCT en face image with corresponding regions. (c) Averaged mOCT en face image, which was acquired in the same region. (d) In the corresponding dynamic mOCT image, hepatocytes become visible with nuclei. (e) Cropped volume representation; size: $270⇥285⇥135\mu \mathrm{m}$ $(xyz)$; scale bar, $100\mu \mathrm{m}$. (a)–(e) Adapted from Ref. 287. (f) 3D reconstruction of a D-FFOCT image stack in explanted macaque retina over a $120\times 120\mu \mathrm{m}$ FOV. Note that FFOCT signal is damped with increasing penetration depth, so upper retinal layers are more clearly visible than lower ones. En face images of (g) inner nuclear layer, (h) outer nuclear layer, and (i) photoreceptor layer presenting a similar appearance to two-photon fluorescence imaging; (j) reconstructed cross section at the location represented by the red dotted line in (f). The cross section in (j) was linearly interpolated to obtain a unitary pixel size ratio. (f) D-FFOCT image of a porcine retinal pigment epithelium cell culture. (g) Overlay of colored D-FFOCT and FFOCT at the interface between the layers of the nerve fibers (white arrows point to nerve bundles that are very bright in static mode and invisible in dynamic mode), ganglion cells (blue and green cells, visible in dynamic mode) and inner plexiform (fibrous network, bottom left, visible in static mode). (f)–(l) Adapted from Ref. 288. (m) HE stained histology of the imaged sample at a different location: (I) cornified layer, (II) granular and spinous layers, (III) basal layer, (IV) lamina propria, (V) muscle, and (VI) glass plate. (n) OCT image of mouse tongue; lamina propria (IV) can be identified by brighter contrast. (o) Corresponding dynamic contrast mOCT image with a focus in the basal cell layer; (I)–(V) and even cell nuclei (*) are visible. (p) Dynamic contrast mOCT image with a focus in the lamina propria; the image size is $380⇥500\mu \mathrm{m}$ ($zx$); scale bar is $100\mu \mathrm{m}$. (m)–(p) Adapted from Ref. 287.

This technique has also been used to combine static, dynamic, and fluorescence contrasts to achieve label-free high-resolution imaging of the living retina and anterior eye with temporal resolution from milliseconds to several hours, allowing for probing biological activity at subcellular scales inside 3D bulk tissue.²⁸⁸ By evaluating signal fluctuations, a significant contrast enhancement was demonstrated using TD FFOCT, which makes cellular and subcellular structures visible. The putative cause of the dynamic OCT signal is the site-dependent active motion of cellular structures in the submicrometer range, which provides histology-like contrast.²⁸⁷ In vivo dynamic contrast OCT was used to quantify layer-resolved microvascular blood flow and volume across the full depth of the mouse neocortex.²⁸⁹

Finally, speckle variations induced by intracellular motion in the urothelium were used as a dynamic contrast to segment the urothelium with only two sequential OCT images. This new method opens the possibility of tracking the distribution of the urothelial cells to identify the microinvasion of bladder tumors. This contrast may provide a new mechanism for OCT to diagnose the invasion of urothelial cancerous cells for the better staging of bladder cancer.²⁹⁰

Another interesting contrast enhancing OCT extension that gained a lot of attention recently is using the extraction of the optical attenuation coefficient, an important tissue parameter that measures how quickly incident light is attenuated when passing through a medium. Successful extraction of this parameter would facilitate tissue differentiation and enhance the diagnostic value of OCT. With current studies showing attenuation coefficient analysis as a promising technique, further efforts in the development of methods to accurately extract the attenuation coefficient and to explore its potential use for more extensive clinical applications are desired.²⁹¹

4.5.

Optical Coherence Elastography: Depth Resolved Optical Palpation

An extremely promising extension of OCT is OCE. The original idea of elastography is not only to exploit the stiffness changes within soft tissue for diagnosis, much like a physician would during palpation, but also to exactly localize and quantify them. The capability of OCT to give 3D insight into tissue is used to visualize the reaction of a sample to a mechanical force. Multiple methods for deriving this reaction from the OCT data have been presented, e.g., image cross correlation,²⁹² feature²⁹³ or speckle tracking,²⁹⁴ optical flow,²⁹⁵ combinations²⁹⁶ thereof, and phase difference analysis.²⁹⁷ From this reaction, the underlying mechanical tissue parameters can be obtained and used for diagnosis. The technology first emerged in US imaging, where systems capable of elastography have been commercially available since the 2000s. Elastography in general is a relative straight forward add-on to OCT, enabling biomechanical contrast at a slightly increased imaging time and data processing complexity. Although a lot of promising approaches and methods utilizing OCE have been published, two major applications are at the moment competing for clinical in vivo diagnosis (Fig. 10).

Fig. 10

(a) Strain in untreated cornea induced by the heartbeat (orange: compression, blue: relaxation). Adapted from Ref. 307. (b) Qualitative OCE image of invasive ductal carcinoma. Adapted from Ref. 298 with permission from AACR. (c) Automated stiffness-based segmentation of mouse breast cancer in vivo. Adapted from Ref. 299.

In this sense, the ophthalmic application of OCE is the most advanced, likely because it could build upon decades of OCT experience in this field, which already addressed issues like motion correction and speed. The only novel aspect from an engineering perspective is mechanical loading, which has to be performed cautiously and preferably contactless. Proposed solutions rage from contact shaker excitation,³⁰⁰ over microscale³⁰¹ air-pulses,³⁰² and acoustic radiation pulses^303,304 to heartbeat driven tissue deformation.^301,305 De Stefano et al.³⁰⁶ demonstrated the feasibility of OCE for identification of keratoconic corneas in humans in vivo using a dynamic compressional approach with intensity-based speckle tracking. Since the compression is applied via a motorized stage, direct contact to the cornea is needed, but quantitative analysis also is enabled. In May 2020, Lan et al.³⁰² quantified the effects of respiration and heartbeat on patient eye motion as a source of perturbation for OCE in vivo and mentioned the idea that this perturbation could also be used as a passive loading method. Only a month later, Nair et al.³⁰⁵ gave another indication for the feasibility of non-contact heartbeat ophthalmic OCE in an ex vivo porcine eye model with controlled intraocular pressure changes. In January 2021, Nair et al.³⁰⁷ achieved heartbeat OCE in an in vivo rabbit model, but the method was not completely non-contact as a glass imaging window was used to minimize bulk motion. The heart pulse compressed the corneal tissue against this imaging window, resulting in the desired displacement. OCE data were acquired as multiple 2D B-scans at the same position, which were synchronized with the rabbit’s heartbeat later in postprocessing. The resulting 2D strain maps clearly show the difference between compression and relaxation induced by the pulse. In addition, they were able to differentiate between an untreated cornea and a cornea stiffened via the Dresden protocol.

Although these are already very promising first results, some challenges for clinical in vivo patient imaging still remain to be addressed. Since pulse waves travel at a limited speed, they affect different regions in the FOV at different points in time, which can lead to underestimation of the resulting strain and is a challenge for synchronization. For clinical diagnosis, en face images and 3D stiffness maps are much more useful than single B-scans. 3D imaging would not only highly increase imaging time but also make the synchronization even more complicated. One also has to be aware that, without the possibility of quantifying the pressure perceived by the investigated tissue, only qualitative OCE results can be expected and quantifying the pressure excited by a heartbeat at every point in the FOV seems impossible at this point in time. However, the potential of ophthalmic OCE for clinical application is huge. The feasibility for diagnosing keratoconus has been shown in the examples above, but also corneal dystrophy or corneal ectasia due to surgical complications are obvious topics in which OCE can deliver fast and uncomplicated diagnosis, as well as in the diagnosis of AMD in which early diagnosis is especially important, as presented by Qian et al.³⁰⁰

When it comes to the field of cancer detection and diagnosis, current literature focuses on biopsy images as proof of concept. Not only is cancer often accompanied by significant changes in stiffness, but also the shape of the outlines of cancer has diagnostic potential.³⁰⁸ A good example is given by Kennedy et al.,²⁹⁸ who showed that cancer margin detection with OCE yields a 24% higher sensitivity than with OCT only. Multiple tumor types have been investigated, and OCE proved to be an accurate predictor of malignancy for all of them. A phase-sensitive, quasi-static compression OCE system was used.³⁰⁹

The future of OCE lies in intraoperative in vivo imaging for determining tumor margins of cancerous tissue to be removed, requiring not only high-speed OCT imaging to minimize the influence of motion artifacts but also high-speed processing to enable real-time view. Depending on the chosen biomechanical model and processing approach, this task requires expensive hardware and well-versed software implementation. Another challenge to keep in mind is OCE’s vulnerability to laser and system instabilities, which are even more difficult to control in a surgical environment.

Nonetheless, surgical applications of OCE are close. Plekhanov et al.²⁹⁹ presented a phase-sensitive, quasi-static compressional OCE method that is capable of segmenting cancerous regions in a mouse model in vivo. They were able to differentiate between viable, dystrophic, and necrotic tumor cells as well as edema zones and validated their findings via comparison with histological results. A 9-day study, comparing the effect of two different chemotherapeutics with untreated control, was performed as well. In a similar study,³¹⁰ the combination of OCE with OCTA was shown to be an easy way to add functional information, improving the quality of chemotherapeutic treatment efficiency studies.

As can be seen in the aforementioned examples, quasi-static compression OCE, mostly phase-sensitive with some sort of compliance layer as stress sensor, is currently the most advanced OCE method. In the last few years, other approaches rose to the horizon of feasibility, e.g., vibrational OCE,^311,312 which uses OCT to record the reaction of a sample to a variation of vibration frequencies. With this, a 3D resonant frequency map of the sample can be generated and viscoelastic behavior can be investigated. Another proposed application, although more focused on usage in studies than for clinical diagnosis, is in intravascular OCE.^313,314 Since atherosclerosis is the leading cause of human death worldwide, continued effort is put into understanding the generation, buildup, and rupture of intravascular plaques.

Overall OCE has come far, and routine clinical usage especially in the ophthalmic field is to be expected in the next few years. The next steps to this goal are achieving faster imaging and data processing and even better motion correction. Although some OCE experts and clinicians argue that there is no need for quantifying tissue stiffness, as long as there is qualitative biomechanical contrast, in the long term it might make a lot of clinical sense to find a quantitative stiffness atlas, which allows for diagnostic guidance.

7.1.

Medical Device Regulations

Europe currently sees the transition from the medical device directives to the medical device regulation as a need to be implemented as national law by each member state. They provide a tight framework not only for medical device industries but also for academic institutions, affecting research and development of new technologies already at its very roots. Clearly, regulatory approval is essentially needed for safe applications on human beings. Academic entities are therefore now challenged to allocate sufficient resources and to build supporting structures to enable internationally competitive research to harmonize the high flexibility needed at early research stages with regulatory demands. Novel technologies need to be convincing enough for industry to pick them up for further development, which needs already substantive validation in a clinical setting with all of the required regulatory hurdles. Only with proper support can a widening of the “valley of depth” be avoided.

7.2.

Standardization

A translation of promising OCT technologies to market standardization is an important topic as discussed by Waterhouse et al.³⁶⁸ For example, the question of appropriate laser power levels for internal organ tissue is not yet covered by laser safety standards. The current assumption is that all internal organs respond similar to the laser exposure. However, already the tissue texture leaves the impression of major differences between the tissue types. With a general standardization on given performance parameters, for instance, the relation to an appropriate standard would be possible at an early stage of translational research. This aims furthermore at the repeatability and reproducibility within multicentral clinical trials. Assessment of performance parameters at an early development stage is difficult because little or no validated ex vivo models are present to benchmark performances of a newly developed device. Most of these arguments merge into the argument of cost-effectiveness during research and development. Having standardized benchmarking steps and milestones alongside the development processes in academia or industry may save resources. Translational research actually is facing more than one “valley of death:” the first one arises from translating laboratory systems and findings to the patient bedside and the second one occurs during the attempt to bring the gained knowledge into clinical practice and health decision-making procedures.³⁶⁹ With clear standards for benchmarking, there might be a higher chance for seeding product development and streamlining translational research to finally appear next to the patient.