Review of endomicroscopic imaging with coherent manipulation of light through an ultrathin probe

Jaeyeon Oh; Chunghyeong Lee; Gookho Song; Mooseok Jang

doi:10.1117/1.JOM.3.1.011004

9 January 2023 Review of endomicroscopic imaging with coherent manipulation of light through an ultrathin probe

Jaeyeon Oh, Chunghyeong Lee, Gookho Song, Mooseok Jang

Author Affiliations +

Journal of Optical Microsystems, Vol. 3, Issue 1, 011004 (January 2023). https://doi.org/10.1117/1.JOM.3.1.011004

Abstract

Endomicroscopy is a technique to visualize microscopic structures of internal tissues through tubular instruments that can be inserted through a small cut or an opening in the body. There has been a growing demand for miniaturizing endoscopic instruments while preserving a high resolution to achieve a real-time histopathologic diagnosis. Meanwhile, there has recently been tremendous progress in the coherent manipulation of light in which an optical wave is deterministically manipulated through a linear, disordered medium for optical focusing and imaging. Here, we review recent research efforts in developing new endomicroscopic imaging schemes based on the coherent formulation and manipulation of optical fibers. In contrast to the conventional schemes using optical fibers as incoherent channels for optical power, these approaches provide a route to fully exploiting useful information transmitted through an ultrathin probe, thereby potentially achieving practical endomicroscopic imaging through a submillimeter thick probe.

1. Introduction

Endoscopy is an indispensable tool for observing objects that cannot be accessed using conventional free-space imaging techniques. In modern medical practices, endoscopic imaging is routinely performed on the gastrointestinal tract, respiratory tract, urinary tract, and reproductive tract through natural orifices or small incisions to access the body cavity.¹ In the area of biomedical imaging, endoscopic imaging techniques have continuously evolved with the shared goal of achieving high-resolution and high-contrast imaging with a small-sized probe for a minimal footprint. There have been some notable technological advances, including a photographic film-based gastrocamera and a glass fiber-based fiberscope in the 1950s and 1960s and charge-coupled device (CCD)-based video endoscopy in the 1980s (see Fig. 1).²^–⁴ To date, video endoscopy has served as the standard diagnostic technique because of its high information content (i.e. high pixel count), real-time features, and compatibility with useful imaging modalities, such as narrow-band imaging and autofluorescence imaging.⁵ Nevertheless, the resolution of conventional video endoscopy³ is limited to the range of 10 to $100 μ m$ , and the size of the probe is typically larger than 1 mm. Therefore, a conventional endoscope can only be used for gross examination of internal tissues, and thus precise histopathologic diagnosis still requires additional biopsy sampling and careful examination under a conventional microscope.

Fig. 1

Overview of endoscopic imaging methods. A timeline and schematics of key technological milestones are presented. eCLE, endoscope-based confocal laser endomicroscopy; pCLE, probe-based confocal laser endomicroscopy; endoscopic OCT, endoscopic optical coherence tomography; CCD, charge-coupled device; and SLM, spatial light modulator.

For the past decades, researchers have explored ways to achieve a lateral resolution comparable to $1 μ m$ to enable visualization of cellular morphology through an endoscopic probe. Such techniques are termed “endomicroscopic” imaging or alternatively “microendoscopic” imaging.⁶^–⁹ Previous approaches can be classified into two categories depending on their focus scanning schemes: proximal scanning techniques and distal scanning techniques. Proximal scanning techniques [i.e., probe-based confocal laser endomicroscopy (pCLE)] use a multicore fiber (MCF), comprising $≳ 10, 000$ cores, in which the cores in effect serve as virtual confocal pinholes for a focused illumination and detection.⁸^,¹⁰ The scanning unit on the proximal side sequentially directs a focused beam to individual cores and a two-dimensional (2D) image is retrieved by stitching the optical signals acquired from each core. In contrast to the proximal scanning scheme, distal scanning techniques (i.e., endoscope-based CLE, eCLE) utilize a single-mode fiber (SMF), and focus scanning is performed on the distal side using a microelectromechanical systems scanning unit (e.g., piezoelectric actuator, electrothermal, electrostatic, and electromagnetic actuators) or a micromotor.⁸ Optical coherence tomography (OCT),¹¹ combined with distal scanning techniques, provides a three-dimensional (3D) volumetric imaging capability with an enhanced imaging depth by virtue of coherence gating.¹² However, the lateral resolution of endoscopic OCT is typically limited to tens of microns. The use of an endocytoscope is another notable approach that integrates high magnification and high numerical aperture (NA) objective lenses on the distal end to acquire a high-resolution image without any scanning. However, the depth sectioning capability of the endocytoscope is substantially worse than that of scanning-based approaches as it indiscriminately collects the scattered light from different tissue depths.¹³ Table 1 summarizes the key characteristics of previous representative endomicroscopic imaging systems.²⁰

Table 1

Comparison of existing endomicroscopic imaging techniques.

	Proximal scanning (pCLE)6,14–16	Distal scanning		Endocytoscopy17,18
	Proximal scanning (pCLE)6,14–16	Confocal (eCLE)14–16	Endoscopic OCT11,12,19	Endocytoscopy17,18
Resolution ( $μ m$ )	1 to 3.5	0.5 to 1.3	8 to 40	1.7 to 4.2
Depth sectioning capability	$O$ (5 to $26 μ m$ )	$O$ (4.2 to $11 μ m$ )	O (5 to $20 μ m$ )	$X$
Field of view ( $μ m$ )	240 to 600	100 to 475	2000 to 8000	120 to 700
Probe diameter (mm)	1 to 3	12.8	0.4 to 5	3.2 to 13.6
Frame rate (frames/s)	9 to 12	0.8 (1024 × 1024 pixels)	50 to 4000 (for $x - z$ section)	30
Advantage	Relatively thin probe and high scanning rate	Improved lateral resolution	3D volumetric Imaging	Nonscanning
Disadvantage	Pixelation issue (i.e., finite inter-core distance)	Frame rate limit due to scanning scheme	Lateral resolution limit	No depth sectioning capability

The previous endomicroscopic imaging approaches have commonly relied on incoherent image formation with the optical power of back-reflected or backscattered light being directly taken as image information. In this incoherent scheme, an optical fiber or an MCF is only used as a channel to transmit the optical power from one end to the other. Interestingly, optical fibers, as a linear optical material, can be thought of as a channel to transmit coherent information (i.e., both amplitude and phase information), and the high-content information can be fully utilized for endomicroscopic imaging and focusing purposes based on the coherent manipulation of light. The recent development in the coherent manipulation of light stems from an original aim of achieving a long-sought goal of optical imaging through a scattering medium. In 2007, it was shown that coherent light can be focused through a highly scattering medium by iteratively optimizing an incident wavefront through a feedback loop.²¹ In general, any linear optical medium, including a scattering medium and optical fibers, can be formulated as a transmission matrix with elements that describe a linear, coherent relation between input and output optical modes.²²^,²³ Based on this transmission matrix formalism and associated linear algebra operations, it has been demonstrated that an object behind a scattering medium can be optically retrieved.²⁴^–²⁸

In the present paper, we review recent research efforts in developing coherent light manipulation techniques for endomicroscopic imaging through an ultra-thin probe with a diameter that is $< 1 mm$ . These new approaches provide an ideal diffraction-limited resolution for a given optical aperture of a probe without any bulky distal optics, as well as an additional capability to freely set an imaging plane without any moving parts. Figure 1 presents an overview of various endoscopic imaging approaches and the advantages of the coherent approach in comparison with the conventional incoherent techniques—(1) reduced probe size, (2) improved resolution, and (3) flexibility in setting the imaging plane. In Secs. 2 and 3, we review the general properties of optical fibers and present an overview of the coherent formulation and associated linear algebra operations for linear optical media. Then, in Secs. 4 and 5, we review recent developments in exploiting the coherent nature of light through multimode fibers (MMFs) and MCFs, respectively, for endomicroscopic imaging, as well as their applications. In addition, in Sec. 6, we review some recent developments based on unique spatial and temporal properties of optical fibers that can potentially extend the usability and functionality of the techniques introduced in Secs. 4 and 5. Section 7 concludes the review with some remarks on the clinical practicality of coherent control of optical waves for endomicroscopic imaging.

2. General Properties of Optical Fibers

2.1.

Wave Propagation in Optical Fibers

Here, we review the basic properties of a step-index optical fiber, a flexible cylindrical waveguide consisting of two different dielectric materials. Step-index optical fibers are composed of an inner “core” part made of a higher refractive index material surrounded by an outer “cladding” part made of a lower refractive index material. In this structure, the light rays with an incidence angle smaller than the acceptance angle, defined as $\sin^{- 1} (\sqrt{n_{core}^{2} - n_{clad}^{2}}) = \sin^{- 1} {NA}_{acc}$ , are transported through the core nearly without any loss due to the total internal reflection at the interface of the two dielectric materials. $n_{core}$ and $n_{clad}$ are the refractive indices of the core and cladding materials, respectively, and ${NA}_{acc}$ is the NA of an optical fiber. Conventional endoscopes typically rely only on this lossless optical energy transport property (i.e., incoherent property) of optical fibers.²⁹

In the coherent manipulation of light transport through optical fibers, discrete optical modes (i.e., guided modes) play an important role in determining both spatial and temporal properties of the light transmitted through optical fibers. Considering the boundary conditions at the interface of the core and cladding materials, only optical waves with certain longitudinal wavevectors and associated transverse field patterns can propagate through step-index optical fibers without loss. Those features of the guided modes are typically calculated by solving Maxwell’s equation with a set of boundary conditions in a cylindrical coordinate. The guided modes may also be thought of as a collection of linearly polarized optical waves that satisfy the two following conditions at the same time: (1) the transverse resonant condition in which the round trip of the optical waves in the transverse direction through internal reflections results in phase delay of integer multiples of $2 π$ (i.e., constructive interference) and (2) the condition for total internal reflection (i.e., transverse wavevector $< k_{0} {NA}_{acc}$ , where $k_{0}$ is the vacuum wavenumber).

The number of discrete optical modes (i.e., supported modes) of a given optical fiber, $M$ , is estimated as²⁹

M \approx \frac{1}{2} V^{2},

where

V = a \cdot (\frac{2 π}{λ}) \cdot {NA}_{acc}

is the dimensionless parameter determined by the radius of the core

a

, the wavelength of light

λ

, and the NA of optical fiber

{NA}_{acc}

. Therefore, when the ratio of the core’s radius to the wavelength and the

{NA}_{acc}

increase, the number of supported modes increases quadratically. Depending on the number of supported modes, the step-index optical fiber can be classified into two types:²⁹ SMF, in which

V

is smaller than or comparable to 2.405, and MMF, in which

V

is larger than 2.405. In fact, a set of discrete guided modes constitutes an orthogonal basis (i.e., distinctive optical channels) for light transmission through optical fibers. Therefore, the number of modes available inside the waveguide is equal to the information transmission capacity of the fiber, which has a density (=

M

divided by the size of the entire probe) that determines the efficiency of the imaging system as it is fundamentally related to the spatial bandwidth product of a given imaging system [i.e., the number of resolvable spots in a field of view (FOV)]. Here we introduce three types of optical fibers as endoscope probes and describe how the image information on the distal end is delivered to the proximal end depending on the fiber type.

2.2.

Overview of Optical Fiber Types

2.2.1.

Single-mode fiber

The SMF transmits most of the optical energy through a single optical mode with a Gaussian-like transverse mode profile with a small core diameter of typically less than $10 μ m$ , a cladding diameter of 80 to $125 μ m$ , and a ${NA}_{acc}$ of about 0.1 [see Fig. 2(a)].³⁰ Therefore, when the light is coupled in or out to the SMFs, the light intensity should be highly confined around the central axis of the fiber with the mode field diameter comparable to the core size. Although SMF can only transmit image information of a single object point at once, this property of SMF can be used to implement a bidirectional pinhole for focused illumination and detection in a confocal imaging configuration. In fact, based on this principle, numerous types of eCLE and endoscopic OCT techniques have been proposed and used for clinical demonstration with a miniaturized beam scanner at the distal tip of the fiber for laser-scanning imaging⁸^,¹² (see Fig. 1). However, the miniaturized beam scanning unit has a relatively slow scanning speed compared with that of external scanning units used for pCLE, such as galvanometer mirror scanners and resonant scanners, and it often makes the size of the entire probe larger than a millimeter, albeit miniaturized.

Fig. 2

Comparison of three types of optical fibers as an endoscope probe. (a) SMF ( $V < 2.405$ ) has a small diameter and supports nearly a single guided mode and thus can only transport an optical field at one point on the distal end with a fixed amplitude and phase. (b) MCF consists of multiple cores, each of which permits nearly a single guided mode. Each core transports the optical fields from the corresponding points with a uniform transmittance, but with a random phase delay due to the fabrication imperfections. (c) MMF (usually $V ≫ 2.405$ ) supports multiple numbers of modes such that the incident light is decomposed into the different guided modes, and individual modes propagate with their corresponding propagation constants. The point spread function of an MMF appears as a speckle pattern—a random fluctuation in both amplitude and phase, and thus the image is significantly distorted on the other end.

2.2.2.

Multicore fiber

The MCF consists of $≳ 10, 000$ cores embedded in a common cladding material with an overall diameter of hundreds of microns. Each core size and acceptance NA are similar to those of the SMF; hence it is equivalent to an array of SMF with an inter-core distance of a few microns to ensure that the guided light is transported through each core without significant inter-core couplings. The MCF directly transports the intensity pattern from one end to the other with the spatial sampling rate of the inter-core distance; however, the phase information is randomized as slight length differences among cores cause different phase delays on each core [see Fig. 2(b)]. Using each core as a bidirectional pinhole, the pCLE technique achieves confocal imaging with a conventional beam scanning unit on the proximal side. In this process, light is successively focused into each core at the proximal end and the optical power emerging from an object point is coupled back into the corresponding core through a lens¹⁰^,¹⁴ (see Fig. 1). However, the finite inter-core distance for preventing crosstalk ( $> 3 μ m$ ) results in inherent pixelation artifacts and reduced lateral resolution [see Fig. 2(b)].³⁰

2.2.3.

Multimode fiber

The MMF has a diameter of the core ranging from tens to hundreds of micrometers that supports a large number of guided modes [see Fig. 2(c)]. For example, a typical MMF with a diameter of $105 μ m$ and a ${NA}_{acc}$ of 0.22 supports approximately 1870 different modes.³¹ As it supports a larger number of modes per area compared with an SMF or an MCF, it is fundamentally more efficient in delivering the image information from one end to the other. In comparison, the SMF inevitably requires an additional bulky scanning unit for distal-end scanning, and the MCF has a facet mainly composed of cladding material, resulting in a significantly lower information transmission capacity.

However, the MMFs have not been conventionally used as endoscopic imaging probes because both the amplitude and phase information and even the polarization are distorted after being transmitted through the fiber,³² as shown in Fig. 2(c). This property can be understood based on the discrete guided modes in an MMF, with the incident light being decomposed into discrete optical modes that independently propagate through an MMF with corresponding propagation constants and transverse mode patterns. From a geometrical viewpoint, discrete guided modes propagate the fiber with different angles to the axis of the waveguide and consequently different propagation distances, resulting in different propagation constants. In this process, the propagation constants of different modes vary significantly, and thus the phase delay of each mode is effectively random after being transmitted through a fiber. Furthermore, bending and fabrication imperfections of fiber cause perturbations and coupling between the linearly polarized modes, and consequently the light transmission through an MMF can be considered to be a process of mode mixing with random phases. As a result, along with the mode-dependent loss and the modal noise, when monochromatic light is incident on one end of the MMF, randomly fluctuating amplitude and phase patterns appear at the other end regardless of the incident object field. Furthermore, different group velocities of each optical mode and each wavelength, respectively, lead to modal dispersion and chromatic dispersion such that a short optical pulse is temporally broadened during fiber transmission.²⁹^,³³^,³⁴ This combined distortion effect of spatiotemporal properties of the MMF precludes its use in endoscopic imaging.

3. Transmission Matrix: Formalism for the Coherent Manipulation of Light

An input-output relationship of a linear optical system, including optical fibers, can be described based on a transmission matrix formalism. A transmission matrix is typically defined on a position basis or a transverse wavevector basis (i.e., plane wave basis). For instance, when defined on a position basis, the transmission matrix element, $t_{m, n}$ , represents the transmittance and phase delay of the wave component from an input point ( $n$ ) to an output point ( $m$ ), where $n$ and $m$ represent the indices for the spatially discretized modes of incoming and outgoing waves. Thus a column vector of the transmission matrix represents a point spread function on the output plane for a certain input mode (e.g., a point source on the input plane), as shown in Fig. 3. The overall characteristics of the transmission matrix vary significantly depending on the configuration of the linear optical system between the input plane and the output plane. For an ideal imaging system, the transmission matrix is given as an identity matrix, whereas for a highly scattering medium, the matrix elements can be described as independent and identically distributed complex Gaussian random variables.

Fig. 3

Transmission matrix formalism in the position basis. Each column of the transmission matrix represents the complex field profile contributed from a certain input point through the linear optical medium of interest.

In the past few decades, researchers have demonstrated tremendous success in measuring the transmission matrix and demonstrating various coherent light manipulation techniques—optical focusing, imaging, and energy transport—through optically disordered media.²²^,³⁵^–³⁷ In recent developments, spatial light modulators (SLMs) and interferometric measurement techniques have played a critical role in that they allow for direct access to the transmission matrix with many input and output entries.²⁶ More specifically, to characterize a transmission matrix, output optical fields are interferometrically measured, and more than hundreds of different incoming wavefronts are incident on the input plane via SLMs such as liquid crystal on silicon (LCoS) and digital micromirror devices (DMDs). For the coherent manipulation of light, the measured transmission matrix can largely be used in two ways: wavefront shaping and computation. Wavefront shaping techniques are physically applied on the input side of the optical system using SLMs to display an optimized wavefront for a desired optical output, such as optical focusing, patterning, and enhanced energy transport.²⁷^,³⁸^–⁴⁰ In contrast, computational techniques are digitally applied on the measured output field to reconstruct optical fields on the input side through various linear algebra operations, such as matrix inversion and singular value decomposition.³⁶

3.1.

Phase Conjugation and Matrix Inversion

To outline the mathematical frameworks of coherent manipulation techniques based on a transmission matrix formalism, let us assume that a transmission matrix $T$ is given for a disordered optical medium on a position basis. The relation between the input and output fields is then described as

E^{out} = T E^{in}

where

E^{in}

and

E^{out}

are the input and output field vectors with the

N

and

M

entries, respectively. Then, the phase conjugation operation on the input plane allows for optical focusing on a specific output position. When the input field is given as the complex conjugate of the

k

’th row of the transmission matrix

E^{in} = {[\begin{matrix} t_{k, 1}^{*} & t_{k, 2}^{*}, & \dots & , t_{k, N}^{*} \end{matrix}]}^{T}

, corresponding output components constructively interfere at the target output position of the index

k

, whereas they randomly interfere at the background modes with random relative phases. Mathematically,

E_{k}^{out} = \sum_{n = 1}^{N} t_{k, n} t_{k, n}^{*} = \sum_{n = 1}^{N} {| t_{k, n} |}^{2}, E_{m \neq k}^{out} = \sum_{n = 1}^{N} t_{m, n} t_{k, n}^{*} = \sum_{n = 1}^{N} | t_{m, n} | | t_{k, n}^{*} | e^{i Δ ϕ_{m, n - k, n}},

where

Δ ϕ_{m, n - k, n}

is the phase difference between two transmission matrix elements,

t_{m, n}

and

t_{k, n}

. This phase difference randomly fluctuates when

m \neq k

, assuming that each matrix element independently follows the complex Gaussian random statistics. Therefore, the ensemble-averaged field amplitude at the targeted position,

⟨ | E_{k}^{out} | ⟩

, is given as

⟨ {| t_{m, n} |}^{2} ⟩ N

, and the background field,

⟨ | E_{m \neq k}^{out} | ⟩

, is given as

⟨ {| t_{m, n} |}^{2} ⟩ \sqrt{N}

. Therefore, on average, the optical focus can be created with the intensity contrast of

N

, with respect to the background. Practically, the number of controllable input modes

N

is limited to the effective number of pixels on the SLM, which varies within

10^{3} \sim 10^{6}

.

Along with the phase conjugation operation, the inversion operation also serves as a powerful tool for coherent manipulation. Often, the inversion operation is applied to the output field in the form of matrix multiplication to retrieve the input field. Mathematically, the estimated input field ${\hat{E}}^{in}$ is given as $T^{- 1} E^{out}$ . However, it should be noted that the inverse operation of the transmission matrix is considerably unstable in a practical sense. Given that the transmission matrix $T$ is represented with $T = U Σ V^{*}$ by singular value decomposition, where $U$ and $V$ are unitary matrices and $Σ$ is a diagonal matrix with non-negative singular values, the inverse matrix is given as $T^{- 1} = V Σ^{- 1} U^{*}$ , where the diagonal elements of $Σ^{- 1}$ are the inverse of the non-negative singular values. In the case in which the columns of the transmission matrix are highly correlated or rank deficient, a large portion of the singular values are close to zero, which is easily corrupted by small experimental noise. To overcome this issue, a method to only take the singular values over a certain threshold value, called truncated singular value decomposition, or a nonlinear optimization algorithm is often used to reduce experimental errors. This approximated inversion operation often leads to fluctuating background fields (i.e., inaccurate retrieval of the input field), similar to the fluctuating background intensity for optical focusing in the phase conjugation operation.

4. Controlling Coherent Waves through Multimode Fibers

When one attempts to implement endoscopic imaging through an MMF, light travels twice through an MMF in opposing directions: (1) from the proximal end to the distal end to illuminate an object and (2) from the distal end to the proximal end to convey the back-reflected or fluorescence light (i.e., image information) from an object. Therefore, to reconstruct an image of the object behind the MMF, it is necessary to cancel out the effects of wavefront distortion on the way in and out. In this section, we review novel MMF-based endomicroscopic imaging techniques that compensate for the effects of wavefront distortions based on the light manipulation techniques introduced in Sec. 3.

4.1.

Optical Focusing

First, the wavefront distortion on the illumination path (i.e., proximal to distal end) can be corrected by physically implementing the phase conjugation operation based on wavefront shaping techniques.⁴¹^–⁴³ The light transmission through MMFs via an SLM can be modeled based on the transmission matrix formalism

E_{m}^{out} = \sum_{n = 1}^{N} t_{m, n} e^{i θ_{n}}

where

t_{m, n}

is an element of the transmission matrix relating the plane of the SLM on the proximal end to the target plane on the distal end and

θ_{n}

is the phase of the outgoing light at the

n

’th element of the SLM. Given that the transmission matrix is characterized, based on the principle of phase conjugation, light can be arbitrarily focused on the desired position (i.e.

m

’th mode) by displaying the phase profile of

t_{m, n}^{*} / | t_{m, n} | (n = 1 \dots N)

on the SLM. Alternatively, without transmission matrix measurements, one may determine the optimal phase of each SLM segment for light focusing by cycling its phase from 0 to

2 π

and iteratively optimizing each SLM segment to maximize the intensity at the target point²¹^,⁴⁴^–⁴⁸ (as illustrated in Fig. 4). It should be noted here that the result of light focusing through iterative feedback is ideally the same as the phase conjugation approach because both methods essentially lead to constructive interference of the transmitted light at the desired position behind an MMF.

Fig. 4

MMF-based endomicroscopy. (a) Principle of optical focusing through an MMF. Each input mode on an SLM is optimized for the focal intensity on the desired point at the distal side through iterative feedback. (b) 2D intensity distribution and 1D profile on the distal end after the wavefront optimization process. Scale bar: $20 μ m$ (left) and $100 μ m$ (right). Adapted from Ref. 31. (c) Principle of wide-field imaging through an MMF. For different incident angles on the proximal side, the complex optical amplitude maps on the distal side are retrieved by an inversion of the transmission matrix. The intensity summation of the retrieved field maps then suppresses the speckle noise in illumination. (d) Reconstructed image of United States Air Force (USAF) target and wide-field image of villus in a rat intestine tissue. Scale bar: $50 μ m$ . Reprinted figure with permission from Ref. 49. Copyright (2022) by the American Physical Society. (e) Principle of confocal imaging through an MMF. With the combined approach of wavefront shaping and matrix inversion, the effect of wavefront distortions on both the input and output paths can be corrected. (f) Reconstructed images without a virtual pinhole (left) and with a virtual pinhole implemented with a digital confocal method ( $T^{- 1}$ , middle) and with a correlation method ( $T *$ , right). Scale bar: $20 μ m$ . Adapted from Ref. 50.

Figure 4(b) shows the resultant light intensity distribution on the distal end when applying the phase-conjugated wavefront on the proximal end.³¹ By virtue of constructive interference, the ideal diffraction-limited resolution could be achieved; however, the focal contrast (i.e., the peak-to-background ratio, PBR) was as low as 920. In the demonstrated experiment, the number of propagating modes in the MMF was estimated to be 1870, and thus the expected value of the focal contrast was about $\frac{π}{4} (N - 1) + 1 ≅ 1470$ based on the assumption that each transmission matrix element follows the independent complex Gaussian distribution and only the phase of the input field is modulated.⁵¹ This discrepancy could be attributed to the following factors: (1) only a fraction of the propagating modes effectively contributes to the target spot due to the cylindrical structure of MMFs,⁴⁶ (2) no polarization control was applied,⁵² and (3) the column vectors of the transmission matrix may be highly correlated in the MMFs.⁵³^,⁵⁴

4.2.

Wide-Field Imaging

To compensate for the wavefront distortion on the collection path, it is generally preferable to computationally apply linear algebra operations on the measured output field, instead of applying the physical wavefront shaping techniques. This is partly due to the fact that the effect of wavefront shaping can be exactly computed for a given output field without the use of an additional SLM. More specifically, based on the inversion operation, the target field $E^{distal}$ on the distal end is reconstructed from the measured field $E^{proximal}$ on the proximal side by

{\hat{E}}^{distal} = T^{- 1} E^{proximal} = T^{- 1} (T E^{distal}) ≅ E^{distal} .

In conjunction with the inversion operation, Choi et al.⁴⁹ proposed a novel approach for wide-field microendoscopic imaging through MMFs, which is composed of two steps: an inversion operation to compensate for the wavefront distortion on the collection path and incoherent speckle averaging to correct for the effect of the wavefront distortion on the illumination path. When the illumination is provided through an MMF, a randomly fluctuating speckle field is developed on the distal side. Therefore, the reconstructed intensity ( $| {\hat{E}}^{distal} |^{2}$ ) is given as the product of the illumination speckle intensity pattern ( $| S |^{2}$ ) and the object reflectance ( $| O |^{2}$ ):

{| {\hat{E}}^{distal} |}^{2} = {| SO |}^{2},

where the speckle field

S

is a granular pattern that looks like random noise. Choi et al. then exploited the fact that the contrast of the incoherent sum of different speckles decreases with

\sqrt{N}

(where

N

is the number of independent speckles).⁵⁵ In the demonstrated experiments, the different speckle patterns on the distal end were generated by injecting light on the proximal end with different incident angles

(θ_{ξ}, θ_{η})

. This process is mathematically described as

\sum_{θ_{ξ}, θ_{η}} {| {\hat{E}}^{distal} |}^{2} = {| O |}^{2} \sum_{θ_{ξ}, θ_{η}} {| S_{(θ_{ξ}, θ_{η})} |}^{2} ≅ α {| O |}^{2} .

Figure 4(d) shows the reconstructed image of the USAF target with a FOV of $200 μ m$ , which is about the same as the core size of the MMF used. To demonstrate the practicality in biomedical application, a wide FOV image of biological tissue—the villus in a rat intestine tissue—was also acquired by stitching the reconstructed images while scanning the distal end of the probe across the specimen. Although the transmission matrix is very vulnerable to any change in the configuration of the MMF, they succeeded in reconstructing objects centimeters apart without any additional calibration of the transmission matrix.

4.3.

Confocal Imaging

Confocal microscopy is an important tool in biological imaging by virtue of its high contrast and depth-sectioning capability.⁵⁶ Commercial confocal microscopy requires a high-quality objective lens and scanning mirror to make a focused spot across the object and a pinhole to reject the out-of-focus signals. In 2015, Loterie et al.⁵⁰ achieved endoscopic confocal imaging through MMFs by combining the two operations of phase conjugation and matrix inversion, respectively, to compensate for the effect of wavefront distortion on the way in and out.

First, the MMF can replace the light-focusing function of the objective lens and the rotating mirror. Once the transmission matrix is measured, the phase conjugate of the transmission matrix at the proximal end can make a focus on the distal end. The reflected light from the target focus is then collected back through the fiber and recorded at the proximal end. Here, the transmission matrix can be used to implement a digital pinhole in two ways: a digital confocal method and a correlation method. The digital confocal method virtually propagates the collected light back to the focal plane by an inverse transmission matrix operation ( $T^{- 1}$ ), and then a digital pinhole mask is applied at the target point. The light intensity inside the virtual pinhole constitutes a single pixel value in the final confocal image. Alternatively, the inverse operation can be estimated with a complex conjugate operation of the transmission matrix when the column vectors of the transmission matrix are nearly orthogonal, $T T^{*} \sim I$ , which is called the correlation method. In particular, in the case of single-mode reconstruction, the correlation method performed comparably to the inversion method, whereas its computation cost is much lower than the inversion method.

Figure 4(f) shows a diagram of endoscopic confocal imaging, and the right panels show three different images of a human epithelial cell; the first image is the result without the digital pinhole operation, the second is the result of the digital confocal method, and the third is from the correlation method. Compared with the first image, the right two panels show higher contrast to the extent that the walls and the nucleus of an epithelial cell are clearly distinguished by virtue of an additional confocal filtering process on each focal spot. However, the point scanning rate was limited to 20 to 100 Hz because of the slow operational speed of the SLM, which is not adequate for practical video-rate endoscopic imaging applications. The point scanning rate can be improved up to 40 kHz by combining an SLM with an acousto-optic deflector, but this comes at the cost of a complex beam shaping configuration and reduction in the effective pixel number of the SLM.⁵⁷

4.4.

Scanning Fluorescence Imaging

From Secs. 4.1 to 4.3, we reviewed coherent manipulation techniques and associated label-free endomicroscopic imaging techniques in the reflection geometry. As label-free imaging based on tissue reflectance generally suffers from low image contrast, fluorescence imaging serves as an important endoscopic imaging modality for diagnosis. In this case, the optical signal from the distal end is the fluorescence signal, which is temporally incoherent, and thus the coherent manipulation techniques cannot be applied on the collection path. In this section, we discuss the use of the wavefront shaping technique to implement scanning fluorescence microscopy through MMFs.

4.4.1.

Fluorescence endomicroscopy

In scanning fluorescence microscopy, light is focused on the object and the focal spot is scanned across the imaging area or volume. As in the diagram of Fig. 5(a), a DMD was used instead of an SLM to also achieve a fast wavefront modulation of the input wave. However, as DMDs intrinsically are only capable of binary amplitude modulation, the arbitrary phase pattern $φ (x, y)$ was encoded in the binary off-axis amplitude hologram (or Lee hologram)⁶⁰ $h (x, y)$ as

t (x, y) = \frac{1}{2} [1 + \cos (2 π (u x + v y) + φ (x, y))], h (x, y) = {\begin{cases} 1; & t (x, y) > 0.5 \\ 0; & o . w \end{cases}},

where

u

and

v

are the carrier frequencies to separate the first order from the zeroth order diffracted beam of

t (x, y)

. Only the first order of the diffracted beam is taken by the first lens and the spatial filter at the Fourier plane of that lens such that the desired wavefront

e^{j φ (x, y)}

is generated at the Fourier plane of another lens, as in Fig. 5(a). Based on this operation, it is possible to measure the transmission matrix and make a focus on the distal end by using the phase conjugate operation on the proximal end. An excited fluorescent signal from the target point is collected through the fiber and simply summed at the proximal end.⁵⁸ In fact, it is not possible to apply a digital pinhole technique, as shown in Sec. 4.3, because we cannot measure the optical field of incoherent light in a holographic manner. Therefore, the fluorescence signal from the fluctuating background intensity could not be filtered out, resulting in a significant decrease in the image contrast.

Fig. 5

Schematics of MMF-based scanning fluorescence endomicroscopy setup and imaging results. (a) Experimental setup of single-photon scanning fluorescence imaging through an MMF. (b) Images of $4 - μ m$ fluorescence beads (top) and monkey brain slice (bottom) acquired using a conventional widefield fluorescence microscope (left) and MMF-based scanning endoscope (right). Scale bar: $20 μ m$ . (c) Experimental setup of two-photon fluorescence imaging through an MMF. (d) Images of fluorescence beads in the lateral field of view ( $28 \times 85 μ m$ ) at $z = 4$ , 14, 32, and $50 μ m$ . Scale bar: $10 μ m$ . Panels (a) and (b) adapted from Ref. 58 and panels (c) and (d) from Ref. 59.

Figure 5(b) shows fluorescence beads (top row) and a monkey brain slice (bottom row) acquired with a fluorescence widefield microscope (left column) and an MMF-based scanning endoscope (right column). It clearly shows the degradation in the image contrast because of the low PBR value. Importantly, the DMD’s fast refresh rate of around 20 kHz could allow for video frame rates as required for in vivo applications.

4.4.2.

Two-photon endomicroscopy

The imaging principle of two-photon excitation microscopy through an MMF is similar to that of single-photon fluorescence imaging through an MMF, except that a high peak flux of the excitation (typically produced by a femtosecond pulsed laser) is required at the distal end. The reason for this is that the probability of two-photon absorption is extremely low and has a quadratic dependence on the instantaneous optical power. Here, the major issue is that, when a short pulse of the laser beam is incident on the proximal side of the MMF, the modal dispersion in the MMF spreads the incidence power over time, and consequently the phase conjugation operation of the ordinary time-averaged transmission matrix (i.e., transmission matrix measured with a continuous laser) does not result in a temporally focused spot.

In 2015, Morales-Delgado et al.⁵⁹^,⁶¹ proposed a method to characterize a time-gated transmission matrix using an ultrafast reference light pulse to selectively measure and utilize only the optical modes in a similar group velocity for the wavefront shaping process. To further reduce the pulse broadening effect due to modal dispersion and group velocity dispersion through a 20-cm-long MMF, graded-index multimode fiber was selected over the step-index MMF, and two prism pairs were introduced on the input beam incident on the proximal side in the reconstruction step. Using the combined techniques, a diffraction-limited focal spot could be made on the distal end with a pulse width of 120 fs, which was nearly six times smaller than the result obtained without applying wavefront shaping (i.e. $\sim 745 fs$ ).

With the nonlinear characteristic of two-photon fluorescence excitation, its excitation volume is much smaller than that of the single photon excitation. In particular, in contrast to the single photon excitation, the two-photon fluorescence excitation at the out-of-focus plane, and the background fluctuating intensity at the focal plane were significantly suppressed [see Fig. 5(c)]. Therefore, it shares the same advantage as the confocal microscope—higher contrast and depth-sectioning capability—even without the confocal gating operation. Figure 5(d) shows the two-photon images of fluorescent beads in various depths ( $z = 4$ , 14, 32, and $50 μ m$ ).

4.4.3.

Biological applications

A few successful demonstrations have been reported to show the applicability of the laser-scanning imaging technique through MMFs with a major emphasis on the capability to visualize cellular or sub-cellular details of biological tissues. Here, we present representative results from tissue culture cells, neuronal imaging, and cochlea imaging.

Figure 6(a) shows two types of tissue culture cells⁶²—baby hamster kidney cells (BHK) expressing GFP and hippocampal tissue culture expressing genetically encoded calcium indicator (GCaMP6f). Through a comparison with epi-fluorescence imaging, the capability to resolve individual cells was validated. The reported frame rate to acquire the images in Fig. 6(a) was 7 to 15 Hz for the circular FOV with a diameter of $100 μ m$ and a spatial resolution of $\sim 2 μ m$ with the use of a DMD. Furthermore, with the aid of a genetically encoded calcium indicator, GCAMP6f, the ability to observe spontaneous neuronal activity was demonstrated⁶² [see Fig. 6(b)]. In a separate study,⁶³ A. Vasquez-Lo pez et al. showed sub-cellular details, such as a dendritic spine and axonal boutons of hippocampal neurons in a rat brain slice with an improved resolution of $\sim 1.3 μ m$ [see Fig. 6(c)]. Finally, Kakkava et al.⁶⁴ demonstrated the use of two-photon fluorescence imaging of hair cells in a fixed cochlea tissue and the capability to perform femtosecond laser ablation of individual hair cell based on the acquired fluorescence image [see Fig. 6(d)].

Fig. 6

Structural and functional images of various biological samples acquired using MMF-based fluorescence endomicroscopic imaging. (a) In vitro images of BHK cells (top) and hippocampal neuronal tissue culture cells (bottom) acquired using epi-fluorescence microscopy (left) and MMF-based single-photon scanning fluorescence endomicroscopy (right). Scale bar: $11 μ m$ . (b) Spontaneous activity of hippocampal neuronal tissue culture cells shown in the bottom of panel (a). The neuronal activity of multiple neurons was recorded by monitoring the fluorescence signal of the calcium indicator GCaMP6f. (c) Ex vivo images of fluorescence-labeled hippocampal neurons in a rat brain slice using confocal microscopy (green) and through an MMF (black). Scale bar: $20 μ m$ . (d) Two-photon fluorescence images of the cochlear hair cells before and after selective ablation marked as a white circle. Scale bar: $10 μ m$ . (e) In vivo fluorescence images of SST neurons (top, middle) and hemorrhage in the primary visual cortex with nonlabelled blood cells (bottom) in the mouse brain. Scale bar: $10 μ m$ . (f) Postmortem histological image of the coronal brain section in the visual cortex and hippocampus. The arrows on the right panel indicate the insertion tract of the MMF probe. Scale bar: $200 μ m$ . Panels (a) and (b) adapted from Ref. 62, panel (c) from Ref. 63, panel (d) from Ref. 64, and panels (e) and (f) from Ref. 65.

Despite the mechanical movements associated with living animals, MMF-based fluorescence imaging techniques have also proven to be effective for in vivo neuronal imaging. The first and second rows of Fig. 6(e) show in vivo fluorescence images of a subpopulation of inhibitory neurons,⁶⁵ somatostatin-expressing (SST) neurons,⁶⁵ labeled with a red fluorescent marker (tdTomato) at different regions of a mouse brain, the visual cortex ( $V 1$ , 0.5 to 0.8 mm in depth) and the cornu ammonis 1 (CA1) region and dentate gyrus of the hippocampus (1.5 and 2 mm in depth). The reported frame rate to acquire the images in Fig. 6(e) was 3.5 Hz for the circular FOV with a diameter of $50 μ m$ and a spatial resolution of about $1.2 μ m$ with the use of a DMD. Additionally, it was shown that the dynamics of hemorrhage in the primary visual cortex⁶⁵ could be visualized in a nonlabelled manner based on the intrinsic optical contrast from red blood cells [appearing as dark clusters in the last row of Fig. 6(e)]. Compared with the ex vivo experiments, in vivo experiments generally result in degradation of the image quality because the optical transmission matrix of MMFs is varied from the precalibrated one due to fiber bending accompanied by the insertion and extraction of MMFs in biological tissue. However, the characteristic structures of the target tissues are recognizable at the neuronal scale, and the object planes at different depths are accessible by digital refocusing without mechanically moving the distal tip of the fiber. More importantly, the reported results show that structural and functional in vivo microscopic imaging is possible in a minimally invasive way with a fiber tract width of about $50 μ m$ [as shown in the postmortem histological image presented in Fig. 6(f)].⁶⁵

To summarize various imaging approaches based on MMFs, Table 2 lists the key results from the representative studies based on each approach.

Table 2

Main performance of the MMF-based endoscopy.

	Resolution (NA)	FOV (core diameter)	Image acquisition speed	Characteristics
1. Wide field⁴⁹	$1.8 μ m$ (0.48)	$200 μ m$ ( $200 μ m$ )	1 s	Possible to extend FOV by five times with image stitching
2. Confocal⁵⁰	$1.5 μ m$ (0.22)	$81 μ m$ ( $105 μ m$ )	4 min 15 s	Depth sectioning capability ( $12.7 μ m$ )
3. Scanning Fluorescence⁵⁸	$2 μ m$ (0.29)	$80 μ m$ ( $100 μ m$ )	—	DMD reduces acquisition time
4. Two-photon⁵⁹	$1 μ m$ (0.34)	$80 μ m$ ( $106 μ m$ )	—	Depth sectioning capability ( $15 μ m$ )

5. Controlling Coherent Waves through Multicore Fibers

The MCF has long been used for clinical endoscopes as well as commercialized confocal endomicroscopic imaging systems (e.g., Mauna Kea Technologies); however, the finite inter-core distance on the facets of the MCF fundamentally limits the resolution with a pixelation issue. In this section, we review two techniques to remove the pixelation artifacts and achieve diffraction-limited imaging through an MCF based on a lensless configuration on the distal end.

5.1.

Using Transmission Matrix Formalism

Kim et al. demonstrated the binarized transmission matrix approach on MCFs using a DMD located on the proximal side, as in the diagram of Fig. 7(a). Here, instead of implementing phase modulation based on the Lee hologram method, as in Sec. 4.4.1, optical focus on the distal side of the MCF is created based on the binarized phase conjugation operation, in which the output fields from each input mode interfere semi-constructively at the target position ( $m$ ’th mode)⁶⁶

E_{m}^{out} = \sum_{n = 1}^{N} a_{n} t_{m, n}, a_{n} = {\begin{cases} 1; & 0 \leq \arg (t_{m, n}) \leq π \\ 0; & otherwise \end{cases}} .

Fig. 7

Principles and results of endomicroscopic imaging through an MCF. (a) Experimental setup of MCF-based scanning fluorescence endomicroscopy. (b) Pixelation-free endoscopic images of $2 - μ m$ fluorescence beads (top) and in vitro cancer cell (bottom) acquired using conventional transmission imaging (left) and MCF-based scanning fluorescence imaging (right). Scale bar: $20 μ m$ . (c) Angular memory effect through an MCF. The object plane and the image plane are placed at the distance of $U$ and $V$ from the distal and proximal facets, respectively. $U$ and $V$ are assumed to be sufficiently large to meet the far-field condition. A point source at the displaced position $δ X$ on the object plane produces an identical but shifted $(δ Y = δ X \cdot V / U)$ speckle pattern on the imaging plane. The object was placed at $U = 161 mm$ from the fiber’s distal facet. (d) Image reconstruction process of speckle correlation imaging. Scale bar for the raw intensity distribution measured from the camera: 10 mm. Scale bar for the intensity autocorrelation function, reconstructed, and ground truth image: 0.5 mm. Panels (a) and (b) adapted with permission from Ref. 66 © The Optical Society and panels (c) and (d) from Ref. 67.

The total intensity of the excited fluorescence signal from the focus was collected by the same fiber bundle and then measured by the photomultiplier tube.

Figure 7(a) shows the experimental setup based on a DMD and an MCF, and Fig. 7(b) shows $2 - μ m$ fluorescence bead (top row) and cancer cell (bottom row) images obtained through the conventional transmission imaging technique (left column) and the endoscopic fluorescence imaging technique based on the transmission matrix (right column) through the same MCF. The pixelation artifact was completely removed, and the resolution was reduced to $1.07 μ m$ , which was less than the diameter of the cores of the MCF,⁶⁶ which is usually about $3.45 μ m$ . Previously, conventional MCF-based microendoscopy (pCLE) captured the light intensity from object points that are optically conjugate to each core of the MCFs, limiting the lateral resolution to the size of the core. In contrast, the transmission matrix formalism and associated wavefront shaping techniques can achieve the diffraction-limited resolution supported by the entire facet of the MCF and remove the pixelation artifacts by compensating for the effect of mode-mixing and phase delays through the cores. However, assuming that each transmission matrix coefficient follows an independent complex Gaussian distribution, the theoretical PBR value in light focusing via binary amplitude modulation is $\frac{N}{2 π}$ for $N$ spatial input modes,⁶⁸ which significantly degrades the image contrast.

5.2.

Using Speckle Correlation

Noninvasive imaging via speckle correlation is one of the representative computational imaging techniques for a thin scattering layer that presents the characteristic input-output relation called the “memory effect.”⁶⁹^,⁷⁰ The memory effect is the result of the short-range correlation within a transmission matrix of a thin scattering medium in which a lateral shift of the point source on the input side results in a lateral shift of the speckle pattern on the output side, assuming that the point source and the measurement plane on the input and output sides are placed at the far-field distance from the scattering medium. This shift-shift memory effect on the far-field manifests when the angular spectrum of the scattered field on the output surface of the scattering medium is nearly conserved for different incidence angles, which is only guaranteed when a diffusive scattering medium has a thickness comparable to the wavelength.²⁸^,⁷¹^,⁷² The MCFs with a small core diameter can be considered to be an infinitesimally thin random phase plate that exhibits an ideal memory effect⁶⁷ [see Fig. 7(c)]. Thus, with a spatially incoherent object on the distal side, the captured intensity image on the proximal side through MCFs is simply given as the superposition of the laterally shifted speckle patterns that are generated from each object point. The mathematical formula for this relation is represented with the simple convolution operation between the object’s intensity distribution $O (r)$ and the speckle intensity pattern $S (r)$ associated with a random phase delay map of the cores of an MCF as

I (r) = O (r) * S (r) .

Then, taking the autocorrelation, denoted as $\otimes$ , of the measured intensity image $I (r)$ , the convolution theorem yields the following relation:

I (r) \otimes I (r) = [O (r) * S (r)] \otimes [O (r) * S (r)] = [O (r) \otimes O (r)] * [S (r) \otimes S (r)] .

The autocorrelation of the speckle pattern is the sharply-peaked function with the diffraction-limited width, $S (r) \otimes S (r) \sim δ (r)$ . Therefore, the autocorrelation of the intensity image will approximate the autocorrelation of the object’s intensity distribution as

I (r) \otimes I (r) \sim O (r) \otimes O (r) .

From the estimated object’s autocorrelation function, one may directly reconstruct the original object function using a phase retrieval algorithm.⁷³ The results associated with each computational process are presented in Fig. 7(d).

Compared with the transmission matrix, this speckle correlation can achieve a diffraction-limited resolution with only a single-shot image. Also, because this method does not depend on any premeasured data, it is free from the issue of fiber bending and translation. However, the fundamental limit of this approach is that the optical field at the object points must be temporally coherent and spatially incoherent, and the object needs to be uniformly illuminated. Otherwise, the mathematical formulation for the correlation imaging breaks down. There have been approaches to reconstruct the fluorescence image even with a nonuniform illumination by utilizing the memory effect on the illumination path, albeit without the single-shot advantage.⁶⁹^,⁷⁴ Second, it does not have depth scanning capability, which restricts the axial dimension of the object below an axial decorrelation length of the speckle point spread function.

To summarize the various imaging approaches based on MCFs, Table 3 lists the key results from each approach.

Table 3

Main performance of the MCF-based endoscopy.

	Resolution (NA)	FOV (fiber diameter)	Image acquisition speed	Characteristics
1. Scanning fluorescence⁶⁶	$1.07 μ m$ (0.38)	$58 μ m$ ( $300 \pm 25 μ m$ )	0.1 s	DMD reduces acquisition time
2. Speckle correlation⁶⁷	$> 160 μ m$ (very low)	$\sim 2 mm$ ( $530 μ m$ )	Single shot	Single-shot image without any calibration, Requirement on the large distance between the object and the input facet

6. Advanced Techniques

The MMF- or MCF-based endoscopic imaging techniques introduced in Secs. 4 and 5 have several intrinsic limitations such as the vulnerability of the precalibrated transmission matrix to mechanical and thermal perturbations and the entangled spatio-temporal distortion in each guided mode. Here, we review some in situ transmission matrix calibration techniques to overcome the external perturbations and some novel techniques to exploit the spatio-temporal characteristics of the optical fiber.

6.1.

In Situ Transmission Matrix Calibration

All of the methods except the techniques relying on the speckle correlation (in Sec. 5.2) are based on the assumption that the transmission matrix is precalibrated and unchanged after insertion. However, the transmission matrix is extremely vulnerable to any changes in the fiber’s configuration (e.g., bending and deformation of fiber) and temperature distribution. For example, the performance of endoscopic imaging based on the precalibrated transmission matrix is valid up to only a few millimeters of fiber-tip translation⁴⁹ or a few degrees of temperature changes.⁵⁰ Plus, recalibration of the transmission matrix is not possible in practical endoscopic imaging scenarios in which the distal end of the fiber is embedded deep inside the body. To overcome this limitation, numerous approaches have been proposed to calibrate the transmission matrix without any physical access to the distal tip.

A straightforward approach for in situ calibration is to place additional elements at the distal end. One may use point-like sources, such as a nonlinear guide star⁷⁵^,⁷⁶ and a virtual beacon source,⁷⁷ to directly create an optical focus by feedback-based wavefront shaping⁴⁵ or optical phase conjugation. Alternatively, a transmission matrix can be calibrated by illuminating the proximal end and measuring the backward propagating wavefronts from the partial reflectors at the distal tip.⁷⁸^–⁸⁰ Recently, it has also been shown that a target object itself can serve as a guide star to identify the core-to-core phase retardations of the MCF and reconstruct a diffraction-limited and pixelation-free image.⁸¹ With the reflection matrix of the MCF measured with core-to-core illumination, the singly-reflected waves from a target object can be coherently accumulated based on the adaptive algorithm, called closed-loop accumulation of single scattering.⁸²

Another notable approach is to estimate the transmission matrix from the partial information of the matrix elements, which is possible due to the cylindrically symmetric behavior of light within conventional optical fibers with a circular cross-section, called the “rotational memory effect.”⁵³^,⁵⁴ The rotational memory effect enables optical access to a large facet area (i.e. isoplanatic patch) from a point-like source at the distal tip.⁵⁴^,⁸³ More remarkably, it has been also shown that the transmission matrix of an MMF can be estimated without any calibration by decomposing the transmission matrix into the guided modes on a circular polarization basis. The circularly polarized modes are less affected by the effect of polarization coupling due to the cylindrical structure of the fiber, thus allowing for the prediction of the transmission matrix of a deformed fiber by estimating only the phase delay values of each mode from the given bending status.⁸⁴ Other computational imaging approaches such as compressive sensing and deep learning methods have also recently been explored for calibration and imaging through MMFs or MCFs.⁸⁵^–⁸⁸

Rather than repeatedly recalibrating the transmission matrix, there also have been continuous efforts to design a bending-invariant structure of optical fibers,⁸⁹ such as an MMF with a parabolic refractive index profile,⁹⁰ a twisted MCF,⁹¹ and even a hybrid structure of the MMF and MCF for both high-resolution imaging and motion-insensitive low-resolution image.⁹² Alternatively, to circumvent the calibration issue in using an MMF, some approaches have been proposed to encode the spatial information to the spectral information by placing a spatio-spectral encoder at the distal tip of a fiber⁹³^,⁹⁴ and reconstruct an image from spectrally resolved detection at the proximal tip.

6.2.

Using Spatio-Temporal Characteristics of Optical Fibers

Coherent light manipulation methods, introduced in Secs. 4 and 5, mostly deal with only the spatial distortion of an optical field transmitted through fibers. However, as each optical mode of a unique spatial field profile has an associated propagation velocity, spatial and temporal distortions through fibers are entangled with each other, resulting in time-varying speckle-like patterns within a broadened pulse duration for an incident optical pulse. Here, we review some approaches that utilize these spatio-temporal characteristics of fibers.

Recently, microendoscopic imaging capability through an MMF has been successfully extended to include depth information and achieve 3D imaging. High-speed wavefront shaping using a DMD, along with time-of-flight light detection, enabled recovery of depth information alongside 2D reflectance images at a near video rate.⁹⁵ Based on spatio-temporal mixing through a fiber, the temporal dynamics of the pulse can be manipulated by only controlling the spatial profile of the incident wavefront, such as in conserving a pulse shape⁹⁶^–⁹⁸ and enhancing the transmittance at a chosen delay time (i.e., temporal focusing).⁹⁹ Furthermore, an ultrafast wavefront shaping device was developed using a 5-m-long MMF as a spatiotemporal mixer to generate a desired vector spatiotemporal field.¹⁰⁰ Finally, relying on the reciprocal relation between time and frequency, the spatio-temporal characteristic further leads to the spatio-spectral mixing of an optical field through a fiber, enabling novel all-fiber spectroscopic techniques.¹⁰¹

7. Conclusion

Here, we reviewed both physical and computational approaches to exploit the coherent properties of MMFs and MCFs for endomicroscopic imaging. In particular, although MMFs intrinsically have a high information transport capacity, they have not been considered to be image-carrying probes due to their random mode mixing property upon transmission. However, recent investigations on measuring and controlling such randomness inside disordered optical media have led to wavefront shaping techniques and computation methods to deterministically use MMFs based on the transmission matrix formalism. These approaches have been demonstrated to provide a diffraction-limited resolution with an ultrathin probe, which is a critical feature for observing cellular and subcellular morphological changes associated with various diseases.

However, MMF-based endomicroscopy techniques have several intrinsic limitations for widespread use in practical clinical applications. First, in their current form, they highly rely on precalibration of the transmission matrix. Although some in situ calibration techniques have been developed, their performance is still insufficient to be used in real clinical practices in which fast and accurate calibration is required and bending states are not exactly known. This limitation significantly degrades the performance of wavefront shaping and computational image retrieval and restricts the probe’s movement that is required to observe different parts inside the body. Second, the coherent manipulation techniques for MMFs implicitly assume the use of a coherent light source and label-free imaging of internal tissues. Although fluorescence staining can be used for laboratory experiments, as described in Sec. 4.4, in general, fluorescent dyes compatible with the human body are limited. In label-free reflectance imaging with a single wavelength, biological cells and organelles typically provide poor contrast. Finally, the limited refresh rate of SLMs practically results in a small FOV. With relatively fast SLMs, such as DMDs, the refresh rate of tens of kHz results in an imaging rate of $\sim 10 fps$ with $\sim 1, 000$ imaging pixels, which is around 100 to 1,000 times smaller than the pixel counts of conventional SD or HD video endoscopes. Thus, the FOV is currently limited to tens of micrometers for a resolution of $\sim 1 μ m$ , indicating that an MMF-based imaging technique alone would not provide proper diagnostic capabilities.

Based on the developments in the past decade, researchers have already shown some potential solutions to the aforementioned limitations. In particular, with the efforts to utilize unique wave phenomena in MMFs based on computational methods, we anticipate that the coherent manipulation of optical fibers could realize the transformative capability of real-time histology diagnosis.

Acknowledgments

This research was supported by the National Research Foundation of Korea grant funded by the Korean government (Grant Nos. NRF-2021R1A5A1032937 and 2021R1C1C1011307), by the Korea Agency for Infrastructure Technology Advancement grant funded by the Ministry of Land, Infrastructure and Transport (Grant No. 22NPSS-C163379-02), by the KAIST-CERAGEM Next Generation Healthcare Research Center, and by the LG Electronics-KAIST Digital Healthcare Research Center. The authors declare no competing interests.

References

1.

H. Zeng, “Diagnostic endoscopy,” J. Biomed. Opt., 19 (7), 076017 https://doi.org/10.1117/1.JBO.19.7.076017 JBOPFO 1083-3668 (2014). Google Scholar

2.

T. Wang et al., “Heartbeat OCT: in vivo intravascular megahertz-optical coherence tomography,” Biomed. Opt. Express, 6 5021 –5032 https://doi.org/10.1364/BOE.6.005021 BOEICL 2156-7085 (2015). Google Scholar

3.

M. J. Bruno, “Magnification endoscopy, high resolution endoscopy, and chromoscopy; towards a better optical diagnosis,” Gut, 52 (Suppl. 4), iv7 –11 https://doi.org/10.1136/gut.52.suppl_4.iv7 GUTTAK 0017-5749 (2003). Google Scholar

4.

G. D. Hadley, “The gastro-camera,” BMJ, 2 1209 –1212 https://doi.org/10.1136/bmj.2.5472.1209 (1965). Google Scholar

5.

W. L. Curvers et al., “Endoscopic tri-modal imaging for detection of early neoplasia in Barrett’s oesophagus: a multi-centre feasibility study using high-resolution endoscopy, autofluorescence imaging and narrow band imaging incorporated in one endoscopy system,” Gut, 57 167 –172 https://doi.org/10.1136/gut.2007.134213 GUTTAK 0017-5749 (2008). Google Scholar

6.

R. Kiesslich et al., “Confocal laser endomicroscopy,” Gastrointest. Endosc. Clin. N. Am., 15 715 –731 https://doi.org/10.1016/j.giec.2005.08.010 (2005). Google Scholar

7.

H. Neumann et al., “Confocal laser endomicroscopy: technical advances and clinical applications,” Gastroenterology, 139 388 –392 https://doi.org/10.1053/j.gastro.2010.06.029 GASTAB 0016-5085 (2010). Google Scholar

8.

J. M. Jabbour et al., “Confocal endomicroscopy: instrumentation and medical applications,” Ann. Biomed. Eng., 40 378 –397 https://doi.org/10.1007/s10439-011-0426-y ABMECF 0090-6964 (2012). Google Scholar

9.

M. Gu, H. Bao and H. Kang, “Fibre-optical microendoscopy,” J. Microsc., 254 13 –18 https://doi.org/10.1111/jmi.12119 JMICAR 0022-2720 (2014). Google Scholar

10.

A. Perperidis et al., “Image computing for fibre-bundle endomicroscopy: a review,” Med. Image Anal., 62 101620 https://doi.org/10.1016/j.media.2019.101620 (2020). Google Scholar

11.

Optical Coherence Tomography, Springer, Berlin, Heidelberg (2008). Google Scholar

12.

M. J. Gora et al., “Endoscopic optical coherence tomography: technologies and clinical applications [Invited],” Biomed. Opt. Express, 8 2405 https://doi.org/10.1364/BOE.8.002405 (2017). Google Scholar

13.

N. Hasegawa, “Magnifying image pickup unit for an endoscope, an endoscope for in vivo cellular observation that uses it, and endoscopic, in vivo cellular observation methods,” U.S. Patent No. US 7,267,648 B2 (2007).

14.

M. B. Wallace and P. Fockens, “Probe-based confocal laser endomicroscopy,” Gastroenterology, 136 1509 –1513 https://doi.org/10.1053/j.gastro.2009.03.034 GASTAB 0016-5085 (2009). Google Scholar

15.

Y. Nakai et al., “Confocal laser endomicroscopy in gastrointestinal and pancreatobiliary diseases,” Dig. Endosc., 26 (Suppl. 1), 86 –94 https://doi.org/10.1111/den.12152 (2014). Google Scholar

16.

S. S. Chauhan et al., “Confocal laser endomicroscopy,” Gastrointest. Endosc., 80 928 –938 https://doi.org/10.1016/j.gie.2014.06.021 (2014). Google Scholar

17.

H. Zhang and X. Ye, “The development and clinical application of microscopic endoscopy for in vivo optical biopsies: endocytoscopy and confocal laser endomicroscopy,” Photodiagn. Photodyn. Ther., 38 102826 https://doi.org/10.1016/j.pdpdt.2022.102826 (2022). Google Scholar

18.

H. Neumann et al., “Review article: in vivo imaging by endocytoscopy,” Aliment. Pharmacol. Ther., 33 1183 –1193 https://doi.org/10.1111/j.1365-2036.2011.04647.x APTHEN 1365-2036 (2011). Google Scholar

19.

Z. Yaqoob et al., “Methods and application areas of endoscopic optical coherence tomography,” J. Biomed. Opt., 11 063001 https://doi.org/10.1117/1.2400214 JBOPFO 1083-3668 (2006). Google Scholar

20.

R. Shukla, “Endoscopic imaging: how far are we from real-time histology?,” World J. Gastrointest. Endosc., 3 183 https://doi.org/10.4253/wjge.v3.i10.183 (2011). Google Scholar

21.

I. M. Vellekoop and A. P. Mosk, “Focusing coherent light through opaque strongly scattering media,” Opt. Lett., 32 2309 https://doi.org/10.1364/OL.32.002309 OPLEDP 0146-9592 (2007). Google Scholar

22.

S. M. Popoff et al., “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett., 104 100601 https://doi.org/10.1103/PhysRevLett.104.100601 PRLTAO 0031-9007 (2010). Google Scholar

23.

M. Kim et al., “Transmission matrix of a scattering medium and its applications in biophotonics,” Opt. Express, 23 12648 https://doi.org/10.1364/OE.23.012648 OPEXFF 1094-4087 (2015). Google Scholar

24.

O. Katz, E. Small and Y. Silberberg, “Looking around corners and through thin turbid layers in real time with scattered incoherent light,” Nat. Photonics, 6 549 –553 https://doi.org/10.1038/nphoton.2012.150 NPAHBY 1749-4885 (2012). Google Scholar

25.

S. Yoon et al., “Deep optical imaging within complex scattering media,” Nat. Rev. Phys., 2 141 –158 https://doi.org/10.1038/s42254-019-0143-2 (2020). Google Scholar

26.

A. P. Mosk et al., “Controlling waves in space and time for imaging and focusing in complex media,” Nat. Photonics, 6 283 –292 https://doi.org/10.1038/nphoton.2012.88 NPAHBY 1749-4885 (2012). Google Scholar

27.

R. Horstmeyer, H. Ruan and C. Yang, “Guidestar-assisted wavefront-shaping methods for focusing light into biological tissue,” Nat. Photonics, 9 563 –571 https://doi.org/10.1038/nphoton.2015.140 NPAHBY 1749-4885 (2015). Google Scholar

28.

S. Rotter and S. Gigan, “Light fields in complex media: mesoscopic scattering meets wave control,” Rev. Mod. Phys., 89 015005 https://doi.org/10.1103/RevModPhys.89.015005 RMPHAT 0034-6861 (2017). Google Scholar

29.

B. E. A. Saleh and M. C. Teich, Fundamentals of Photonics, John Wiley & Sons, Inc., New York (1991). Google Scholar

30.

G. Oh, E. Chung and S. H. Yun, “Optical fibers for high-resolution in vivo microendoscopic fluorescence imaging,” Opt. Fiber Technol., 19 760 –771 https://doi.org/10.1016/j.yofte.2013.07.008 (2013). Google Scholar

31.

I. N. Papadopoulos et al., “Focusing and scanning light through a multimode optical fiber using digital phase conjugation,” Opt. Express, 20 10583 https://doi.org/10.1364/OE.20.010583 OPEXFF 1094-4087 (2012). Google Scholar

32.

D. Psaltis and C. Moser, “Imaging with multimode fibers,” Opt. Photonics News, 27 24 https://doi.org/10.1364/OPN.27.1.000024 OPPHEL 1047-6938 (2016). Google Scholar

33.

K. Okamoto, Fundamentals of Optical Waveguides, Elsevier, New York (2006). Google Scholar

34.

K. P. Ho and J. M. Kahn, “Linear propagation effects in mode-division multiplexing systems,” J. Lightwave Technol., 32 614 –628 https://doi.org/10.1109/JLT.2013.2283797 JLTEDG 0733-8724 (2014). Google Scholar

35.

S. M. Popoff et al., “Controlling light through optical disordered media: transmission matrix approach,” New J. Phys., 13 123021 https://doi.org/10.1088/1367-2630/13/12/123021 NJOPFM 1367-2630 (2011). Google Scholar

36.

Y. Choi et al., “Overcoming the diffraction limit using multiple light scattering in a highly disordered medium,” Phys. Rev. Lett., 107 023902 https://doi.org/10.1103/PhysRevLett.107.023902 PRLTAO 0031-9007 (2011). Google Scholar

37.

M. Kim et al., “Maximal energy transport through disordered media with the implementation of transmission eigenchannels,” Nat. Photonics, 6 581 –585 https://doi.org/10.1038/nphoton.2012.159 NPAHBY 1749-4885 (2012). Google Scholar

38.

I. M. Vellekoop, A. Lagendijk and A. P. Mosk, “Exploiting disorder for perfect focusing,” Nat. Photonics, 4 320 –322 https://doi.org/10.1038/nphoton.2010.3 NPAHBY 1749-4885 (2010). Google Scholar

39.

M. Jang et al., “Wavefront shaping with disorder-engineered metasurfaces,” Nat. Photonics, 12 84 –90 https://doi.org/10.1038/s41566-017-0078-z NPAHBY 1749-4885 (2018). Google Scholar

40.

J. Ryu et al., “Optical phase conjugation assisted scattering lens: variable focusing and 3D patterning,” Sci. Rep., 6 23494 https://doi.org/10.1038/srep23494 SRCEC3 2045-2322 (2016). Google Scholar

41.

Z. Yaqoob et al., “Optical phase conjugation for turbidity suppression in biological samples,” Nat. Photonics, 2 110 –115 https://doi.org/10.1038/nphoton.2007.297 NPAHBY 1749-4885 (2008). Google Scholar

42.

M. Jang et al., “Method for auto-alignment of digital optical phase conjugation systems based on digital propagation,” Opt. Express, 22 14054 https://doi.org/10.1364/OE.22.014054 OPEXFF 1094-4087 (2014). Google Scholar

43.

M. Cui and C. Yang, “Implementation of a digital optical phase conjugation system and its application to study the robustness of turbidity suppression by phase conjugation,” Opt. Express, 18 3444 https://doi.org/10.1364/OE.18.003444 OPEXFF 1094-4087 (2010). Google Scholar

44.

S. Bianchi and R. di Leonardo, “A multi-mode fiber probe for holographic micromanipulation and microscopy,” Lab Chip, 12 635 –639 https://doi.org/10.1039/C1LC20719A LCAHAM 1473-0197 (2012). Google Scholar

45.

I. M. Vellekoop, “Feedback-based wavefront shaping,” Opt. Express, 23 12189 https://doi.org/10.1364/OE.23.012189 OPEXFF 1094-4087 (2015). Google Scholar

46.

R. di Leonardo and S. Bianchi, “Hologram transmission through multi-mode optical fibers,” Opt. Express, 19 247 https://doi.org/10.1364/OE.19.000247 OPEXFF 1094-4087 (2011). Google Scholar

47.

R. N. Mahalati et al., “Adaptive control of input field to achieve desired output intensity profile in multimode fiber with random mode coupling,” Opt. Express, 20 14321 https://doi.org/10.1364/OE.20.014321 OPEXFF 1094-4087 (2012). Google Scholar

48.

A. M. Caravaca-Aguirre et al., “Real-time resilient focusing through a bending multimode fiber,” Opt. Express, 21 12881 https://doi.org/10.1364/OE.21.012881 OPEXFF 1094-4087 (2013). Google Scholar

49.

Y. Choi et al., “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett., 109 203901 https://doi.org/10.1103/PhysRevLett.109.203901 PRLTAO 0031-9007 (2012). Google Scholar

50.

D. Loterie et al., “Digital confocal microscopy through a multimode fiber,” Opt. Express, 23 23845 https://doi.org/10.1364/OE.23.023845 OPEXFF 1094-4087 (2015). Google Scholar

51.

I. M. Vellekoop, “Controlling the propagation of light in disordered scattering media,” (2008). Google Scholar

52.

T. Čižmár and K. Dholakia, “Shaping the light transmission through a multimode optical fibre: complex transformation analysis and applications in biophotonics,” Opt. Express, 19 18871 https://doi.org/10.1364/OE.19.018871 OPEXFF 1094-4087 (2011). Google Scholar

53.

L. V. Amitonova, A. P. Mosk and P. W. H. Pinkse, “Rotational memory effect of a multimode fiber,” Opt. Express, 23 20569 https://doi.org/10.1364/OE.23.020569 OPEXFF 1094-4087 (2015). Google Scholar

54.

S. Li et al., “Memory effect assisted imaging through multimode optical fibres,” Nat. Commun., 12 –3751 https://doi.org/10.1038/s41467-021-23729-1 NCAOBW 2041-1723 (2021). Google Scholar

55.

J. W. Goodman, Speckle Phenomena in Optics: Theory and Applications, 2nd ed.SPIE Press, Bellingham, Washington (2020). Google Scholar

56.

J. B. Pawley, Handbook of Biological Confocal Microscopy, Springer US, New York (2006). Google Scholar

57.

T. Čižmár and K. Dholakia, “Exploiting multimode waveguides for pure fibre-based imaging,” Nat. Commun., 3 1027 https://doi.org/10.1038/ncomms2024 NCAOBW 2041-1723 (2012). Google Scholar

58.

A. M. Caravaca-Aguirre and R. Piestun, “Single multimode fiber endoscope,” Opt. Express, 25 1656 https://doi.org/10.1364/OE.25.001656 OPEXFF 1094-4087 (2017). Google Scholar

59.

E. E. Morales-Delgado, D. Psaltis and C. Moser, “Two-photon imaging through a multimode fiber,” Opt. Express, 23 32158 https://doi.org/10.1364/OE.23.032158 OPEXFF 1094-4087 (2015). Google Scholar

60.

S. A. Goorden, J. Bertolotti and A. P. Mosk, “Superpixel-based spatial amplitude and phase modulation using a digital micromirror device,” Opt. Express, 22 17999 https://doi.org/10.1364/OE.22.017999 OPEXFF 1094-4087 (2014). Google Scholar

61.

E. E. Morales-Delgado et al., “Delivery of focused short pulses through a multimode fiber,” Opt. Express, 23 9109 https://doi.org/10.1364/OE.23.009109 OPEXFF 1094-4087 (2015). Google Scholar

62.

S. Ohayon et al., “Minimally invasive multimode optical fiber microendoscope for deep brain fluorescence imaging,” Biomed. Opt. Express, 9 1492 https://doi.org/10.1364/BOE.9.001492 BOEICL 2156-7085 (2018). Google Scholar

63.

S. A. Vasquez-Lopez et al., “Subcellular spatial resolution achieved for deep-brain imaging in vivo using a minimally invasive multimode fiber,” Light: Sci. Appl., 7 110 https://doi.org/10.1038/s41377-018-0111-0 (2018). Google Scholar

64.

E. Kakkava et al., “Selective femtosecond laser ablation via two-photon fluorescence imaging through a multimode fiber,” Biomed. Opt. Express, 10 423 https://doi.org/10.1364/BOE.10.000423 BOEICL 2156-7085 (2019). Google Scholar

65.

S. Turtaev et al., “High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging,” Light: Sci. Appl., 7 92 https://doi.org/10.1038/s41377-018-0094-x (2018). Google Scholar

66.

D. Kim et al., “Toward a miniature endomicroscope: pixelation-free and diffraction-limited imaging through a fiber bundle,” Opt. Lett., 39 1921 https://doi.org/10.1364/OL.39.001921 OPLEDP 0146-9592 (2014). Google Scholar

67.

A. Porat et al., “Widefield lensless imaging through a fiber bundle via speckle correlations,” Opt. Express, 24 16835 https://doi.org/10.1364/OE.24.016835 OPEXFF 1094-4087 (2016). Google Scholar

68.

D. Akbulut et al., “Focusing light through random photonic media by binary amplitude modulation,” Opt. Express, 19 (5), 4017 https://doi.org/10.1364/OE.19.004017 OPEXFF 1094-4087 (2011). Google Scholar

69.

J. Bertolotti et al., “Non-invasive imaging through opaque scattering layers,” Nature, 491 232 –234 https://doi.org/10.1038/nature11578 (2012). Google Scholar

70.

O. Katz et al., “Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations,” Nat. Photonics, 8 784 –790 https://doi.org/10.1038/nphoton.2014.189 NPAHBY 1749-4885 (2014). Google Scholar

71.

G. Osnabrugge et al., “Generalized optical memory effect,” Optica, 4 886 https://doi.org/10.1364/OPTICA.4.000886 (2017). Google Scholar

72.

H. Liu et al., “Physical picture of the optical memory effect,” Photonics Res., 7 1323 https://doi.org/10.1364/PRJ.7.001323 (2019). Google Scholar

73.

J. R. Fienup, “Phase retrieval algorithms: a comparison,” Appl. Opt., 21 2758 https://doi.org/10.1364/AO.21.002758 APOPAI 0003-6935 (1982). Google Scholar

74.

N. Stasio, C. Moser and D. Psaltis, “Calibration-free imaging through a multicore fiber using speckle scanning microscopy,” Opt. Lett., 41 3078 https://doi.org/10.1364/OL.41.003078 OPLEDP 0146-9592 (2016). Google Scholar

75.

A. Li et al., “Focus scanning with feedback-control for fiber-optic nonlinear endomicroscopy,” Biomed. Opt. Express, 8 2519 https://doi.org/10.1364/BOE.8.002519 BOEICL 2156-7085 (2017). Google Scholar

76.

U. Weiss and O. Katz, “Two-photon lensless micro-endoscopy with in-situ wavefront correction,” Opt. Express, 26 28808 https://doi.org/10.1364/OE.26.028808 OPEXFF 1094-4087 (2018). Google Scholar

77.

S. Farahi et al., “Dynamic bending compensation while focusing through a multimode fiber,” Opt. Express, 21 22504 https://doi.org/10.1364/OE.21.022504 OPEXFF 1094-4087 (2013). Google Scholar

78.

R. Y. Gu, R. N. Mahalati and J. M. Kahn, “Design of flexible multi-mode fiber endoscope,” Opt. Express, 23 26905 https://doi.org/10.1364/OE.23.026905 OPEXFF 1094-4087 (2015). Google Scholar

79.

R. Kuschmierz et al., “Self-calibration of lensless holographic endoscope using programmable guide stars,” Opt. Lett., 43 2997 https://doi.org/10.1364/OL.43.002997 OPLEDP 0146-9592 (2018). Google Scholar

80.

G. S. D. Gordon et al., “Characterizing optical fiber transmission matrices using metasurface reflector stacks for lensless imaging without distal access,” Phys. Rev. X, 9 041050 https://doi.org/10.1103/PhysRevX.9.041050 PRXHAE 2160-3308 (2019). Google Scholar

81.

W. Choi et al., “Flexible-type ultrathin holographic endoscope for microscopic imaging of unstained biological tissues,” Nat. Commun., 13 4469 https://doi.org/10.1038/s41467-022-32114-5 NCAOBW 2041-1723 (2022). Google Scholar

82.

S. Kang et al., “High-resolution adaptive optical imaging within thick scattering media using closed-loop accumulation of single scattering,” Nat. Commun., 8 2157 https://doi.org/10.1038/s41467-017-02117-8 NCAOBW 2041-1723 (2017). Google Scholar

83.

S. Rosen et al., “Focusing and scanning through flexible multimode fibers without access to the distal end,” (2015). http://arxiv.org/abs/1506.08586 Google Scholar

84.

M. Plöschner, T. Tyc and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics, 9 529 –535 https://doi.org/10.1038/nphoton.2015.112 NPAHBY 1749-4885 (2015). Google Scholar

85.

S. Li et al., “Compressively sampling the optical transmission matrix of a multimode fibre,” Light Sci. Appl., 10 88 https://doi.org/10.1038/s41377-021-00514-9 (2021). Google Scholar

86.

J. Shin et al., “A minimally invasive lens-free computational microendoscope,” Sci. Adv., 5 eaaw5595 https://doi.org/10.1126/sciadv.aaw5595 STAMCV 1468-6996 (2019). Google Scholar

87.

U. G. Būtaitė et al., “How to build the optical inverse of a multimode fibre,” (2022). https://arxiv.org/abs/2204.02865 Google Scholar

88.

A. Abdulaziz et al., “Robust real-time imaging through flexible multimode fibers,” (2022). https://arxiv.org/abs/2210.13883 Google Scholar

89.

D. Loterie, D. Psaltis and C. Moser, “Bend translation in multimode fiber imaging,” Opt. Express, 25 6263 https://doi.org/10.1364/OE.25.006263 OPEXFF 1094-4087 (2017). Google Scholar

90.

D. E. Boonzajer Flaes et al., “Robustness of light-transport processes to bending deformations in graded-index multimode waveguides,” Phys. Rev. Lett., 120 233901 https://doi.org/10.1103/PhysRevLett.120.233901 PRLTAO 0031-9007 (2018). Google Scholar

91.

V. Tsvirkun et al., “Flexible lensless endoscope with a conformationally invariant multi-core fiber,” Optica, 6 1185 https://doi.org/10.1364/OPTICA.6.001185 (2019). Google Scholar

92.

Y. Du et al., “Hybrid multimode - multicore fibre based holographic endoscope for deep-tissue neurophotonics,” Light: Adv. Manuf., 3 408 –416 https://doi.org/10.37188/lam.2022.029 (2022). Google Scholar

93.

R. Barankov and J. Mertz, “High-throughput imaging of self-luminous objects through a single optical fibre,” Nat. Commun., 5 5581 https://doi.org/10.1038/ncomms6581 NCAOBW 2041-1723 (2014). Google Scholar

94.

S. M. Kolenderska et al., “Scanning-free imaging through a single fiber by random spatio-spectral encoding,” Opt. Lett., 40 534 https://doi.org/10.1364/OL.40.000534 OPLEDP 0146-9592 (2015). Google Scholar

95.

D. Stellinga et al., “Time-of-flight 3D imaging through multimode optical fibers,” Science, 374 1395 –1399 https://doi.org/10.1126/science.abl3771 (2021). Google Scholar

96.

W. Xiong et al., “Spatiotemporal control of light transmission through a multimode fiber with strong mode coupling,” Phys. Rev. Lett., 117 053901 https://doi.org/10.1103/PhysRevLett.117.053901 PRLTAO 0031-9007 (2016). Google Scholar

97.

W. Xiong et al., “Principal modes in multimode fibers: exploring the crossover from weak to strong mode coupling,” Opt. Express, 25 2709 https://doi.org/10.1364/OE.25.002709 OPEXFF 1094-4087 (2017). Google Scholar

98.

P. Ambichl et al., “Super- and anti-principal-modes in multimode waveguides,” Phys. Rev. X, 7 041053 https://doi.org/10.1103/PhysRevX.7.041053 PRXHAE 2160-3308 (2017). Google Scholar

99.

W. Xiong, C. W. Hsu and H. Cao, “Long-range spatio-temporal correlations in multimode fibers for pulse delivery,” Nat. Commun., 10 2973 https://doi.org/10.1038/s41467-019-10916-4 NCAOBW 2041-1723 (2019). Google Scholar

100.

M. Mounaix et al., “Time reversed optical waves by arbitrary vector spatiotemporal field generation,” Nat. Commun., 11 5813 https://doi.org/10.1038/s41467-020-19601-3 NCAOBW 2041-1723 (2020). Google Scholar

101.

B. Redding, S. M. Popoff and H. Cao, “All-fiber spectrometer based on speckle pattern reconstruction,” Opt. Express, 21 6584 https://doi.org/10.1364/OE.21.006584 OPEXFF 1094-4087 (2013). Google Scholar

Biography

Jaeyeon Oh received his BSc degree from the Department of Bio and Brain Engineering at Korea Advanced Institute of Science and Technology (KAIST), Korea, in 2022. Currently, he is pursuing an MSc degree in the Department of Bio and Brain Engineering, KAIST, Daejeon, Republic of Korea. His research interests include acoustooptics and nanophotonics.

Chunghyeong Lee received his BSc degree from the Department of Bio and Brain Engineering at KAIST, Korea, in 2020. Currently, he is pursuing an MSc degree in the Department of Bio and Brain Engineering, KAIST, Daejeon, Republic of Korea. His research interests include spectral imaging and nanophotonics.

Gookho Song received his BSc degree from the Department of Electrical Engineering and Biological Engineering at Inha University, Korea, in 2020 and his MSc degree in bio and brain engineering from KAIST, Korea, in 2022. Currently, he is pursuing a PhD in the Department of Bio and Brain Engineering, KAIST, Daejeon, Republic of Korea. His research interests include acoustooptics and computation-based deep tissue imaging.

Mooseok Jang received his BSc in physics from KAIST in 2009 and his PhD in electrical engineering at California Institute of Technology (Caltech), Pasadena, in 2016. He is an assistant professor in the Department of Bio and Brain Engineering at KAIST, Korea. His research interests include optical imaging, complex optics, acoustooptics, and the application of machine learning techniques to microscopic imaging.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Jaeyeon Oh, Chunghyeong Lee, Gookho Song, and Mooseok Jang "Review of endomicroscopic imaging with coherent manipulation of light through an ultrathin probe," Journal of Optical Microsystems 3(1), 011004 (9 January 2023). https://doi.org/10.1117/1.JOM.3.1.011004

Received: 8 August 2022; Accepted: 20 December 2022; Published: 9 January 2023

Access the abstract

JOURNAL ARTICLE
23 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 1 scholarly publication.

Explore citations on Lens.org

KEYWORDS

Matrices

Optical coherence

Optical transmission

Coherence imaging

Optical fibers

Endoscopy

Fluorescence

1.

Introduction

Fig. 1

Table 1

2.

General Properties of Optical Fibers

2.1.

Wave Propagation in Optical Fibers

2.2.

Overview of Optical Fiber Types

2.2.1.

Single-mode fiber

Fig. 2

2.2.2.

Multicore fiber

2.2.3.

Multimode fiber

3.

Transmission Matrix: Formalism for the Coherent Manipulation of Light

Fig. 3

3.1.

Phase Conjugation and Matrix Inversion

4.

Controlling Coherent Waves through Multimode Fibers

4.1.

Optical Focusing

Fig. 4

4.2.

Wide-Field Imaging

4.3.

Confocal Imaging

4.4.

Scanning Fluorescence Imaging

4.4.1.

Fluorescence endomicroscopy

Fig. 5

4.4.2.

Two-photon endomicroscopy

4.4.3.

Biological applications

Fig. 6

Table 2

5.

Controlling Coherent Waves through Multicore Fibers

5.1.

Using Transmission Matrix Formalism

Fig. 7

5.2.

Using Speckle Correlation

Table 3

6.

Advanced Techniques

6.1.

In Situ Transmission Matrix Calibration

6.2.

Using Spatio-Temporal Characteristics of Optical Fibers

7.

Conclusion

Acknowledgments

References

Biography

Show All Keywords

Keywords/Phrases

Search In:

Publication Years