1.

Introduction

Multimodal retinal imaging provides more comprehensive anatomic and functional information of the retina than any single technology alone, thereby benefiting both clinical diagnosis and fundamental investigation of major blinding diseases. Several retinal imaging technologies have already been combined for multimodal imaging. For example, spectral-domain optical coherence tomography (SD-OCT) has been integrated with adaptive optics-based scanning laser ophthalmoscopy (SLO) for both retina cross-section and photoreceptor morphology imaging.¹ Color fundus photography, fluorescein angiography (FA), autofluorescence imaging, and SD-OCT were applied together to study cuticular drusen, soft drusen, and subretinal drusenoid deposits in patients.² And confocal SLO and SD-OCT were used together to study the phenotype of pseudoxanthoma elasticum.³

In our previous work, we demonstrated the integration of photoacoustic ophthalmoscopy (PAOM) with SD-OCT for multimodal imaging of both optical absorption and scattering properties of the retina.⁴ We also demonstrated the integration of photoacoustic microscopy, for non-ophthalmic applications, with confocal scanning laser microscopy using the same illuminating light source by simultaneously collecting the reflected photons.⁵ When adapted to retinal imaging, collecting the reflected portion of the PAOM illumination light, an SLO can be added to the system without increasing retinal light exposure. Furthermore, by adding another light source at 488 nm, FA,⁶ another established ophthalmic imaging technique, can be integrated into the system. In this work, we report integration of PAOM with SD-OCT, SLO, and FA. We also demonstrate the capabilities of this multimodal platform by providing comprehensive anatomic imaging and cross-modality validation.

SLO is an optical scattering-based ophthalmic imaging technology that is widely used in screening, diagnosis, and assessing treatments in patients. Also, SLO is extensively combined with contrast agents, such as fluorescein and indocyanine green (ICG),⁷ to improve imaging contrast and develop functional imaging capability. FA–SLO is a well-established clinical ophthalmic imaging technique that provides diagnosis and treatment information by visualizing the anatomy of and leakage from retinal and choroidal vessels. The identification of abnormal fluorescence patterns using FA allows the discrimination of pathological states associated with choroidal neovascularization (CNV)⁸ and age-related macular degeneration (AMD).⁹ However, the lack of depth (axial) resolution in FA limits the depth accuracy in the evaluation of microvolumes of leakage, and FA imaging cannot be correlated with intrinsic physiological properties.

High depth resolution can be achieved using SD-OCT, which detects the interference between the back-reflected light from the sample and the reference arm based on low-coherence interference. SD-OCT provides cross-sectional retinal imaging at less than 10 μm axial resolution and is invaluable for the diagnosis and management of many severe blinding diseases.¹⁰ Moreover, SD-OCT can measure retinal blood flow by detecting the Doppler shift impinged on the reflected photons.¹¹ The limitation of SD-OCT, however, is that it is not sensitive to the optical absorption properties of the retina that are strongly associated with various physiological parameters, including oxygen saturation (${\mathrm{sO}}_2$) and melanin concentration in retinal pigment epithelium (RPE).

PAOM is an optical absorption-based ophthalmic imaging modality that has the capability to create high-contrast images of retinal and choroidal vessels and melanin by detecting photoacoustic (PA) signals due to the strong optical absorption of hemoglobin and melanin.^4,12,13 Also, PAOM has the potential to map the ${\mathrm{sO}}_2$ in retinal vessels based on spectroscopic measurements.^14–16 Although PAOM and SD-OCT have comparable lateral resolutions, the axial resolution of PAOM is not as good as that of SD-OCT,⁴ which makes PAOM not suitable for visualizing the fine layer structures in the retina.

A multimodal system has the potential to combine the merits of SLO/FA, SD-OCT, and PAOM for a more comprehensive evaluation of the retina based on their complementary contrast mechanisms. By combining SD-OCT with PAOM, a high-resolution cross-sectional anatomic retinal layer structure can be obtained with the high-contrast vascular image from PAOM. More importantly, the future combination of Doppler SD-OCT, which shows retinal blood flow, with spectroscopic PAOM, which provides retinal ${\mathrm{sO}}_2$, can eventually evaluate oxygen metabolism^15,16 in the retina. By combining SLO with PAOM, the PAOM light source will be shared with the SLO without increasing retinal light exposure. While PAOM detects PA signals from the absorbed portion of the illumination photons, SLO will generate a scattering-based image by collecting the reflected portion of the illumination photons, which otherwise would be discarded. By adding a 488-nm laser excitation and detecting the corresponding fluorescence emissions, integration of PAOM, SD-OCT, and FA–SLO becomes possible.

2.

Methods and Materials

2.1.

Experimental System

A schematic diagram of the experimental system is shown in Fig. 1. A frequency-doubled Nd:YAG laser (SPOT-10-100, Elforlight Ltd., Daventry, U.K.; output wavelength: 532 nm; maximum pulse energy: $10\mu \mathrm{J}$; pulse duration: 2 ns; pulse repetition rate: 24 kHz) was used as the illumination source for reflection-SLO and PAOM. The laser output was delivered by a single-mode optical fiber and was further collimated to 2 mm in diameter. A galvanometer (GM, QS-7, Nutfield Technology, Windham, NH) scanned the laser beam and delivered the light onto the retina using a telescope configuration (L1 and L2). In PAOM, the induced PA signals from the retina were detected by an unfocused needle ultrasonic transducer (40-MHz central frequency, 15-MHz bandwidth, $0.5\times 0.5{\mathrm{mm}}^2$ active element size), which was placed in contact with the eyelid coupled by ultrasound gel. The detected PA signals were amplified by 60 dB and were digitized by a data acquisition board (CS14200, Gage Applied Technology, Lockport, IL) at a sampling rate of $200\mathrm{MS}/\mathrm{s}$. The reflection-SLO shared the same illumination laser with PAOM and the reflected light from the retina was detected by an amplified photodiode (PD2, 1607-AC-FS, NewFocus). A high-speed data acquisition board (CS Cobra, Gage Applied) digitized the SLO signals at a sampling rate of $2\mathrm{GS}/\mathrm{s}$. The laser pulse energy was 40 nJ, which was reported to be eye-safe by different groups.^4,17 Because reflection-SLO and PAOM shared the same optical illumination, pulse energy was the same as that for PAOM only.

Fig. 1

Schematic diagram of the integrated PAOM, SD-OCT, and SLO system. SLD: super luminescent diode; SPEC: spectrometer; Ref: OCT reference arm; PD: photodiode; APD: avalanche photodetector; DM: dichroic mirror; HM: hot mirror; GM: 2-D galvanometer; BS: beam splitter; UT: ultrasonic transducer; AMP: amplifier. A long-pass filter (labeled by the dotted line) and an APD were introduced to acquire the fluorescence emission in FA–SLO. The PD2 replaced the APD and the filter was removed in reflection-SLO.

For FA–SLO, we added a CW 488-nm argon-ion laser (161C-410-21, Spectra Physics) as the excitation source. A dichroic mirror (T495lpxr, Chroma Technology, Bellows Falls, VT) was used to combine the 532 and 488-nm light. After further linking up the SD-OCT sample arm by a hot mirror (FM02, Thorlabs, Newton, NJ), the three beams shared the same scanning and delivery mechanisms as described above. The emitted fluorescence light passed through a long-pass filter (FEL0500, Thorlabs; cut-off wavelength: 500 nm; labeled by the dotted line since it was not used in reflection-SLO), and was detected by an avalanche photodetector (APD, APD110A2, Thorlabs). Due to laser safety concerns, we performed FA–SLO and PAOM sequentially rather than simultaneously.

A fiber-based SD-OCT was integrated with PAOM and SLO as previously detailed.¹⁸ Briefly, the SD-OCT consists of a broadband light source (IPSDD0804, InPhenix, Livermore, CA; center wavelength: 840 nm; 6-dB bandwidth: 50 nm), a $50\times 50$ single-mode fiber coupler, a reference arm, a sample arm coupled with the PAOM by a hot mirror, and a home-built spectrometer. The exposure time of the line scan CCD camera (Aviiva SM2, e2v) in the spectrometer was $36\mu \mathrm{s}$, which gave an OCT A-line rate of 24 kHz. We carefully aligned the optical illuminations of PAOM, SD-OCT, and FA–SLO to be coaxial so that images from all modalities were laterally registered.

The axial resolution of PAOM and SD-OCT was previously quantified to be 23 and $6\mu \mathrm{m}$, respectively.⁴ The reflection- and FA–SLO provided only poor axial resolution because no pinhole was used to eliminate the out-of-focus signals and the effective numerical aperture of a rat eye is very low. Both SD-OCT and PAOM were shown to have comparable lateral resolution ($\sim 20\mu \mathrm{m}$) when imaging the rat retina in vivo,⁴ and both reflection-SLO and FA–SLO were expected to have comparable lateral resolution with SD-OCT and PAOM, with a minor difference caused by the differences in optical wavelengths.⁵

3.

Results and Discussion

In the reflection-SLO experiments, the in vivo fundus images are shown in Fig. 2, where the images in the left column are acquired from an albino rat and the images in the right column are from a pigmented rat. In PAOM [Fig. 2(a)], the PA signal amplitudes generated from retinal vessels are high (bright) due to the strong optical absorption of hemoglobin; in comparison, the retinal vessels appear dark in the SLO image [Fig. 2(b)] because fewer photons were reflected. Since PAOM and SLO shared the same optical illumination but detected different groups of photons (absorbed photons versus. reflected photons), the two modalities show opposite contrasts. Additionally, PAOM demonstrates the capability of visualizing choroidal vasculature [CV in Fig. 2(a)] at high contrast due to the minimal melanin concentration in the RPE of albino eyes. The reflection-SLO is not capable of visualizing the subretinal vascular network because of strong optical reflection from the retinal nerve fiber layer. In the en face SD-OCT fundus image [Fig. 2(c)], similar backscattering contrast with SLO is revealed and the difference between Fig. 2(b) and 2(c) is primarily caused by the different illumination wavelengths (532 nm in SLO and 840 nm in SD-OCT). Because SD-OCT and PAOM/SLO illumination light were well aligned before in vivo imaging,⁴ the SD-OCT image is registered with both the PAOM and SLO images.

Fig. 2

In vivo multimodal retinal imaging of an albino rat (left column) and a pigmented rat (right column) when the SLO worked in the reflection mode. Panels (a) and (d) are PAOM fundus images; panels (b) and (e) are reflection-SLO images; and panels (c) and (f) are en face SD-OCT images. RV: retinal vessels; CV: choroidal vessels. Bar: $500\mu \mathrm{m}$.

Figure 2(d) to 2(f) are images acquired simultaneously in a pigmented rat eye, where the differences among the three modalities are similar to those seen in the albino eye. When we compare the results between albino and pigmented eyes, the differences are in the visualization of the RPE and choroid. In PAOM images of the albino eye, vessels in both the retina and choroid can be observed, although the choroidal vessels are not clearly resolved. In the pigmented eye, the high melanin concentration in the RPE strongly absorbs the illumination light of PAOM, which creates high-amplitude PA signals and, consequently, permits the visualization of melanin in RPE. On the other hand, the PAOM laser cannot penetrate through the RPE due to its strong optical absorption and, as a result, the vessels in the choroid cannot be imaged. For a similar reason, SLO shows worse contrast in the pigmented eye than in the albino eye because the RPE is highly absorptive in the pigmented eye. The bright spots at the center of both reflection-SLO images are caused by the specular reflection from the L2 (Fig. 1) and can be removed by optimizing the optical setup in the future.

Figure 3 shows the imaging results with FA–SLO. In both the albino eye (left column) and the pigmented eye (right column), the image quality from PAOM and SD-OCT are comparable to those in Fig. 2. However, when we compare the FA–SLO images [Fig. 3(b) and 3(e)] and the reflection-SLO images [Fig. 2(b) and 2(e)], the signal-to-noise ratio was improved by 26 dB as a result of the fluorescence detection. Additionally, the FA–SLO image of the pigmented eye [Fig. 3(e)] shows a 15-fold increase in the contrast-to-background ratio compared to that of the albino eye [Fig. 3(b)] because the background fluorescence emission from the choroid is absent due to the strong optical absorption by the melanin in the RPE of the pigmented eye.

Fig. 3

In vivo multimodal retinal imaging of an albino rat (left column) and a pigmented rat (right column) when the SLO worked in the FA mode. Panels (a) and (d) are PAOM fundus images, panels (b) and (e) are FA–SLO images; and panels (c) and (f) are en face SD-OCT images. RV: retinal vessels; CV: choroidal vessels. Bar: $500\mu \mathrm{m}$.

In Fig. 3(e), we can observe some bright spots (indicated by arrows), which may be caused by the vertical capillaries connecting two horizontal capillary layers in the retina.^19,20 In FA–SLO, fluorescence emission accumulates along these vertical capillaries and generates higher fluorescence amplitude than parallel capillaries in the fundus image. In contrast, dark spots can be observed in the SD-OCT image [Fig. 3(e)] from the corresponding locations because these vertical capillaries attenuate the SD-OCT probing light more. In PAOM [Fig. 3(d)], we also can observe similar effects from the vertical capillaries on the fundus image; however, they are not as obvious as in the SD-OCT fundus image.

Although both PAOM fundus [Figs. 2(d) and 3(d)] and SD-OCT fundus [Figs. 2(f) and 3(f)] images have bright backgrounds when imaging pigmented eyes, the background signals come from different sources. PAOM is an optical absorption-based technology; as a result, the background may contain contributions from RPE melanin, the choroidal vascular network, and choroidal melanin. Due to the poor axial resolution of PAOM ($32\mu \mathrm{m}$), we cannot identify the contribution from every possible source, and additional studies are necessary. Although there is cell-to-cell variation in RPE melanin content at the microscopic level,²¹ the current PAOM cannot resolve such variations due to limited lateral resolution. SD-OCT is an optical scattering-based technology and its near-infrared probing light is capable of penetrating deeper into the retina than the visible light used in PAOM. Thus, we hypothesize that the “streaky background” in Figs. 2(f) and 3(f) are caused by major choroidal vessels.^22,23

To demonstrate further the complementary nature of these three modalities, a PAOM B-scan, 1D FA–SLO profile, and SD-OCT B-scan are compared in Fig. 4. The dashed line in Fig. 3(d) indicates the locations of the 1-D profile and B-scan images in their corresponding original images. The PAOM B-scan [Fig. 4(a)] specifically imaged five individual retinal vessels and the RPE due to strong optical absorption of hemoglobin and melanin. Unlike clearly resolved multiple retinal layers in the SD-OCT B-scan, other anatomic layers except for the vasculature and the RPE were invisible in the PAOM B-scan due to their low optical absorption coefficients²⁴ and poor axial resolution in the PAOM.⁴ The 1-D profile in FA–SLO [Fig. 4(b)] is an integration of all the fluorescence emissions along the depth at each illumination position in the retina. The fluorescence intensity peaks correspond to retinal vessels, where the local concentration of fluorescein is the highest. The SD-OCT B-scan [Fig. 4(c)] resolved the anatomic layers well based on the different optical scattering properties in each retinal layer, but demonstrated low contrast in visualizing the retinal vessels as a result of the back-scattering-determined contrast mechanism. The vertical lines in Fig. 4 highlight the locations of the retinal vessels in each modality and further demonstrate the spatial registration among them.

Fig. 4

Comparison of (a) PAOM B-scan, (b) 1D FA profile, and (c) SD-OCT B-scan acquired from the same spatial location [indicated by the dashed line in Fig. 3(d)] in a pigmented eye. Bar: $100\mu \mathrm{m}$.

When we compare the PAOM and FA–SLO images, much finer details of the retinal capillaries can be observed in the FA–SLO image than in the PAOM image. Figure 5 shows the magnified views of the regions highlighted by the dashed box in Fig. 3(d). In both PAOM [Fig. 5(a)] and FA–SLO [Fig. 5(b)] images, the major retinal vessels with a diameter of $\sim 100\mu \mathrm{m}$ can be observed with high contrast; however, only the FA–SLO visualized the capillary network, as indicated by the group 1 arrows in Fig. 5(b). Although the capillary network itself was not imaged by PAOM, some shadows created by the capillaries can be observed and correspond well with the imaged capillaries by the FA–SLO, as pointed out by the group 2 arrows. Because we used an unfocused ultrasonic transducer in PAOM, only a limited field of view (FOV) can be achieved, and the detection sensitivity drops towards the periphery of the FOV, which leads to a low contrast-to-noise ratio in the peripheral area.²⁵ Since we tried to place the center (most sensitive region) of the FOV around the optic disk, PAOM did not resolve the shadows created by the capillaries (pointed out by the group 1 arrows), which are farther from the optic disk.

Fig. 5

Comparison of (a) PAOM and (b) FA–SLO images within the region outlined by the dashed square in Fig. 3(d). Bar: $100\mu \mathrm{m}$.

A more interesting finding in comparing PAOM with FA–-SLO is highlighted by the group 3 arrows. The dark region in the PAOM image potentially can be attributed to a lower melanin concentration in the RPE. This hypothesis is supported by the FA–SLO, where the dark region in the PAOM image becomes bright. The reason is that when the melanin concentration is high in the RPE, the fluorescence excitation light is significantly attenuated and the fluorescence emission from the choroid is blocked and cannot be detected; however, in RPE regions with low melanin concentration, more fluorescence emission can be generated and detected, which leads to high signal amplitudes in the FA–SLO, as shown in Fig. 5(b).

A question worth asking is why PAOM cannot visualize retinal capillaries in the same detail as FA-SLO does. In retinal imaging, PAOM and FA–SLO have comparable lateral resolution since in both modalities the lateral resolutions are determined by the effective numerical aperture of the subject eye, and the optical wavelengths of PAOM (532 nm) and FA–SLO (488 nm) are not dramatically different. One possible reason is that in the pigmented eye, FA–SLO has much lower background signal than PAOM because of the presence of RPE melanin; however, even in the albino eye, where the PAOM background is low, PAOM does not display a densely packed capillary network. We hypothesize that such a discrepancy is caused by the low ultrasound detection sensitivity in PAOM. With a finite lateral resolution, a capillary can occupy only a single pixel in PAOM, and its PA amplitude is usually much lower than the PA amplitudes of the major retinal vessels. As a result, the PA signals from capillaries may fall into the noise level of the ultrasound detector and cannot be imaged. A potential solution is to employ a novel ultrasound detector with much lower noise equivalent pressure values, such as the optical micro-ring resonator.²⁶ Such a detector also has a much wider detection directivity than a traditional ultrasonic transducer,²⁶ which would offer a larger FOV with more homogeneous sensitivity distribution in PAOM. Although FA–SLO detected a more detailed retinal vasculature than PAOM, fluorescein sodium may cause side effects in some patients,⁶ and FA–SLO is not capable of measuring ${\mathrm{sO}}_2$ due to its fluorescence-based contrast mechanism.

4.

Summary

In conclusion, a multimodal system that provides complementary contrast mechanisms was developed for comprehensive anatomic and potentially functional retinal imaging. The system integrated SD-OCT and PAOM with both reflection-SLO and FA–SLO to provide optical absorption, optical backscattering, and fluorescence properties of the retina. High-quality in vivo images were acquired from both albino and pigmented rat eyes using all modalities, and the complementary optical contrasts were demonstrated. Such a multimodal system is particularly important for the future development of PAOM to facilitate the evaluation of PAOM in comparing to established modalities to maximize its potential values in both fundamental investigation and clinical use in detecting blinding diseases.