Since the invention of the ophthalmoscope by von Helmholtz1 enabled the first transformative drawings of the retina, the development of retinal fundus photography includes few fundamental advances. The principles of indirect ophthalmoscopy2 and the Gullstrand principle of spatially separating illumination and imaging ray paths in the pupil3 underpin modern fundus camera design.4,5 The incorporation of digital technology has enhanced overall functionality, but a fundamental advance is certainly the invention of the scanning laser ophthalmoscope (SLO).22.214.171.124.–11
The principle behind the SLO is to scan a focused illumination spot across the retina while recording the intensity of the backscattered light transmitted through the pupil. Importantly, this enables the inversion of the Gullstrand principle, employing a small part of the pupil for the illumination and dedicating the remainder of the pupil for capturing the light scattered by the retina. This allows a large pupil area to be dedicated to light detection,6 which is of great benefit given the low reflectivity of the retina and the limited illumination intensity that may safely illuminate the retina.1213.–14 An important distinction of the SLO from the conventional retinal fundus camera, which employs simultaneous en face illumination and imaging of the whole retina, is that the SLO involves illumination and detection of light for only a small area in the retina at any instant, and this enables better control of stray light and consequently higher image contrast but at the expense of reduced optical efficiency. We report here a modification of the SLO that yields images that combine high contrast and high optical efficiency in a single instrument.
The retina exhibits a large, spatially variant and wavelength-dependent tissue point-spread function (PSF) that, at red wavelengths, can be two orders-of-magnitude larger than the optical PSF, reducing the contrast of retinal images recorded by both the SLO and the fundus camera. An important development of the SLO is the incorporation of a confocal aperture located in the image plane to spatially filter the light transmitted to the photodetector.8,10 Only light scattered from a small, axially elongated volume close to the focus of the illumination is then detected and light from the extended tissue PSF, consisting principally of multiply scattered light, is rejected. Images recorded by confocal SLOs are, therefore, typified by lower levels of optical scattering and increased image contrast but lower optical efficiency than a conventional SLO. Although the increase in contrast can be approximately understood in terms of an extended tissue PSF introducing increased blurring, for imaging of the vasculature it is more useful and accurate to employ a physical model involving multiple light paths of near-ballistic photons propagating through three-dimensional (3-D) retinal structures:15,16 images of the retinal vasculature recorded by a confocal SLO are formed by light that is transmitted twice through the retinal vessels while for a nonconfocal SLO, the dominant optical component is transmitted just once through a vessel followed by backscattering from an extended tissue volume around the vessel that is imaged onto the detector. Vasculature in images recorded with a confocal SLO consequently exhibit up to twice the contrast of images recorded with a conventional SLO.
However, the confocal SLO requires the imaging return path to be well-corrected for all imaging fields, accounting for the combined effect of ocular and instrument optical aberrations, such that the width of the system optical PSF does not exceed the size of the confocal aperture; not meeting this condition yields a substantial reduction in signal-to-noise ratio (SNR) and contrast. For this reason, SLOs typically require adaptive optics to enable tight degrees of confocality. An alternative to the confocal mode is to operate the SLO with a larger (nonconfocal) aperture to provide a sufficient SNR albeit with a reduction in image contrast, which is referred to as relaxed-confocal or direct mode.6,11 Note that the aperture (direct or confocal to an even greater extent) also enhances image quality as it blocks some reflections that occur at ocular and instrument surfaces. Finally, an annular aperture offers the complementary function of recording only the scattered light, rejecting the direct back-reflection contribution; this mode of operation is called indirect, scatter, dark-field, or Tyndall-mode.8,10,11,13,1718.19.–20
Several researchers have reported on the benefit provided by the use of different pinhole apertures as spatial filters,13,1718.–19,2122.–23 including multichannel imaging,20,23,24 however, confocal imaging with small pinholes is limited to relatively small fields of view of or an order-of-magnitude less than this for diffraction-limited imaging achievable with adaptive-optics correction.2526.27.–28 The effect of optical aberrations on the image of the retina is much more severe than for the illuminating beam since image formation uses the whole pupil, so the numerical aperture for the imaging return path is typically 10 times greater than for the illuminating beam yielding commensurately higher levels of aberration. Consequently for field angles greater than about 30 deg, the effective imaging PSF, including the effects of an extended retinal-tissue PSF, exceeds the dimensions of the imaged pinhole preventing confocal operation. Adaptive-optic correction of these field-dependent aberrations is not practical since the scan speeds exceed the bandwidth of available adaptive-optic systems. Therefore, for wide-field imaging, exceeding 100 deg, high-quality SLOs are able to maintain a reasonably low aberration, low numerical aperture spot for the illumination path; but the high contrast typical of confocal operation has not been previously possible for wide-field imaging.
We report here the first demonstration of a hybrid pinhole technique, where a ultrawide-field SLO operates simultaneously in direct, indirect, and confocal modes to provide the ability to optimally combine the high resolution and contrast of confocal imaging with the high sensitivity and aberration tolerance of direct imaging. The proposed solution directs the direct and confocal light components to independent detectors to enable a spatially optimized reconstruction of the retina that retains a wide field of view while increasing image contrast. We demonstrate experimentally, using a commercial SLO, a doubling of contrast in the central retina macula region while retaining the native ultrawide-field, of up to 200 deg, and high SNR of the conventional SLO.
The objective of our hybrid pinhole technique is to employ confocality to increase the contrast of images recorded by an SLO with direct-mode operation, in particular, in ultrawide-field imaging, in which optical aberrations restrict the field of view of the confocal mode. Increased contrast and control of light paths is important to enable accurate chemical sensing such as in vascular oximetry measurements at the retina where it promises reduced sensitivity to pigmentation, specular reflections, and variations in vessel diameter. In this section, we describe the assessment of the contrast increase by: (a) simulation using Monte-Carlo modeling of the retinal light paths involved in imaging and (b) validation measurements of the contrast of in vitro images of blood-filled capillaries located at the retinal plane of an artificial eye (as a realistic and well-controlled target). These experiments were conducted for a range of aperture diameters.
We used an approach of optically modeling the complete optical system including the optical instrument and light propagation through the retina using commercial optical ray-trace software. We implemented our model using OpticStudio from Zemax LLC, although the principles are applicable with other commercial ray tracers. The technique exploits the near equivalence of the deterministic ray tracing employed in commercial optical design programs such as Zemax and the statistical ray tracing employed in Monte-Carlo modeling to enable a very efficient means of performing Monte-Carlo simulation of the complete image-formation process,29 integrating polarization-sensitive ray tracing of the SLO and ocular optics with Monte-Carlo modeling of light propagation in the retinal layers within a single holistic model. We employed a simplified and generic model of an SLO rather than the specific optical design of the commercial SLO in this simulation; however, this has no significant impact on the results presented. The model of the human ocular optics includes the anterior segment with cornea, anterior chamber, iris aperture, and lens; we modeled the eye fundus with the following retinal layers:30,31 retina, retinal pigment epithelium, choroid, and sclera; and several cylindrical blood vessels (filled with fully oxygenated blood) were embedded at a depth of within the retina. The thickness of the retina varies with field angle and from subject to subject,3233.–34 as does the thickness of the choroid.3536.–37 For the purpose of this investigation, we assumed thicknesses of 200 and , respectively, as reasonable and typical values for a Monte-Carlo simulation as have been used elsewhere.30,31 For pupil diameters in the range 2 to 6 mm, the form of modeled and recorded images is dominated by optical scattering within tissue, rather than diffraction or optical aberrations, and so we have found no significantly effect of pupil size on modeled images. However, the higher number of rays captured by a larger pupil yields a higher SNR for a given computation time and so we have assumed a pupil diameter of 6 mm. The simulation experiment is done with a complete optical model that includes 3-D crystalline lens, cornea, and retinal components such as blood vessels, etc. The layout of the modeled system and eye are depicted in Fig. 1, and parameters used for the simulation are summarized in Table 1. The scanning illumination beam at the retina was modeled as an injected beam of photons with a Gaussian transverse spatial profile with a half width ( intensity) of at the retina. A total of photons were launched, traced, and scattered through the system, and a 50-mm focal-length lens was placed in front of the eye to focus the backscattered photons emerging from the eye pupil onto the detector. The simulation was repeated for each scanning angle to render an image. A generic pinhole aperture was implemented as a pixelated detector and spatial integration of the incident light enabled simulation of a range of aperture dimensions.
Properties of the retinal layers used for simulation; thickness T, anisotropy factor g, scattering parameter μs, and absorption parameter μa are shown for wavelengths λ=532 nm and λ=633 nm. Parameters were taken from previously published data30,38–41 except for choroid at λ=633, in which blood (vessel) properties were assumed.
|Layer||T (μm)||λ=532 nm||λ=633 nm|
|g||μs (mm−1)||μa (mm−1)||g||μs (mm−1)||μa (mm−1)|
Simulated images of blood vessels generated by Monte-Carlo modeling, together with quantitative assessments extracted from such images are shown in Fig. 2. The left column corresponds to , in which the absorption by hemoglobin has low sensitivity to oxygenation, and the right column corresponds to , in which absorption decreases with increasing oxygenation. Images in (a) and (b) and (c) and (d) are the simulation of the scans for direct mode and confocal mode, respectively, employing a relaxed-confocal aperture size of and a more confocal aperture of (diameters of the apertures projected onto the retina). The images include four cylindrical vessels, as shown in Fig. 1, with diameters of 20, 40, 60, and , corresponding to retinal vessels ranging from retinal capillaries to the larger arterioles and venules. Normalized intensity profiles (averaged along the vertical direction) of the vessels are plotted in (e) and (f) for a range of vessel diameters and aperture diameters. It can be readily appreciated that contrast is increased by reducing the pinhole aperture diameter (calculated by integrating signal from a smaller detector circular area) at the expense of reducing signal. The variations in calculated contrast as a function of the pinhole size are shown in the lower graphs (g) and (h)
We now describe the enhancement of contrast for in vitro imaging of blood-filled capillaries within an artificial eye imaged using a modified ultrawide-field SLO (Optos 200Tx). The phantom eye, shown in Fig. 3(a), consists of a lens and an artificial retinal plane separated by one focal length immersed in an index-matching medium (water was used in this case). The retinal plane is composed of Spectralon, which simulates the combined volume-scattering nature of the retinal layers (as an approximation to the layered structure of the retina). A quartz capillary with inner diameter and outer diameter is located in front of the Spectralon surface (with center at from the surface), simulating the geometry of a retinal vessel.42 The capillary was filled with fully oxygenated horse blood.
An image of this phantom vasculature recorded using the SLO with illumination light at wavelength of 532 nm is shown in Fig. 3(b), and intensity profiles across the vasculature are shown in Fig. 3(c) for diameters of pinhole aperture varying between 50 and . The intensity profiles were normalized to the measured background intensity of light scattered by the Spectralon. The trade between vascular contrast and optical sensitivity is clear from Fig. 3(d), which summarizes the calculated contrast against signal strength for various sizes of the pinhole aperture. Signal strength values are normalized with respect to the largest pinhole, which corresponds to direct mode, in which the illuminating light scattered into an extended volume within the Spectralon is imaged onto the pinhole aperture. The main contrast mechanism is then single-pass transmission of light through the capillary prior to incidence upon the Spectralon yielding a lower contrast. As the diameter of the pinhole is reduced, confocality is increased and the scattered light transmitted by the aperture is increasingly dominated by double-pass light transmission through the capillary, and an increase in contrast15 is observed that is accompanied by a 12-dB reduction in signal transmitted through the pinhole. This well-controlled in vitro experiment, mimicking the in vivo imaging of retinal blood vessels, confirms the increase in contrast with increased confocality.
In addition to the reduced signal associated with confocality, the confocal signal is suppressed at large field angles as increasing optical aberrations on the return path increase the PSF, thus limiting the field of view of the confocal SLO. In Sec. 3, we present a method to increase contrast at the central retina while maintaining wide-field imaging and report experimental validation for imaging of human retinas.
Multimode Aperture: Hybrid Pinhole
We describe now the hybrid pinhole configuration that enables imaging in confocal, direct, and indirect modes simultaneously. The hybrid pinhole consists of a reflective chrome-on-glass annulus: light transmitted through the annulus, which is the confocal light, is imaged onto the confocal detector while the reflected indirect light by the chrome is imaged onto a second indirect detector. The acquisition is simultaneous, so images from the confocal and indirect channels are inherently registered and the pixel-wise addition of these images corresponds to the direct mode. This multimode aperture enables the benefits of high-contrast imaging for a confocal SLO to be combined with the increased SNR and wide field of the direct-mode SLO.
A comparative illustration of the three modes of operation is shown in Fig. 4(a). The SLO with a single aperture can operate only in a single mode: either direct, indirect, or confocal depending on whether the aperture is large, annular, or small. It is also shown in Fig. 4(a) how the larger aperture collects light that is scattered within an extended volume at the retina, a small aperture transmits only a confocal, predominantly backscattered component, and an annular aperture transmits only the indirect light scattered away from the focused spot in the retina. The hybrid pinhole is shown in Fig. 4(b) where the confocal light is transmitted through the annulus for the confocal channel and the scattered light is reflected by the annulus for the indirect channel. The reconstructed image from the hybrid pinhole configuration is an agile, spatially variant weighted sum of the signal from the confocal, and indirect channels to enable spatial optimization of contrast and optical sensitivity. The tilted confocal chrome-on-glass pinhole was slightly elliptical so as to appear circular to the incoming beam. Absorptive, black felt with a rounded hole was adhered to the glass substrate to implement the outer aperture of the annulus as shown in Fig. 4(c). The diameters of the outer and inner apertures were selected to be 1500 and , respectively, ( and 13 times the size of the illumination spot projected at the pinhole plane); the former, corresponding to the direct-mode native aperture of the SLO, and the latter is a compromise between optical sensitivity and contrast, as highlighted in Fig. 3(d). Reducing the size of the pinhole to achieve greater confocality yields only modest increases in contrast of the vasculature: from 91% to 99% but with the penalty of an 8-dB reduction in signal strength. The slow degradation in contrast of the vasculature with reduced confocality is associated with the extended nature of the vasculature yielding a slow transition from double pass of light through the vasculature for confocal mode to single pass for direct mode.
The hybrid pinhole described in Sec. 3 was implemented in a direct-mode, wide-field SLO (Optos 200Tx), and retinal images were recorded to experimentally demonstrate the simultaneous increase in image contrast and ultrawide view. The SLO attains an on-axis resolution of , and the time for a complete retinal scan is . The instrument is designed to be nonmydriatic, and therefore, no pupil dilation was used. The illumination of the retina is unchanged from the conventional clinical instrument, safety mechanisms were left in place, and so the safety standards of the unmodified instrument were maintained. Although the device was modified only in the imaging return optical path, we followed the procedure described by the manufacturer to verify that all safety mechanisms of the device, which is Conformité Européenne approved for clinical use, were functioning normally. Therefore, the device was guaranteed to remain eye safe during our experiments. All participants who volunteered for examination were informed on all aspects of the study and provided informed consent. The study was approved by the Research Ethics Committee of the College of Science and Engineering at the University of Glasgow.
We synergistically combine the confocal and indirect images to generate the multimode imageFig. 5, we show images and in (a) and (b). The calculated weighting function is shown in (c) and the multimode image is shown in (d). The strong corneal-reflex artifact apparent in the center of Fig. 5(b) occurs due to the absence of the optical spatial-filter on this channel. The SLO incorporates a corneal reflex blocker that, in its direct-mode imaging, suppresses the reflection, but it was not required to replicate this blocking function in the indirect channel of the modified SLO since it is strongly attenuated by , that is the reflection only occurs where the confocal signal is strong and this reflection has little effect on the final reconstruction as is appreciated in Fig. 5(d). Nevertheless, the area where this reflection is strong was detected by thresholding the image from the indirect channel, and was set to unity in this region, resulting in a central disc that can be discerned in the example weighting function depicted in Fig. 5(c). Note that for the larger vertical field angles, in which higher optical aberrations increase the size of the optical PSF, the confocal signal is reduced in magnitude and the indirect channel resembles direct-mode operation, as is apparent from the higher intensity in the peripheral regions in Fig. 5(b). In this paper, we, therefore, compare results for multimode operation using the hybrid pinhole with direct-mode operation using a conventional direct-mode pinhole; that is, we compare the modified SLO with the unmodified direct-mode SLO.
Representative conventional and multimode images of a single retina are shown in the upper and lower rows, respectively, of Figs. 6(a) and 6(b). Figure 6 shows images recorded at a wavelength of (a) 532 nm and (b) 633 nm. The highlighted areas in the left-most images are shown in a larger scale on the right. It is apparent that the multimode images exhibit greater contrast and are sharper at both wavelengths due to the increased confocality. Note also that in the 633-nm images, arterioles that run from the center of the magnified images toward the bottom right (highlighted with an arrow) are visible in the multimode image, but not in the conventional image. This is mainly due to the increased contrast arising from the double-pass transmission associated with confocal operation.
Representative examples of intensity profiles across arterioles and venules from these images are plotted in Fig. 7, showing a clear increase in the vascular contrast by use of the hybrid pinhole. Note for instance that the cross-sectional intensity profile for the vessel highlighted by the arrow in Fig. 6(b) corresponds with vessel labeled #1 in Fig. 7(b), which is a clear example of the benefit provided by the higher degree of confocality. The parameters within the plots are the calculated contrast for the direct-mode (red plots) and multimodal images (blue plots). Using the approximation of the Beer–Lambert law (that is, assuming near ballistic photons and neglecting scattering), one might expect a doubling in effective path length associated with confocality to double the optical density of the vasculature, which for typical direct-mode vascular contrast of corresponds to an approximate doubling in contrast. We observe an average increase in the contrast of a factor of for the larger-caliber veins and arteries by use of the hybrid pinhole (mean and standard deviation from various profiles at both 532 and 633 nm wavelength), in reasonable agreement with this expectation.
Intensity profiles from images of the Monte-Carlo simulation and in vitro imaging of blood-filled capillaries shown in Figs. 2 and 3 were averaged vertically to reduce noise (as vessels or capillaries are oriented vertically). In order to provide an equivalent assessment, the intensity profiles shown in Fig. 7 were calculated by the following procedure. First, a region, in which the vessel showed no curvature, was manually segmented ( along the vessel depending on its geometry). Forty different intensity profiles were then extracted perpendicular to the vessel direction, at equispaced intervals along the length of the vessel segment. The profiles were ultimately averaged to enhance SNR and smooth retinal texture.
The contrast of the vasculature varies with wavelength and blood oxygenation: at 532 nm, the absorption coefficient of blood shows only a small dependence on oxygenation, and so the contrast of veins and arteries of similar caliber are quite similar, whereas at 633 nm, the extinction coefficient of highly oxygenated arterial blood is much lower than for the partially deoxygenated venous blood and so the contrast of retinal arterioles is much lower.43 In consequence, for vessels #1 and #2 in Fig. 7 (arterioles), the contrast is insufficient to be observed reliably at 633 nm in the direct-mode SLO, but with the multimode SLO, the contrast is sufficiently increased to enable reliable manual tracking.
As illustrated by the examples shown in Fig. 6, the multimode operation provides imaging with increased contrast, which is particularly evident for blood vessels imaged at 633 nm, in which some arterioles are discernible only in the multimode images. The inner aperture of the hybrid pinhole provides a higher suppression of reflexions from both ocular and instrument surfaces, and this increases the contrast of the images, but the main the source of increased contrast is due to the rejection of light that has diffused into neighboring volumes from the illumination spot at the retina. The consequent approximate doubling in vessel contrast demonstrated here represents a suppression of the effects of retinal texture and an improvement in the quality of information recorded by the retinal scans.15
Multimode images are visually sharper than conventional direct-mode images because of the reduced light paths involved in image formation, but noise is higher because of two main factors: first, in the central region where the confocal counterpart is stronger, the multimode resembles a confocal image and unavoidably fewer photons are detected; second, the implementation of a hybrid pinhole using imperfect chrome-on-glass (55% reflectivity) patterned on a glass substrate (90% transmissivity) introduced a further 10% reduction of the confocal signal and a 45% reduction in the indirect signal. This is the reason why multimode images shown in Fig. 6 also have a reduced SNR in the periphery. While the SNR reduction for the confocal imaging can be deemed unavoidable for a direct-mode optimized instrument, optical losses at the hybrid pinhole can be readily mitigated with a more efficient implementation.
We have implemented a two-channel hybrid pinhole in order to combine confocal and relaxed-confocal signals. The concept can be readily extended to -channel imaging, for example, by detecting light on concentric rings at the pinhole plane. This would capture a 3-D data cube of the retina (for a single illumination wavelength) where the extra dimension corresponds to the size of the pinhole aperture. This is indeed the case for the Monte-Carlo simulation in Sec. 2. This would offer the possibility to adjust the compromise between contrast and SNR in postdetection. Previous research have reported the use of annular apertures for narrow field of view ()13,1718.–19,2122.–23 or the combination of information from differently shaped apertures for very narrow field of view ();20,23,24 our multimode implementation provides a wide-field capability (the native field of view of the SLO, of up to 200 deg) and high-contrast imaging simultaneously.
We report a hybrid imaging modality of a conventional SLO for simultaneous imaging in confocal and direct modes that enables recording of retinal images with increased contrast for the central field while retaining wide-field imaging with high sensitivity. We have validated the technique using Monte-Carlo modeling and demonstrated the technique experimentally for in vitro imaging and for imaging the retinas of volunteers using a wide-field SLO. The measured contrast of the vasculature over the macula was increased by typically a factor of two compared to the conventional direct mode.
The method is based on the simultaneous acquisition of two independent images, and different information are captured in these images. Multichannel detection schemes have been demonstrated before either as a two-channel or time-sequential multichannel in a very narrow field of view by using diffraction-limited adaptive-optics SLO to gain great insight into the retina structures.20,24 Our emphasis is here in the higher contrast achieved on vasculature and other retinal texture while in the ultrawide-view imaging regime, with potential for enhanced clinical detection of small features ranging from blood vessels, exudates, etc. In particular, since the method provides confocal imaging with increased control of light paths in the central fields of the images,15 it offers prospects for enhanced quantitative chemical sensing in the retina, including retinal oximetry with improved accuracy.
G. M. and J. H. are employees of Optos plc, who partially funded this work. The device used for demonstration is the 200Tx model from Optos plc. No patents have been filed on the reported technology. Authors declare no financial interest in the manuscript.
Authors would like to thank Dr. J. Fernandez Ramos for helpful input regarding optical parameters of the ocular tissues involved in the Monte-Carlo simulations. Authors are grateful for funding from Engineering and Physical Sciences Research Council (EPSRC) (No. EP/K503903/1), Innovation Centre for Sensor and Imaging Systems (CENSIS) (No. 69196), and Optos plc (No. 69222). G. C. thanks the Leverhulme Trust for the support.
Guillem Carles obtained a BSc and MSc degrees in physics from the University of Barcelona, MSc degree in computer vision and artificial intelligence from the University Autonomous of Barcelona, and PhD in physics on computational imaging from the University of Barcelona in 2011. He moved to University of Glasgow in 2012 and conducted postdoctoral research in computational imaging aspects including multiaperture optics, retinal imaging and 3-D microscopy. He is a Leverhulme Trust Early Career fellow.
Jano van Hemert is the research director of Optos plc (a Nikon Company), a medical device company specialising in retinal imaging. He has an MSc and a PhD degrees from Leiden University. He is a senior member of IEEE, and honorary fellow of the University of Edinburgh. He has held research posts at the National Research Institute for Mathematics and Computer Science of the Netherlands, The Netherlands Organisation for Scientific Research and the Vienna University of Technology.
Andrew R. Harvey conducted PhD and postdoctoral research THz instrumentation at St Andrews University, United Kingdom, until 1995 when he moved to DRA in Malvern, United Kingdom to research advanced imaging techniques. He returned to academic research at Heriot–Watt University in 2001 to develop techniques in computational and spectral imaging and their applications in systemic biomedical and retinal imaging. He became full professor in 2007 and moved to Glasgow University in 2011, where he heads the Imaging Concepts Group.