Fast visible and extended near-infrared multispectral fundus camera

Abstract. We present a multispectral fundus camera that performs fast imaging of the ocular posterior pole in the visible and near-infrared (400 to 1300 nm) wavelengths through 15 spectral bands, using a flashlight source made of light-emitting diodes, and CMOS and InGaAs cameras. We investigate the potential of this system for visualizing occult and overlapping structures of the retina in the unexplored wavelength range beyond 900 nm, in which radiation can penetrate deeper into the tissue. Reflectance values at each pixel are also retrieved from the acquired images in the analyzed spectral range. The available spectroscopic information and the visualization of retinal structures, specifically the choroidal vasculature and drusen-induced retinal pigment epithelium degeneration, which are hardly visible in conventional color fundus images, underline the clinical potential of this system as a new tool for ophthalmic diagnosis.


Introduction
Optical imaging systems for noninvasive ocular diagnosis remain crucial to assist ophthalmologists in their daily clinical practice. To promptly diagnose ocular and systemic diseases that manifest early in the retina, [1][2][3][4] efforts have focused on the acquisition of good-quality fundus images. Commercial fundus cameras widely used in the clinical setting include digital cameras that produce color images, with only three spectral bands (R, G, and B). As a result, some retinal structures and substances with different spectral signatures associated with specific ocular diseases 5,6 may go undetected due to metamerism or limited spectral range. The capability of identifying these spectral absorption properties in-vivo is key to clinical diagnosis and treatment. Multispectral imaging (MSI) technology, which increases the amount of extractable spectral information from biological tissues using more than three spectral bands, has been used in the last few years to advance the diagnosis of many diseases. [7][8][9][10] Newly developed fundus cameras that include MSI hold great promise for the analysis of the retina, since they combine the usefulness of spectroscopy with imaging, thus providing both spectral and spatial information of retinal landmarks. For instance, they can spotlight vascular occlusions, vein hemorrhages, drusen, and exudates, and also build oxygenation maps of the retinal vasculature, contributing in this way to the early diagnosis and treatment of conditions such as age-related macular degeneration (AMD) and diabetic retinopathy. [11][12][13][14][15][16] However, most MSI-based retinographs consist of modified traditional fundus cameras whose original illumination or detection systems have been replaced by tunable filters 11,17 or lightemitting diodes (LED), 12,18 or alternatively, by sequential MSI sensors. 19 They are in effect scanning-based systems susceptible to generating severe motion artifacts and pixel misregistration, because the eye is constantly moving; for this reason, some of them only include a few bands. To bypass these issues, snapshot MSI retinal cameras have been put forward for functional retinal mapping. [13][14][15] However, these approaches are limited by maximum resolutions of 350 × 350 pixels, which is insufficient to observe the smallest retinal structures for clinical purposes. Additionally, they present extensive computational cost. 20 Lastly, most MSI fundus cameras reported to date are limited to the analysis in the visible (VIS) range, ultimately limiting the physician's ability to detect, discriminate, and further investigate diseases as they develop; only one of them performs measurements up to 850 nm. 12 In this report, we present a customized MSI fundus camera that overcomes these limitations, being able to perform fast imaging of the retina in both the VIS and near-infrared (NIR) regions (400 to 1300 nm), through a considerable number of spectral bands (15) and with great spatial resolution. The unexplored range from 900 to 1400 nm, often called second NIR window, is of current interest for biologists 21 since it corresponds to the wavelengths in which radiation can penetrate deeper into the tissue (longer wavelengths are highly and less absorbed by water and melanin, respectively), thus providing information about occult or overlapping structures and diseases affecting the retinal pigment epithelium (RPE) and the choroid.

Description of the System
The developed MSI fundus camera was designed using the Zemax optical simulation software (Radiant Zemax LLC). Customized illumination and detection strategies were conceptualized to achieve, through the whole working wavelength range, the best light collection efficiency, aberration balance and cost (only commercial optical components were used), the most uniform field of illumination over the retina, and the minimal presence of ghost images. The resulting system (Fig. 1) has two detection arms: one for the spectral range from 400 to 950 nm (VIS-NIR detection arm) and another for 960 to 1300 nm (NIR detection arm). Both arms share the objective lens, placed in front of the patient's eye. Regarding the illumination arm, custom-made rings of LEDs were used rather than the most traditional approach of common fundus cameras, where a mask is inserted in the optical path to create a continuous illumination ring on the pupil. Specifically, 32 surface-mount device LEDs were welded in a ring-shaped configuration on three printed circuits board, each responsible for the illumination in a narrowband (Figs. 1 and 2): the VIS ring (2a), with 18 LEDs (9 twin pairs arranged symmetrically) emitting from 400 to 700 nm; the NIR ring 1 (2b), with 10 LEDs (5 twin pairs) emitting from 700 to 1100 nm; and the NIR ring 2 (2c), with 4 LEDs with a peak wavelength of 1213 nm. The former 15 LED clusters have peak wavelengths of 416, 450, 471, 494, 524, 595, 598, 624, 660, 732, 865, 955, 1025, 1096, and 1213 nm. The light from the rings, attached to a custom-made aluminum structure (1), is then combined in a common optical path by means of two dichroic mirrors with cutoff wavelengths at 1150 nm (3) and 700 nm (4).
After some lenses, mirrors and stops that prevent the back reflected light from the optical surfaces entering inside the detection optical path (5 to 7), a 45-deg custom-made holed mirror (13) conjugated with the patient's pupil plane couples the illumination (reflected) and detection (transmitted through the mirror's central hole) paths. This configuration assesses the geometrical pupil separation required to avoid ghost images caused by reflections (from the cornea and lens) of the illumination in the acquired fundus images. Finally, the LED rings are imaged by the objective lens (11 and 12). A properly designed set of lenses (14,16, and 18 in the VIS-NIR; 14 and 25 in the NIR) is placed in the detection paths after the holed mirror in order to image the retina in the two cameras (21 and 27), and to compensate for the patients' spherical refraction (range: ± 15 D) (19). The split of the VIS-NIR and the NIR detection paths is achieved with a dichroic mirror (15) with a cutoff wavelength of 950 nm. Field (17 and 22) and aperture (20 and 26) diaphragms are also used. The camera sensors are a CMOS detector (Orca Flash 4.0, Hamamatsu, Japan), with 2048 × 2048 pixels and 16-bit depth, and an InGaAs camera (C12741-03, Hamamatsu, Japan), with 640 × 512 pixels and 14-bit depth, for the VIS-NIR and NIR detection arms, respectively. Finally, a fixation target (8) is placed at a retinal conjugate plane in the illumination arm to achieve adequate patient fixation while performing measurements. All the optical components of the system were carefully selected so that their working spectral range matched the working spectral range of the arm they are part of, in order to minimize the undesired reflections and maximize the transmittance. In particular, some  Journal of Biomedical Optics 096007-2 September 2019 • Vol. 24 (9) components, such as the objective lens elements, must work for both the VIS and NIR ranges. In those cases, components with a broadband antireflection coating were chosen, ensuring a reflectance of <1.25% between 400 and 1000 nm according to the manufacturer's specifications (Edmund Optics Inc). For longer wavelengths, this reflectance tends to increase (4%), although it did not affect significantly the overall performance of the system. LEDs and cameras are synchronized by means of trigger signals in the electronics, allowing the sequential recording of the whole set of 15 spectral images in 612 ms [264 ms for the CMOS (VIS-NIR) and 348 ms for the InGaAs (NIR)], thus avoiding the imaging problems caused by eye movements. A user-friendly MATLAB interface was developed to control the system's operation and to set the parameters.
The resulting MSI prototype, built in accordance with ISO standards for ophthalmic instruments and fundus cameras (ISO 15004-2:2007 22 and ISO 10940:2009 23 ), has an angular field of view of 30 deg, magnifications of 1.20 (VIS-NIR) and 0.88 (NIR), and a resolution better than the limit imposed by the imaging sensors (pixel size 6.5 and 20 μm for the CMOS and the InGaAs, respectively). The stability of the system was tested through a mid-term analysis by performing 10 measurements in 5 consecutive days, early in the morning and late in the afternoon, and short-term repeatability (10 complete streams of images taken consecutively) using an artificial eye (OEMI-7, Ocular Instrument Inc.). Averaged digital levels on the same portion of the image were then calculated for all wavelengths and measurements; percentages of variation (standard deviation/average) for both the mid-and short-term analyses were found to be under 2.98% and 1.98%, respectively. Since the LEDs do not work with a closed-loop intensity control, periodic calibrations of the system must be carried out to guarantee an accurate retrieval of the spectral features of the retina at all wavelengths along all LEDs lifecycle. 9 Finally, after the light hazard protection evaluation, the MSI camera was classified as group 1 (nonhazardous ophthalmic instrument).

Selection of Wavelengths
In addition to commercial availability, optical characteristics such as radiometric power, angle of emission and dimensions, the wavelengths of the LEDs (Fig. 2 and Table 1) were carefully selected to obtain the most accurate information from the spectral characteristics of structures and biological substances present in the ocular fundus. 10,24 The LEDs from 400 to 500 nm were chosen to investigate the superficial layers of the retina, in particular the nerve fiber (NF) layer and internal limiting membrane, and also the fluorescent retinal components (carotenoids of photoreceptor pigments acting as a yellowish filter in the macula, and lipofuscin that accumulates as a product of phagocytosis of photoreceptors in the outer segments and concentrates in the RPE). Similar to the conventional red-free images obtained with a green filter in commercial fundus cameras, wavelengths within the range of 500 and 700 nm were chosen to study the retinal vascular layers and to discriminate between arteries and veins (oxygenated and deoxygenated blood). It can be noticed that two LEDs with similar peak wavelengths (595 and 598 nm) were included in the system (Table 1), being the spectral width of the latter (76 nm) much larger than for the rest of LEDs in the VIS range. This LED was chosen to sample part of the 530 to 580 nm range, where narrow spectral width LEDs with enough radiometric power and reduced size to be integrated in the system were not commercially available, thus avoiding this gap in the spectral sampling. Melanin absorbs in the spectrum used by traditional retinographs, but in the range from 450 to 600 nm, it is obscured by other dominant ocular components such as hemoglobin, lutein, and rhodopsin. 12 Beyond 600 nm, melanin is the dominant retinal pigment and thus, images at these wavelengths enhance the visualization of retinal structures that contain this pigment, such as the underlying RPE. In addition to deeper penetration in the retina due to the lower absorption of melanin, LEDs with peaks at 732 and 865 nm were expected to provide information about the oxygenation of the deepest retinal and superficial choroidal vasculature (CV). The longest wavelength LEDs (peaks from 955 to 1213 nm) were selected to study the choroid, since the retina appears progressively more transparent, even projecting light into the sclera, which can be used for imaging congenital hypertrophy of the RPE, choroidal ruptures, and melanomas. 12 Lipids (drusen) also show absorption peaks in the NIR that exceed water absorption, and therefore images taken at longer wavelengths can be relevant in the study of AMD. 25

Fundus Reflectance Retrieval
To compute reflectance values at each pixel (i; j) for a given wavelength (λ n ) from the spectral images recorded, we acquired images from a calibrated reference white (BN-R98-SQC, Table 1 Peak wavelengths, full width at half maximum (FWHM) and power of the LEDs, and exposure times of the cameras used in the multispectral acquisition for each of them. These times are optimal to compensate for the different powers of the LEDs, the transmittance/ reflectance of the optical components of the system and the eye, and the spectral sensitivity of the cameras. Gigahertz-Optik GmbH, Germany) to compensate for the differences in power of the LEDs and sensitivity of the cameras along the spectral range. Dark current images taken without ambient light and any eye in front of the system were also recorded immediately after the acquisition of each multispectral sequence of the ocular fundus, as a measurement of the background noise. Ocular fundus, reference white, and dark current images were captured using the same exposure times for each spectral band ( Table 1). The reflectance was then calculated as commonly described in the literature: 9,10,26 Despite the fact that we use the term reflectance in accordance with other spectroscopic applications, it is noteworthy that Refl λn also includes the transmission of the ocular media (cornea, lens, aqueous humor, and vitreous) besides the reflectance of the retina itself.

Results
As a preliminary test to evaluate the potentiality of the MSI fundus camera, a preclinical study was conducted on healthy and diseased retinas of a small group of volunteers at the Instituto de Microcirugía Ocular (Barcelona, Spain) (Fig. 3). All patients provided written informed consent before any examination and ethical committee approval was obtained. The study complied with the tenets of the 1975 Declaration of Helsinki (Tokyo revision, 2004).
As anticipated, short wavelengths are reflected and scattered back more superficially, thus facilitating the observation of structures such as the optic disk and nerve fibers. In images from 500 to 700 nm, the different spectral absorptions of oxygenated and deoxygenated hemoglobin result in contrast changes between retinal veins and arteries and the surrounding fundus. For wavelengths in the NIR range, particularly those longer than 950 nm, information from layers beyond the retina such as the choroid and its vasculature is available without the use of invasive dyes due to the reduced absorption of tissue chromophores and the subsequent deeper penetration of light. Specifically, a large amount of backscattered light coming from the sclera forms the image at 1213 nm. Image acquisition beyond 1300 nm was not possible due to the high absorption of preceding ocular media and low quantum efficiency of the InGaAs camera. Notably, the image at 416 nm is of poorer quality, mainly due to the higher absorption of the lens. Figure 4 shows the spectral images from two healthy individuals with high and low melanin concentration. Differences between the sequences are evident: for shorter wavelengths, a high concentration of melanin hinders the visualization of deeper structures such as the CV; these structures appear only evident in the eye with less melanin. In contrast, at longer wavelengths, where melanin absorption is strongly reduced, the choroid can be seen in both cases. In addition, Fig. 5 shows a conventional color fundus image taken with a mydriatic retinal camera (TRC-50DX, Topcon Medical Systems, Inc., Japan) as well as spectral images of an eye of a patient aged 78 years old with an alteration of the RPE. It can be seen that while drusen-induced RPE degeneration is hardly visible in the color image, it is clearly noticeable in images corresponding to mid and long wavelengths in which light penetrates deeper into the tissue.  The reflectance of the RPE lesion and that of the surrounding healthy fundus is shown in Fig. 6. The reflectance of the lesion was computed as the average of the reflectance image values within a circle of 0.5-deg radius from the center of the drusen [blue circle in Fig. 6(a)]. For the estimation of the mean reflectance of the surrounding healthy tissue, four circular areas around the drusen were used [red circles in Fig. 6(a)] to minimize any local effect caused by other retinal structures. We should note that the reflectance curves are in accordance with the combined effect of the spectral fundus reflectance and ocular media transmittance: 27 the values increase from blue to red, due to an increase in the retinal reflectance, and then decrease at the longest infrared wavelengths, due to the stronger absorption of the water. Nevertheless, there were relevant differences between the lesion and the surrounding tissue in terms of spectral reflectance, in particular in the red and the first NIR wavelengths. On the contrary, an analog analysis in terms of the digital levels of the three color channels of the image acquired with the conventional fundus camera provided very similar values (R, G, and B) for the degenerated RPE region (236, 106, and 27) and the surrounding tissue (231, 101, and 27), as expected.

Conclusions
This MSI fundus camera, which achieves fast imaging of the posterior pole at wavelengths from 400 to 1300 nm, will allow visualizing retinal and choroidal structures from an unprecedented perspective that can facilitate the early diagnosis of retinal diseases such as AMD, diabetic retinopathy, and melanomas. Currently, color retinography is the standard method to detect retinal lesions, such as drusen, exudates, neovascularization, and hemorrhages. However, they might exclude long red and NIR wavelengths, which provide additional diagnostic information, particularly from deeper layers. In contrast, MSI technology with extended sensitivity toward the NIR improves the contrast of specific retinal and choroidal structures through a spectral analysis as shown with the developed system. Moreover, it can obtain useful information on metabolic processes such as oxygenation of retinal tissues without the use of ocular angiography.
Further studies to test this prototype for clinical diagnosis are currently ongoing in the clinical setting with healthy and diseased eyes. The aim of these studies is to correlate retinal and choroidal spectroscopic observations with color and morphological findings as measured with the multispectral camera, with conventional color retinography, and with optical coherence tomography.

Disclosures
No conflicts of interest, financial or otherwise, are declared by the authors.