2.1.

Subjects

We examined six eyes (three right and three left eyes) of three healthy subjects (aged 22, 24, and 30 years) and three subjects with suspected glaucoma (aged 32, 37, and 46 years). The latter three subjects were known to have a conspicuous optic disc, their intraocular pressure was within the normal range, and there were no visual field defects. All subjects gave their written informed consent in accordance with the declaration of Helsinki. To minimize the influence of light scatter and wavefront errors, the exclusion criteria were dry eye syndrome, cornea defects, edema, and cataract. Refractive errors were limited to $-2$ to $+1$ dpt spherical and $-0.5$ to $+0.5$ dpt cylindrical. All subjects underwent a preliminary examination to rule out eye diseases.

2.2.

Eye Model

To validate the depth estimation and to show its possibilities and limitations, an eye model (scale 1:1) incorporating an interchangeable fundus was designed and realized (Fig. 1). The scale was chosen to realistically mimic the geometry of the human eye, especially the anterior parts starting with the cornea. This is a crucial factor in terms of strong reflections and disturbing stray light by the interaction of illumination and imaging light. To avoid low-contrast images due to overlaying reflections and stray light in the imaging path, the separation of illumination and imaging at these parts of the eye is necessary.

Fig. 1

Exploded view of the eye model and photography of the fundus model and the eye model; the components from left to right: changeable fundus on a miniature dovetail translation stage, housing, lens mount for the lens, lens mount for cornea and pupil, lens, changeable pupil slider (here, an 8-mm pupil diameter), and custom cornea.

The fundus had a radius of 10 mm and papilla excavations of 0.2, 0.4, 0.8, and 1.0 mm, respectively (Fig. 2). The eye model’s total length was 24.25 mm. The cornea was a custom, uncoated N-BK7 lens (outer radius 7.707 mm, inner radius 8.886 mm, and thickness 1.5 mm). The lens was an uncoated achromat (AC127-19, Thorlabs GmbH, Munich, Germany). The lens mount, housing, and fundus were printed with a 3D printer (Objet30 Prime, Stratasys Ltd., Eden Prairie, USA) with a resolution of $25\mu \mathrm{m}$. The material used was white plastic (VeroWhite™, Stratasys Ltd., Eden Prairie, USA). The fundus models were also painted with red acrylic lacquer (Molotow™, Feuerstein GmbH, Lahr, Germany), and vessels were drawn with a thin brush by a professional artist.

Fig. 2

Cross section of the fundus model with a fundus radius of 10 mm and a papilla depth of 0.8 mm, as an example.

2.3.

Technical Setup (LF Imaging Device)

The technical setup, shown in Fig. 3, consisted of a mydriatic fundus camera (FF450, Carl Zeiss Meditec AG, Jena, Germany) and an LF imager (R12, Raytrix GmbH, Kiel, Germany) connected via a c-mount adapter. The LF imager was operated with the Raytrix LF software (RxLive 5.0.046.0, Raytrix GmbH, Kiel, Germany).

Fig. 3

Technical setup consisting of a fundus camera with an adapted LF imager which is operated with the Raytrix LF software (RxLive 5.0.046.0, Raytrix GmbH, Kiel Germany). The fundus camera is equipped with a broadband halogen lamp, while four narrow bands are realized by means of bandpass filters ($450\pm 20$,$520\pm 20$,$600\pm 20$, and $650\pm 20\mathrm{nm}$).

The LF imager consisted of a Basler industrial camera (type: acA4024-29um, Baser AG, Ahrensburg, Germany) and a microlens array. This camera had a $4024\times 3036\text{pixel}$ monochrome 16 bit CMOS sensor (type: Sony IMX226CLJ-C) with an effective sensor diagonal of 9.33 mm, a pixel size of $1.85\times 1.85\mu \mathrm{m}$, and a maximum frame rate of 29 frames per second. The microlens array had lenses with a diameter of $80\mu \mathrm{m}$ in a hexagonal grid of three focal lengths to increase the imageable depth.^9,10

The setup corresponds to a focused plenoptic camera, sampling the LF of an intermediate image with microsubimages, i.e., the microlenses project small subimages of the scene onto the sensor. Each subimage shows a slightly different view. If an object point can be detected in at least two subimages, a virtual depth can be estimated similar to stereo photography approaches. The detection of two corresponding object points is done by autocorrelation of small pixel patches, which require a minimum level of contrast of image structure. Parameters for adapting this autocorrelation function are given in the Appendix. If the corresponding object points are known, a triangulation is performed and the disparity or virtual depth is determined. A detailed description of the algorithmic image reconstruction and depth estimation is given by Perwaß and Wietzke.¹⁰

The use of a focused plenoptic camera enabled image reconstruction with high spatial resolution in contrast to a conventional LF camera, the lateral resolution of which is limited by the number of microlenses.^7,8 To achieve the maximum angular resolution of the LF imaging and thus make use of the maximum depth resolution, the $f$ number of the preceding optic (here, the optic of the fundus camera) should be equal to the LF camera.^8–10 The LF camera has an $f$ number of 2.4. To best match the optical aperture of the fundus camera, the field of view of the fundus camera optics was set to 30 deg (20 deg, 30 deg, and 50 deg were possible). With the resulting lateral magnification of 1.5 and 2.25 in the direction of the optical axis, the depth resolution of the technical setup can be estimated at $18\mu \mathrm{m}$.

To perform spectral depth measurements, four bandpass filters (Thorlabs GmbH, Munich, Germany) with central wavelength ± bandwidth (full-width at half-maximum) values of $450\pm 20\mathrm{nm}$ (blue, FB450-40), $520\pm 20\mathrm{nm}$ (green, FBH520-40), $600\pm 20\mathrm{nm}$ (orange, FB600-40), and $650\pm 20\mathrm{nm}$ (red, FB650-40), respectively, were inserted into the illumination path of the fundus camera optics using customized sliding filter mounts. The eye model was illuminated with green light, with a wavelength of $520\pm 20\mathrm{nm}$.

2.4.

Optical System Characterization

To determine the optical performance of the overall system, the modulation transfer function (MTF) at the optical axis was determined and compared with the MTF of the same optical system utilizing a conventional CCD imager instead of the LF imager. For this purpose, a razor blade edge was positioned vertically in the center of the fundus plane of the eye model. It was homogeneously back-illuminated with green light (520 nm). A total focus LF image and a conventional image using a CCD camera (Stingray F-046, $780\times 560\text{pixels}$, $8.3\times 8.3\mu \mathrm{m}$, Sensor Sony ICX415, Allied Vision, Stadtroda, Germany) were taken. The MTF curves were then calculated using MATLAB (R2018b, MathWorks) by edge differentiation of the average edge profile of 50 rows each with a length of 400 pixels, with a subsequent Fourier transform.²⁴

2.5.

Imaging Procedure

The imaging procedure and analysis are shown in Fig. 5. Each subject underwent an examination via optical coherence tomography (OCT Spectralis, Heidelberg Engineering GmbH, Heidelberg, Germany) with the glaucoma module (Glaucoma Module Premium Edition, Software Version 6.0, Heidelberg Engineering GmbH, Heidelberg, Germany) using the ONH-RC scan pattern (optic nerve head-radial circle) (OCT-scan, Fig. 5). After detecting and confirming the fovea position, the center of the basal membrane opening (BMO) was detected, adjusted if necessary, and confirmed. The $c$-curve (meaning corneal radius) was individually measured via an autorefractor (CX 1000, Rodenstock GmbH, Munich, Germany) and entered. The acquiring process included 24 radial scans centered above the BMO with a rotation distance of 7.5 deg and three circle scans ($3.5/4.1/4.7\text{-}\mathrm{mm}$-diameter). For the radial scans, 25 continuous scans were cumulated, whereas 100 continuous scans were cumulated for the circle scans. Subsequently, the same eye of each subject was set under mydriasis (Mydriaticum Stulln, Pharma Stulln GmbH, Stulln, Germany). After a period of 15 min for pupil dilatation, LF images were taken (LF image acquisition, Fig. 5).

For each illumination bandpass filter, three images were captured, and the filter sequence was randomized. For illumination, the standard light source of the fundus camera was used (Fig. 3). The illumination intensity was individually set as bright as was acceptable to each subject. The shutter time and gain of the LF imager were set close to saturation, avoiding overexposing the papilla region. In the eye-model study, 30 images were taken for each papilla depth. For each image, the LF fundus camera was readjusted, including the illumination intensity and exposure parameters of the LF imager.

2.6.

Image and Measurement Analysis

For OCT-depth measurement, 2 of the 24 radial OCT scans were selected with an excellent visible base of the optic cup and prominent vessel structure above (or slightly behind) the neuroretinal rim of the papilla for each subject (OCT-depth measurement Fig. 5). One of the selected scans was more vertically oriented, whereas the other was more horizontally oriented; this depended strongly on the individual vessel tree of each subject. In the analysis, only horizontal and vertical scans were used. In each selected scan, the lowermost base position equal to the deepest spot in the optic cup and the uppermost vessel position equal to the uppermost position of the optic disc were marked graphically by horizontal lines; the distance between these two lines, equal to the depth, was measured with a vertical line (Spectralis, Glaucoma Module Premium Edition, Software Version 6.0, Heidelberg Engineering GmbH, Heidelberg, Germany) (Fig. 15). This measurement process was done with the help of the overlay tool and its option to measure distances in $\mu \mathrm{m}$. Both results were averaged.

For LF image analysis, Raytrix LF software (RxLive 5.0.046.0, Raytrix GmbH, Kiel, Germany) was used. Detailed software parameters and settings used can be found in the Appendix. The image processing procedure within the Raytrix LF software is shown in Fig. 4. First, the input image passes preprocessing, which was disabled here to not change the image data. In a second step, subimage autocorrelation and triangulation for depth estimation were performed. The third step generated a depth map, and data were filtered conservatively here. In the last processing step, the corresponding depth map was visualized.

Fig. 4

Image processing chain of the LF image analysis using Raytrix LF software (RxLive 5.0.046.0, Raytrix GmbH, Kiel, Germany). The input image is first preprocessed (not used here); subimages are autocorrelated, triangulated, and depth estimated; a depth map is generated in which the papilla depth was measured; and finally the depth-map is visualized.

For the analysis of the study data, the image quality of the LF images was first subjectively proved regarding sharpness, motion blur, and artifacts. The best of three images was taken for further analysis (LF image analysis, image selection, Fig. 5).

Fig. 5

Flowchart of the subject study using subject 4 as an example.

The quality of the fundus images regarding depth reconstruction at different wavelengths was evaluated by analyzing the Michelson contrast and by counting the number of reconstructed depth positions using MATLAB (R2018b, MathWorks). This evaluation was restricted to a region of interest (ROI) due to the presence of image artifacts with varying intensity and manifestation, different occurrences of the fixation needle, the influence of the aperture diaphragm, and a change in visibility of structures. To ensure a comparable evaluation across all images, the center of the ROI needed to correspond with the center of the papilla, and a circular region was defined. To assure minimal differences between the center positions, the more prominent rim was marked in clockwise order for each image: north, east, south, and west. The horizontal center of the papilla was calculated by dividing the distance between the east and west coordinates, whereas the vertical center used north and south.

Furthermore, the horizontal and vertical extensions of the papilla allowed for the calculation of the largest papilla radius with 185 pixels. The Michelson contrast was analyzed by selecting measurement positions in the total focus image along an imaginary ring at a distance of two times the papilla radius from its center at the three largest venules. Arterioles were not considered because of the limited visibility, especially at 650 nm. Three measurement positions were stored per venule, one above and the other two on opposite sides of the venule. Concerning the small but nearly visible bubble-like artifacts that increased the gray values locally, the measurement was taken with an eroded version of the total focus image using a disc-shaped structuring element with a radius of 8 pixels. This erosion operation worked like a filter and stored the lowest gray value inside the structuring element at its center. The three separate Michelson contrast values per image were averaged for a clear presentation.

As a second quality parameter, the number of reconstructed depth positions per image was counted inside a circular ROI with a diameter of 5.5 times the largest papilla radius of 185 pixels. The factor 5.5 represents the maximum possible ROI, avoiding the overlap of the aperture diaphragm while also keeping the ROI inside the image itself. Papilla depth was then determined at the wavelength with the most depth measurement points (LF image analysis, optimal wavelength selection, Fig. 5).

For this purpose, the position in the $Z$ direction of the optic nerve head was determined on an average circle area of 20 pixels, including the ground of the optic nerve head. The position in the $Z$ direction of the surrounding area was determined based on the two radial scans from the OCT and the selected measurement locations. Again, an averaged circle area of 20 pixels was used. Both measuring points were selected manually. The depth of the optic disc was determined from the difference between the optic disc periphery and its ground. This depth determination was performed according to the OCT scans in the horizontal and vertical directions. The resulting depths of the optic nerve head were averaged for each eye. The depth specifications correspond to a virtual estimation in virtual mm (v-mm) (LF-depth estimation, Fig. 5), which can be converted to mm with the calculated image scale of 2.25 in the $Z$ direction.

The papilla depth of the eye model was determined on an averaged circle area of 85 pixels, including the complete ground of the optic nerve head. All 30 measurements for each papilla depth were averaged and compared with the printed depth. The imaging procedure and analysis are shown in Fig. 5. Six subjects were examined. Each subject was examined with four wavelengths for LF imaging and underwent an OCT examination. Three LF images per wavelength were taken, and one OCT scan (ONH-RC scan pattern) was conducted. Of the three LF images taken per wavelength, one was chosen for analysis. The four resulting LF images per subject were analyzed with regard to vessel contrast and depth data point found. The image with the highest vessel contrast and most datapoints was selected. Within this image, the papilla depth was estimated horizontally and vertically. Both data points were averaged. A corresponding depth measurement was performed in horizontal and vertical OCT slices. Both data points were averaged. Results of the LF estimation and OCT measurement were compared.

3.1.

MTF Determination

The minimum MTF amplitude for the LF imager occurred at $\sim 90\text{cycles}/\mathrm{mm}$ and was therefore chosen as the cutoff frequency. The MTF graph of the LF imager and the conventional camera have very similar progressions, whereas the MTF of the LF imager shows a slightly better performance in the middle with $\sim 56\text{cycles}/\mathrm{mm}$ at a relative amplitude of 0.5 in the object space compared with $45\text{cycles}/\mathrm{mm}$ for the conventional imaging (Fig. 6).

Fig. 6

MTF curves of the overall system in object space (fundus of the model eye), using the LF setup in total focus mode (circles) and the system using a conventional CCD imager.

3.2.

LF Imaging

It was possible to perform 3D fundus imaging in an eye model as well as in vivo by means of a novel LF-based optical setup. Subsequent digital refocus was possible in all LF images. Figure 7 shows examples of digitally refocused images of the papilla of subject S1 with suspected glaucoma. Figure 7(a) is focused on the retina and features sharp vessels and a blurred papilla bottom. Figure 7(b) is focused on the papilla bottom; the retinal structures are blurred, with circular artifacts in a hexagonal grid. Figure 7(c) shows the total focus image with the maximum depth of focus.

Fig. 7

Refocused partial images ($\sim 10\text{-}\mathrm{deg}$ field of view) of a suspiciously glaucomatous papilla: (a) retina focused, (b) papilla bottom focused, and (c) total focus image with maximum depth of focus (full field).

3.3.

Eye Model Measurements

Depth estimation of the eye model was possible over a 30-deg field of view (Fig. 8). Colored values represent the reconstructed depth estimation at the specific location. Depth estimation worked in regions with high-contrast structures, i.e., vessels and papilla. Papilla depths were measured from the middle of the papilla to the surrounding flat image region. Results are shown in Fig. 9. A linear relationship between the virtual depth and the model papilla depth can be assumed.

Fig. 8

(a) Total focus image of the eye model with a papilla depth of 1.0 mm. (b) corresponding depth map with papilla cupping clearly visible; depth map showing only small gaps in gray; horizontal depth is measured as $Z$-difference between papilla center and retinal surface (dashed line, measurement area in dashed circles).

Fig. 9

Relationship between the measured depth of the papilla in v-mm and the depth of the papilla in the eye model in mm. The boxplots represent the distribution of model papilla depths over $n=30$ LF measurements per model. Dotted line: linear fit with $y$ = measured virtual papilla depth and $x$ = model papilla depth.

3.4.

Spectral Imaging Versus Depth Estimation

Figure 10 shows the LF images in total focus for subject S1. The vessel contrast is reduced at 600 and 650 nm. At 650 nm especially, veins are barely visible, whereas the choroidea becomes more prominent. The highest Michelson contrast per subject was found at wavelength 520 nm (see Fig. 11). The standard deviation was comparably low for all wavelengths, excluding 450 nm but including the favorable 520 nm.

Fig. 10

Total focus images of subject S1 for all wavelengths. From left to right: 450 nm (blue), 520 nm (green), 600 nm (orange), and 650 nm (red). Vessel contrast decreases with increasing wavelength.

Fig. 11

Averaged Michelson contrast and standard deviation of three venules per subject and wavelength: 450 nm (blue), 520 nm (green), 600 nm (orange), and 650 nm (red).

Figure 12 shows all total focus images with the corresponding depth map for all subjects and wavelengths. The maximum numbers of reconstructed depth positions per eye were as follows: 219,504 (S1), 295,941 (S2), 203,701 (S3), 146,922 (S4), 332,798 (S5), and 270,805 (S6). The normalized distribution per wavelength was included in this figure. The highest number of reconstructed depth positions per subject was typically at 520 nm. Only for subject S4 was the wavelength of 600 nm slightly better, with $<2\%$ more reconstructed depth positions. The other subjects showed a decreased number of values between 32% and 63%. The lowest number of reconstructed depth positions per subject was at 650 nm. Depth estimation only works in image areas with structures. Those structures are the retinal vessels, papilla border, and structures within the papilla. Choroidea was not measured.

Fig. 12

LF images of all subjects and wavelengths. Total focus images with the corresponding depth map overlays and measurement points normalized to the maximum per subject are shown. Most measurement points were found at a wavelength of 520 nm (for subjects S1 to S3, S5, and S6). Subject S4 had slightly more found measurement points at 600 nm. The internal fixation needle in the first intermediate image plane of the fundus camera (straight line within the fundus images) is imaged at S1 to S4 and S6 and depth-estimated thereon.

Furthermore, the papilla depth was greatest for subject S5, corresponding to the deepest measured excavation with the OCT shown in Fig. 13. This subject has suspected glaucoma. The flattest papilla was observed in subject S2. The comparison of the OCT measurement and the LF depth estimation shows a linear relationship and an offset (Fig. 13).

Fig. 13

Comparison of LF papilla depth measurements with OCT data. Dots: papilla depth measurement with LF camera in v-mm compared with OCT measurements. Dotted line: linear fit with $y$ = LF papilla depth and $x$ = papilla depth measured with OCT; triangles: papilla depth measurement with LF camera corrected with the calculated magnification of depth of 2.25 in mm compared with OCT measurements, solid line: linear fit with $y$ = corrected LF papilla depth and $x$ = papilla depth measured with OCT.

Figure 13 shows additionally corrected LF depth papilla measurements. The measurements were corrected with the calculated depth magnification of 2.25. They are in better agreement with the OCT measurements. Differences in the corrected LF papilla depth and the OCT measurements (Fig. 14) show that flat papillae are overestimated with a maximum measurement difference of $138\mu \mathrm{m}$ and deep papillae are slightly underestimated with a minimum measurement difference of $-25\mu \mathrm{m}$ at subject S1.

Fig. 14

Depth differences of the magnification corrected LF depth estimation and the OCT measurements.

Figure 15(a) shows a vertical papilla OCT scan of subject S1. The blue dots indicate the scaled vertical depth profile of the LF system. The excavation with a flat bottom and steep walls can be clearly seen in both systems.

Fig. 15

(a) Vertical papilla OCT scan of subject S1 with the vertical depth profile of the LF system (blue dots) and (b) SLO scan of the OCT showing the scan orientation.

3.5.

Artifacts

For subject S3 especially, image acquisition was challenging due to unsteady fixation and small pupil despite mydriasis, resulting in significant image artifacts. The artifacts are visualized in Fig. 16. Reflections and scattering from the cornea and lens led to large checkerboard-like artifacts that cannot be subsequently removed. Internal system reflections were visible in all images of all subjects, especially with green light. Presumably, these reflections were generated between the last system lens or prism and microlens array [Fig. 16(a)]. All images taken with red light show round, blurred dots in a hexagonal grid [Fig. 16(b)].

Fig. 16

(a) Image artifacts at 520 nm (total focus, subject S3) and (b) image artifacts at 650 nm (total focus, subject S1).