As light from objects enters the eye (Fig. 1 ) it undergoes refraction, governed by Snell’s law, at the transition between the outer air and the anterior surface of the cornea. The light front undergoes further refraction as it passes through the posterior surface of the cornea and enters the anterior chamber of the eye. The light front then passes through the pupillary opening of the iris to enter the eye lens or crystalline lens. This is followed by several additional refractions taking place within the eye lens itself. Upon exiting the eye lens, the light front travels through the vitreous cavity, ultimately striking the retina of the eye (thereby forming an upside down, left-right inverted, and warped retinal image), where it is received by the photoreceptors—the rods and cones—and converted to electrochemical signals. These electrochemical signals are subsequently processed and compressed by the neural network cascade of the retina, at which point the “retinal image” does not exist any more as an image, but rather as a spatiotemporal neural spike pattern. This spatiotemporal neural spike pattern is then transmitted via the optic nerve to the visual cortex, where it is further processed at multiple levels and merged with other sensory inputs, ultimately leading to what we would call “visual perception” in a rather abstract (virtual) manner [see, for example, chapters 1 to 5 in Ref. 1 for an interesting (popular) description of the concept of vi sual perception (cognition)]. Therefore, we would like to emphasize the fact that visual perception is profoundly different from retinal images [and point-spread-functions (PSF) for that matter, which are a useful metric for optical engineering], and that we do not attempt to simulate the retinal processing cascade, let alone the visual cortex in this work. After all, even a successful simulation of these processing schemes would yield only a spatiotemporal neural spike pattern, but not a visible image that would resemble “visual perception.” In contrast, the visual perception simulation technique discussed here is admittedly not biologically motivated. However, it is a rather straightforward technique that yields results people can relate to, judging from their own personal experience (e.g., subjects who have one emmetropic eye, and one eye with a certain defect can confirm the visual experience by viewing the original image source and the simulated outcome with the respective other eye).
Computer ray-tracing permits simulations of the optical properties of the human eye (e.g., Refs. 2, 3, 4, 5, 6, 7) and of visual perception under various eye defects.4 Three-dimensional (3D) scenes or two-dimensional patterns of point sources, including digital photographs (images), serve as “light-giving” input objects for a ray-tracing simulation. In ophthalmic ray-tracing, the path of the light rays is calculated between these input objects and the retina of a computer eye model (e.g., Gullstrand’s exact schematic eye model,8, 9 Fig. 2 , or more elaborate eye models, e.g., Refs. 10, 11, 12, 13, 14) using Snell’s law of refraction. The image formed on the retina is upside down, left-right inverted, and warped due to the curvature of the retina. To obtain a first-order visual perception without having to simulate the neural function (processing) of the retina and the visual cortex, Fink 4 devised a back-projection method for the retinal image through an idealized eye (i.e., an emmetropic eye with only minor aberrations but no additional eye defects) to an output screen of the same dimensions and at the same distance from the eye as the input object (Fig. 3 ).
We have previously used Gullstrand’s exact schematic eye model with improved parameters (Figs. 2 and 3)4 with six spherical refractive surfaces, a spherical retina, and an added iris (for determining the degree of both on-axis and off-axis line-of-sight aberrations to be simulated), to study visual perception under various eye conditions such as myopia (nearsightedness), hyperopia (farsightedness), cataract caused by microvacuoles, dislocated intraocular lens after cataract surgery, and refractive scotomata (visual field defects) caused by the usage of correction lenses in automated perimetry (visual field testing method).4, 15, 16, 17, 18 While a qualitatively valuable, analytically calculable, and successful test environment for studying visual perception, Gullstrand’s exact schematic eye model also has its limitations, predominantly because of the sphericity of its surfaces and, resulting from that, a very limited customizability.
To obtain more realistic and quantitative results, aspheric surfaces must be considered (e.g., Refs. 10, 11, 12). Ray tracing with aspheric surfaces can still be analytically calculable: Langenbucher, 19 for example, report on an algebraic method for ray tracing through the optical system of an eye with aspheric surfaces. Their method is restricted to second-order surfaces (quadric surfaces). In Sec. 2, we introduce a new ophthalmic ray-tracing tool for the simulation of visual perception, termed simEye,20 and discuss its underlying mathematical framework using Zernike polynomials2, 21, 22, 23 for extending the ray-tracing process to include “arbitrary,” nonspherical surfaces. In Sec. 3, we give examples of simulations, obtained with simEye, of visual perception under instances of emmetropia, regular astigmatism, irregular astigmatism, and (central symmetric) keratoconus—eye conditions that are characterized by aspheric corneal surfaces.
Zernike polynomials are a set of orthogonal polynomials used in geometrical optics for representing different kinds of eye aberrations.2 More recently, their use for modeling or fitting of the refractive surfaces responsible for the aberrations has been proposed (e.g., Ref. 23). The Zernike polynomials can be defined in cylindrical coordinates as a function of (radius), (polar angle), and (height)2, an angle , and a height . In the case of eye surfaces, such surface data can either originate from biometric measurements performed on real eyes or from model eyes. The goal is to obtain a surface that best approximates the surface defined by the input points. This is done by calculating the height from the Zernike polynomial fit with and as input parameters is the number of input points, is the number of polynomials (from index 0 to index ), is the height of the input point at , and is the coefficient for the ’th Zernike polynomial. We perform a least-squares minimization of the average distance between the surface to be fitted and the fitting surface created by the Zernike polynomials (e.g., Ref. 22), expressed as the mean squared error ) is obtained when all partial derivatives vanish for the Zernike polynomials via matrix inversion and multiplication (e.g., Refs. 22, 23, 24, 25).
This mathematical formulation has been implemented as a software package, termed simEye, in standard American National Standards Institute C and runs on UNIX platforms, such as Linux and Mac OS X. The surface input data are arranged in three columns (one column for each coordinate) that define the surface in cylindrical coordinates. simEye then attempts to fit the given surface starting with the user-specified number of Zernike polynomials. The measure chosen for evaluating the quality of the surface fit is the root-mean-squared error (RMSE) defined as (other error measures may be applied as well)
In the ray-tracing procedure used here (for more details see Ref. 4), light rays are propagated from an input object (e.g., digital image) through the eye under investigation (i.e., the eye with modeled eye defects), taking Snell’s law of refraction into account, to form a warped retinal image (due to the retinal curvature) that is upside down and left-right inverted. This is apparently not how we visually experience (perceive) the world. To obtain a first-order realistic visual perception without having to simulate the neural function (processing) of the retina and the visual cortex as mentioned in Sec. 1, this retinal image is then back-propagated through a defect-free, idealized eye (e.g., Gullstrand’s exact schematic eye with no additional eye defects, Fig. 2) to an output screen of the same dimensions and at the same distance from the eye as the input object (Fig. 3).4, 15 This ray-tracing procedure unwarps the retinal image, flips it right-side up and, from a visual perception perspective, projects the image to where it originates (i.e., “we see things where they are”).
Using the geometric optics ray tracing above, it is not sufficient to flip the retinal image both vertically and horizontally in order to obtain a first-order simulation of visual perception for the following reasons: (1) the retinal image is warped due to the curvature of the retina (eyeball) in contrast to our experience of visual perception; (2) while the source image is, for computer-simulation purposes, a digital image and hence consists of well-defined, discrete, and equidistant individual pixels, the computer-simulated retinal image is characterized by a subpixel resolution and therefore does not adhere to well-defined, equidistant individual pixels any more. A horizontal and vertical flip operation on the retinal image would only be possible if one were to “average out” the subpixel resolution in order to arrive at a pixel-based (i.e., grid-based) retinal image. Such an averaging procedure would produce artifacts and would, in addition, not remove the warping of the retinal image, in other words, it would not produce a realistic experience of visual perception.
Because light rays could fail to hit the output image due to refraction while propagating through both the eye with modeled defects and the defect-free eye for back-propagation, the path of the light rays is reversed for practical purposes, making the output image the starting point for the ray-tracing procedure (Fig. 3).4, 15 This means that the light rays are propagated from the pixels of the output image through Gullstrand’s exact schematic eye (i.e., defect-free eye) and subsequently through the eye with the modeled eye defects toward the input source image. This guarantees that all pixels of the output image will be filled with color information from the input source image, thereby reducing void light ray calculations.
To describe the trajectory of the individual light ray, several mathematical and optical calculations must be performed for each of these surfaces. These can be summarized as
• Calculation of the intersection point between the light ray and the corresponding eye surface.
• Calculation of the corresponding surface normal in the intersection point.
• Calculation of the direction of the refracted light ray by applying Snell’s law of refraction.
Calculation of the intersection point between the light ray and the corresponding eye surface
A light ray is described in a linear algebraic, parameterized form with a point of origin and a (normalized) direction vector multiplied by a scalar parameterdefine all points along the trajectory of the light ray. Recalling the mathematical formulation of the eye surfaces, a surface based on Zernike polynomials is defined as follows: (details are in Ref. 26):25, 26
Calculation of the corresponding surface normal in the intersection point
With the intersection point determined, we proceed to calculate the normal of the corresponding surface at this point. This is necessary to obtain the new direction vector of the refracted light ray in the next step.
The surface normal is defined as follows:is the partial derivative with respect to of the fitting surface , and is the partial derivative with respect to of the fitting surface . These partial derivatives can be obtained analytically by differentiating Eqs. 1, 2 with respect to and .
Calculation of the direction of the refracted light ray by applying Snell’s law of refraction
To determine the direction of the refracted light ray, we apply Snell’s law of refractionis the angle between the original light ray (before refraction) and the surface normal at the intersection point, is the angle between the refracted light ray and the normal, and and are the refraction indices on either side of the refracting surface. Since the surface normal is already known, one needs only to calculate the new refracted angle and from that the new direction vector for the light ray (details are in Ref. 26). Once the new direction vector is obtained, it only remains to replace the previous one with the new one and to set the point of origin of the light ray as the intersection point with the last surface.
We have performed computer ray-tracing simulations, using simEye,20 of the visual perception under instances of the following four eye conditions:
1. emmetropia (normal vision)
2. regular astigmatism
3. irregular astigmatism
4. central symmetric keratoconus.
All four eye conditions above are characterized by aspheric corneal surfaces.
To create an eye model to be used with the simEye ray-tracing procedure, we have, without loss of generality, rebuilt Gullstrand’s exact schematic eye model (Fig. 2). We have fitted its spherical refractive surfaces (with the exception of the anterior corneal surface, which is to be modeled according to the respective eye condition) and the spherical retina with respective sets of Zernike polynomials both for distance viewing (see Table 1 for surface parameters8) and for maximal accommodation (see Table 2 for surface parameters8). We would like to emphasize that any of these surfaces can be replaced by surfaces that are fitted to more elaborate and realistic eye models (e.g., Refs. 10, 11, 12, 13), otherwise modeled data, or to actual biometric data27 (see also Sec. 4).
Gullstrand exact schematic eye parameters for focus at infinity (⩾5m) (Ref. 8).
|Position (mm)||Radius (mm)||Refractive index|
Gullstrand exact schematic eye parameters for maximal accommodation (10.23cm) (Ref. 8).
|Position (mm)||Radius (mm)||Refractive index|
It is important to note that in the following, the correct viewing distance for the simulated visual perceptions, depicted in Figs. 4, 5, 6, 7, 8 , is a few centimeters from the picture plane with one eye covered (i.e., monocular viewing). The reason for this is that in the following simulations (see Table 3) the source image has a height of and a width of and is viewed from a distance of (distance viewing) with the eye to be simulated (i.e., eye with eye defect). Therefore, we simulate a visual field of about radially, which allows for both on-axis and off-axis line-of-sight aberrations. The dimension of all simulated images is , including the original image source (Fig. 4, top). The simulation results for maximal accommodation are not shown. The diameter of the pupil is user-adjustable in simEye and was set to for all depicted simulation results (see Table 3 ).
1. (a) Emmetropic (normal) visual perception with spherical cornea (Fig. 4, bottom): We used the following expression to generate a spherical anterior corneal surface:(Tables 1, 2). The center of the simulated perception is clear as opposed to the periphery (Fig. 4, bottom), compared to the original image source (Fig. 4, top) used in the simEye simulation. The apparent blurriness in the periphery is, in this case, the result of the assumed sphericity of the cornea, which leads to spherical aberration. However, it resembles the naturally occurring blurry perception in our peripheral vision due to the reduced retinal receptor density. This can be further demonstrated by approaching Fig. 4, bottom, with one eye covered. As one gets closer to the image, the peripheral blurriness seems to disappear. (Tables 1, 2) and . Because of the asphericity of the anterior corneal surface (Fig. 5, top), the peripheral simulated perception is improved (Fig. 5, bottom), that is, it is less blurry, compared to the peripheral simulated perception with a spherical cornea (Fig. 4, bottom).
- , (Tables 1, 2) and . Because the vertical radius of curvature, , of the anterior corneal surface is significantly shorter than the horizontal one, , an instance of regular astigmatism is introduced (Fig. 6, top), which manifests itself as an arc-like, structural image distortion along the vertical ( axis) with the horizontal axis being the symmetry axis (Fig. 6, bottom).
- , (Tables 1, 2), and . The anterior corneal surface for this particular instance of irregular astigmatism was obtained by applying the above formula to the nasal half of the cornea and the swapped set of radii of curvature to the temporal half. The resulting point cloud of input data was subsequently fitted with a set of 51 Zernike polynomials, resulting in the anterior corneal surface depicted in the top part of Fig. 7. The resulting visual perception exhibits five distinct areas (Fig. 7, bottom): the upper left and right are characterized by a more arc-like, structural image distortion akin to the visual perception under regular astigmatism, whereas the lower left and right are characterized by a more Gaussian-type, fuzzy blur without any apparent structure to it. In the image center, a relatively undistorted viewing channel is visible.
- , , , and (Tables 1, 2). The visual perception under this particular instance of central symmetric keratoconus, that is, keratoconus symmetrically centered around the optical axis, exhibits three zones of varying degrees of distortion (Fig. 8, bottom): In the central region, image blurriness paired with slight image enlargement is exhibited because of the increased central corneal thickness ( radially from the optical axis, Fig. 8, top) due to the central symmetric keratoconus compared to the “normal” central corneal thickness marked as a black line in Fig. 8, top. This central region is surrounded by an annular region ( radially from the optical axis, Fig. 8, top) of image blurriness because of the reduced corneal thickness due to the central symmetric keratoconus compared to the normal corneal thickness in that region marked as a black line in Fig. 8, top. Surrounding this annular region is a peripheral region where the regular image blurriness due to the spherical aberration of the anterior corneal surface is exhibited (compare to Fig. 4, bottom).
Parameters and data for surface fitting (columns 1 and 2) and ray tracing with simEye (columns 3 to 6). The computation times are based on an Apple PowerMac G5 Dual 2GHz with 8GB of RAM running Mac OS X Tiger. Only one CPU was used per ray-tracing simulation with simEye.
|Number ofZernikepolynomialsfor fitting||RMSE offit||Outputpicturedimensions(m)||Distancefrom theeye(m)||Calculatedlight raysper pixel||Run timeper still-image(min)|
Table 3 summarizes the parameters, data, and results for surface fitting and ray tracing with simEye for all the eye conditions simulated above.
The computer ray-tracing tool, simEye, presented here permits simulations of the optical properties of the human eye (both on-axis and off-axis line-of-sight aberrations). Further it allows for a first-order approximation of the visual perception under various eye defects without the need for simulating the neural processing of the retina and the visual cortex. This is accomplished by back-projecting through an idealized eye the retinal image of an object or scene produced by an eye with a certain eye defect. Obviously the choice of an unimpaired Gullstrand exact schematic eye model as the back-projecting idealized eye introduces spherical aberrations in addition to the visual effects produced by the eye with eye defects. However, simulations using a Gullstrand eye with an aspheric cornea [see case 1(b) in the Sec. 3] as the back-projecting idealized eye show that this is a minor effect that does not impact the overall visual perception, in particular in the central visual field. A more sophisticated eye model with reduced spherical aberration could be employed to further reduce this side effect.
simEye, in contrast to earlier ray-tracing simulations,4, 15, 16, 17, 18 permits the introduction of arbitrary surfaces (e.g., aspheric surfaces), represented or fitted by a set of Zernike polynomials. Furthermore, the usefulness of Zernike polynomials for fitting actual surface data, in addition to wave-front data (optical aberrations),21, 22 is demonstrated, in agreement with Carvalho’s findings.23
It should be pointed out that, although the corneal surfaces considered in this study are mathematically modeled (i.e., somewhat artificial), the presented Zernike-based surface-fitting framework is equally applicable to realistic, biometrically measured data of any surface within the eye under investigation (e.g., Ref. 27). Furthermore, surfaces that do not exhibit symmetries with respect to the optical axis, such as tilted or laterally dislocated surfaces, can be fitted as well by expanding the surface-fitting framework to incorporate rotation and translation matrices (e.g., Ref. 24) that operate on the sets of Zernike polynomials used for the fits. For example, this would allow for the simulation of off-center, asymmetric keratoconus conditions. Moreover, additional surfaces can be introduced, necessary for more elaborate and realistic eye models (e.g., Refs. 11, 12). For example, this would allow for the simulation of a multishell crystalline lens with varying refractive index.
The Stiles-Crawford effect 28 is currently not considered in simEye since the neural function of the retinal receptors is not modeled. The Stiles-Crawford effect describes the angular dependence of retinal sensitivity. Light rays that enter the pupil near its center (axial light), which are parallel to retinal receptors, are more effective than oblique rays, which enter the pupil near its margins (off-axis light). Therefore, light passing through the periphery of the pupil is less efficient at stimulating vision than light passing near the center of the pupil (i.e., axial light forms sharper images than off-axis light) and hence increases the depth of focus. Since the Stiles-Crawford effect is not modeled, the spherical and aspherical aberrations exhibited in the periphery (i.e., at high eccentricities) of the simulated visual perceptions may be overestimated. One feasible way to mitigate the Stiles-Crawford effect within simEye is to choose a small pupil diameter for the simulations, because the Stiles-Crawford effect, like most aberrations, is bigger with bigger pupils. Similarly, the contrast sensitivity function (CSF), sometimes also called visual acuity, is currently not considered in simEye. The CSF tells us how sensitive we are to the various frequencies of visual stimuli. If the frequency of visual stimuli is too high, we will not be able to recognize the stimuli pattern any more because of the limited number of photoreceptors in the retina. Since the density of photoreceptors in the retina drops exponentially toward higher eccentricities, the effect of the CSF, akin to the Stiles-Crawford effect, can be mitigated in simEye again by using a small pupil size for the simulations. Furthermore, chromatic aberration of the human eye is currently not considered in simEye.
The ray-tracing procedure outlined here is ideally suited for parallel processing on a cluster computer (or computers accessible via the Internet akin to, e.g., SETI@home, http://setiathome.ssl.berkeley.edu/) since each light ray can be calculated independently from each other. Thus, an almost perfect linear speedup with the number of available central processing units (CPUs) can be expected for each still image to be generated with simEye. The same holds true for the generation of ray-traced movies 20where each frame of a movie can be independently calculated from one another. Using simEye as a front end, we have been able to create an optimized software package that allows for the generation of animated movies at a rate of about , that is, one movie frame every , enabling near real-time performance.
simEye may have a wide range of applications in science, optics, and education. This tool may help educate (train) both the lay public, for example, patients before undergoing eye surgery, and medical personnel, such as medical students and professionals. Moreover, simEye may help simulate and investigate optical lens systems, such as cameras, telescopes, microscopes, and robotic vision systems. Furthermore, it may help study the visual perception through multifocal intraocular lenses and through intracorneal lenses. Finally, simEye may be used as a scientific research tool to investigate the visual perception under a variety of eye conditions, in addition to the ones presented here, and after various ophthalmic surgical procedures such as cataract surgery and LASIK (e.g., Refs. 29, 30).
The authors would like to thank Vincent McKoy for a critical review of the manuscript and Wayne Waller for invaluable discussions about the complexity and concept of visual perception. One of the authors (WF) would like to thank Erich W. Schmid for having introduced him to the field of ray tracing in ophthalmology. One of the authors (DM) would like to acknowledge the support of the Summer Undergraduate Research Fellowship (SURF) program at Caltech. This research and the fellowship were supported by National Science Foundation Grant No. EEC-0310723.