1.

Introduction

The performance of a fire-control system is an important index used to characterize the performance of modern fighter planes. If fighter pilots depend only on their eyes when flying over a battlefield, they would not have the advantage in rapidly changing modern warfare. If the fire-control radar that was developed in the 1950s is a milestone in the history of fighter aircraft, then the helmet-mounted display (HMD) is another. HMDs can cross-link short-range air-to-air missiles, the seeker head of which is constantly on alert, to truly realize “first look, first shoot.” Therefore, HMDs allow the pilots to focus only on data displayed on the goggles and frees them from complex instrumentation operations. Since aircraft speeds are faster than ever, traditional onboard aiming devices cannot suit the needs of modern air warfare. Thus, the advent of the HMDs solved this difficult problem. According to a study by the Israeli Research Institute, operational capacities of fighter planes armed with HMDs have increased by three times.

An HMD is a device used in some modern aircraft, especially combat aircraft. HMDs project information in a manner similar to head-up displays (HUDs) on an aircrew’s visor or reticle. HMDs allow pilots to obtain situational awareness and/or cue weapon systems to the direction their head is pointing. Applications that allow cuing of weapon systems are referred to as helmet-mounted sight and display or helmet-mounted sights (HMS).

According to the U.S. Army rotary-wing aviation standards of 2000, for a system to be classified as an HMD it must consist of at least an image source, collimating optics in a head mount, and a head-directed sensor.¹ Figure 1 shows a block diagram showing the four elements of the U.S. Army rotary-wing aviation HMD: image source, display optics, helmet, and head/eye tracker.

Fig. 1

Block diagram of a basic U.S. army rotary-wing aviation helmet-mounted display (HMD).

In 2007, Manning and Rash² provided a more generalized description of HMDs that is pertinent to both military and commercial applications. The same basic four elements are employed, but the scopes are expanded.

There are several HMD classification schemes³ that can be employed, including those based on an image source, image display technology, imagery presentation mode, and optical design approach. For example, Shontz and Trumm³ defined three HMD categories: one-eye, occluded; one-eye, see-through; and two-eye, see-through. This classification scheme was based on the mode by which the imagery is presented to the eyes. The classification of HMDs by optical design is even more complicated. The simpler and more predominant types of HMDs use optical designs based on the reflective and refractive lens elements that relay the HMD image to the eye. There are many other optical design approaches. For example, in 1994, Cameron and Steward⁴ provided an HMD type that is based on a visor projection. Vos and Brandt gave another approach that allows for low weight and provides a compact design using holographic optical elements.⁵ Meanwhile, in 1995, Johnston and Willey proposed a different design⁶ that used lasers to scan and relay an image directly onto the retina of the user’s eye. Finally, an optical design approach,⁷ which uses wave-guide technology, was recently reported by BAE Systems.

2.

Brief Historical Overview Since 1916

The official history of HMDs starts nearly a century ago with Albert Bacon Pratt of Lyndon, Vermont.⁸ During the height of World War I, between 1915 and 1917, Albert was awarded a series of U.S. and U.K. patents for an “Integrated helmet mounted aiming and weapon delivery system,” which is shown in Fig. 2. The concept and the potential applications^9,10 of HMDs have fascinated military strategists for decades. As a result, various militaries from across the world have actively pursued the research, development, application, and fleet introduction of many helmet-mounted technologies for over 40 years. One of the earliest HMD sighting systems to be fielded was the electromechanical linkage head-tracked sight that was used to direct the fire of a gimbaled gun in the U.S. Army’s AH-1G Huey Cobra attack helicopter during the 1970s.¹¹ The first aircraft containing simple HMD devices appeared for experimental purposes to aid in targeting heat seeking missiles in 1975.

Fig. 2

Albert Pratt’s helmet-mounted display.

After the Cobra head tracker system, the Navy introduced an electro-optical head-tracking system into the Phantom F-4J and F-4N fixed-wing jet aircraft and coupled it with the radar and AIM-9H Sidewinder missiles¹² in 1973 to 1979. Honeywell Corporation introduced the visual target acquisition system (VTAS)¹³ that consisted of photodiodes on either side of a halo assembly, which was mounted on the standard fixed-wing flight helmet. The VTAS was the first generation of HMDs whose performance was simple and could only display the azimuth of the signals. The field of view (FOV) was only 3 to 6 deg.

VTAS received praise for its effectiveness in targeting off-boresight missiles; however, the United States did not attempt to field it; it was integrated into late-model Navy F-4 Phantoms equipped with the AIM-9 Sidewinder. HMDs were also introduced in helicopters during this time. VTAS was discontinued in the 1970s because of its technological limitations.

The first complete visually coupled system to achieve operational use was the integrated helmet and display sight system (IHADSS)¹⁴ introduced by the U.S. Army in the AH-64 Apache attack helicopter in 1984. It was a new helmet concept in which the role of the helmet was expanded to provide a visually coupled interface between the aviator and aircraft. The head tracking electro-optical technology in the IHADSS was similar to the Navy’s VTAS. However, the HMD technology was much more capable and provided higher-resolution dynamic video imagery by using a miniature 1 in cathode ray tube (CRT) with relay optics.¹⁵

The Honeywell IHADSS is a monocular system used in all current Apache and Mangusta attack helicopters.¹⁶ A recent major upgrade makes the IHADSS compatible with the Apache’s Arrowhead PNVS thermal imager. The CRT-based display unit attaches to the side of the helmet and feeds the pilotage, navigation, and weapon-aiming symbology and imagery from the nose-mounted infrared camera to the crewmember’s right eye via a half-mirror optical combiner. The helmet, designed for use with night vision goggles (NVG), features a 40 deg horizontal (H) by 30 deg vertical (V) FOV and an electro-optical head tracker, and it may be upgraded by adding a plug-and-play night HUD module.

The Israeli display and sight helmet (DASH) III was the first modern Western HMD to achieve operational service. The development of DASH began during the mid-1980s when the Israeli Air Force issued a requirement for F-15 and F-16 aircraft. The first design entered production around 1986, while the current GEN III helmet entered production during the early-to-mid 1990s. The DASH GEN III featured a completely embedded design in which the complete optical and position sensing coil package was built into the helmet, either the United States Air Force standard HGU-55/P or the Israeli standard HGU-22/P, using a spherical visor to provide a collimated image to the pilot.¹⁷ A quick-disconnect wire powered the display and carried the video drive signals to the helmet’s CRT. DASH was closely integrated with the aircraft’s weapon system via an MIL-STD-1553B bus.

The DASH HMS system by Elbit Systems was the second generation of HMDs that could display the azimuth of the signals as well as the customary flight and navigation data. Its FOV reached 20 deg; however, the majority of the second-generation HMD systems¹⁸ were monocular. Therefore, the scope of its applicability was limited. The DASH III has been exported and integrated into various legacy aircrafts, including the MIG-21. It also forms the baseline technology for the U.S. joint helmet-mounted cueing system (JHMCS).

After the U.S. withdrawal from the advanced short range air-to-air missile, the U.S. pursued and fielded the JHMCS in conjunction with the Raytheon AIM-9 X for the 12th and 19th Fighter Squadrons at Elmendorf AFB, Alaska. The Navy conducted a research development test and evaluation on the F/A-18C as the lead JHMCS platform, but instead fielded it first on the F/A-18 Super Hornet E and F aircraft in 2003. The USAF is also integrating the JHMCS into its F-15E and F-16 aircrafts.

The JHMCS utilizes magnetic head tracker technology and provides a monocular visor-projected display of stroke-written dynamic symbology from a ½-inch miniature CRT and relay optics.¹⁹ The JHMCS provides a daytime air-to-air and air-to-ground off-boresight targeting capability, which is valuable when used with the high off-boresight missile seeker technology.

The third generation of HMDs, led by the JHMCS of the U.S. Air Force, has $>40\mathrm{deg}$ FOV, can display all the HUD data on the goggles, and can display an image or video signal from the night vision or infrared imaging equipment.

3.

Optical Design for HMD

The HMD optical system delivers image or video information, which comes from the source images to the pilot’s sight.^20,21 It is not only relevant to the image quality, but also interrelated with weight and comfort of HMDs. Thus, a key question is how to design an HMD optical system.

The idea of a single optimal HMD design is an unobtainable goal because of the many variations in user tasks and the users themselves.²² For example, the specifications for an HMD designed for the pilot of a fighter jet flying at 10,000 feet will not meet the needs of a helicopter pilot flying close to the ground. Therefore, many of the performance requirements and tradeoffs are based on the applications of the user and environment.

There are a number of important parameters²³ in an HMD optical design. These include (1) FOV, (2) exit pupil (eye box) size and shape, (3) optical eye relief, (4) transmission (optical throughput), (5) beamsplitter transmission/reflection coefficients for see-through HMDs, (6) modulation transfer function (MTF), (7) distortion, (8) weight, (9) center-of-mass (CM), and (10) volume or the space required.

While it may be tempting to identify only a select few of these parameters as being universally important, the intended use of the HMD is, in fact, the deciding factor for which parameters should push the optical design. Nonetheless, there are a few optical system parameters that are fundamentally important to the vast majority of designs, especially for avionic HMDs. These include weight, FOV, MTF, exit pupil size, and eye relief.

The weight of the optics comes from the optical elements themselves, such as lenses, mirrors, prisms, beamsplitters, and the housing for these optical elements. We can control the weight of the optics by choosing all kinds of materials used for the optical elements. Optical designers have begun to explore ways of replacing traditional optical devices that had rotational symmetry in the late 1980s. Sweatt, who pioneered the holographic optical element field,²⁴ had a student named Chen who made great progress in reducing the weight of optical system.²⁵ Since 1996, diffractive optical elements²⁶ have been used in HMDs to reduce the weight, axial distortion, transverse distortion, and chromatic aberration. In addition, holographic elements offer additional weight savings. In 1992, Wood determined the use of holographic beamsplitters (combiners) in the refractive optics of the HMD optical designs,²⁷ which utilized their wavelength-selective characteristics and did not introduce any additional optical power.

A lightweight HMD is important for reducing wearable fatigue. The weight associated with the optics is important from both the ergonomic and safety perspectives. The additional head-supported weight of the HMD can produce neck muscle fatigue, which can degrade performance and increase the potential of injury due to dynamic loading during crashes. Thus, it is desirable to minimize the head-supported weight in HMD designs.

In most cases, image source makes up a portion of the weight. The image source is a display device which uses a small high-resolution CRT²⁸ or liquid-crystal display (LCD).^29,30 However, the CRT is bulky and heavy. Moreover, it has high power consumption and requires a high voltage. Thus, these deficiencies constrain its HMD application. A new revolutionary technology, OLED displays,³¹ took the place of LED displays and liquid crystal on silicon displays, which replaced the mini-CRTs during the past decade. The first generation high-definition OLED displays have been integrated and they have reduced the weight, power consumption, and operating voltage.³²

Another fundamental optical parameter is the FOV,³³ which describes how extensive the image appears to the user. The FOV can be formally defined as the maximum image angle of the view that can be seen through an optical device. The FOV is affected by the magnification and image source size in which a greater magnification and/or image source size results in a larger FOV. Typically, HMDs present an FOV to the viewer that matches one-to-one (conformally) with the FOV of the sensor that is used to capture the original image of the outside world. In principle, a larger FOV allows for more information to be made available, assuming that the image source and sensor have the resolution to properly support the increased FOV. Consequently, HMDs designed for pilotage attempt to maximize the FOV and match the human visual system. The human eye^34,35 has an instantaneous FOV that is roughly oval and typically measures 120 deg vertically by 150 deg horizontally. Considering both eyes together, the overall binocular FOV measures $\sim 200\mathrm{deg}$ (H) by 120 def (V).³⁶ Figure 3 shows the human visual system’s binocular FOV.

Fig. 3

Human visual system’s binocular field of view.

If our goal is to create an opaque fully immersive visual environment for gaming, simulation, or training, a large FOV would be desirable to stimulate the ambient visual mode and provide a more compelling immersion. One example of a wide-FOV HMD is the U.S. Army’s Aviation Combined Arms Tactics Trainer, which is a mobile reconfigurable training system for helicopter pilots that relies on the HMDs for all the out-the-window visuals.³⁷ This system uses a Rockwell Collins’s HMD that provides a 100 deg (H) by 52 deg (V) FOV, which was recently upgraded to SXGA resolution. Conversely, if the goal is a safety-of-flight-qualified HMD, then the head-supported weight and CM become important, and a more moderate FOV of 40 deg (H) by 30 deg (V) is acceptable. The IHADSS is an example of a 40 deg (H) by 30 deg (V) FOV that has been successfully used in the U.S. Army’s AH-64 Apache helicopter since the early 1980s.³⁸

The MTF³⁹ is a metric that defines how well an optical system transfers the modulation contrast from its input to its output as a function of spatial frequency, which is a measure of detail in a scene and is usually defined as how rapidly the luminance changes within a region. Figure 4 shows a plot of such a transfer, which is called an MTF curve.

Fig. 4

Typical modulation transfer function (MTF) curve.

Within an HMD system, every major component, e.g., sensor, image source, optics, has its own MTF. If the system is linear, its total MTF can be obtained by multiplying the MTFs of the system’s individual components.⁴⁰ To accurately predict the image quality of an HMD system, it is necessary to determine how the overall system will affect the resolution and contrast. The MTF performs this function. The MTF of an optical system is perhaps the most widely accepted metric for the image quality seen through the optical system. It defines the fidelity to which an outside scene is reproduced in the final viewed image. A perfect system would have an MTF of one across all spatial frequencies.⁴¹ The degradation that is present in a practical HMD optical system’s MTF is a result of the residual (uncorrected) aberrations in the system and is limited by the diffraction effects.

The exit pupil and eye relief presented in Fig. 5 are closely related. The exit pupil is the volume in space where the eye must be placed in order to see the full image. An exit pupil has three characteristics: size, shape, and location. Tsou⁴² suggests that the minimum exit pupil size should include the eye pupil ($\sim 3\mathrm{mm}$), an allowance for eye movements that scan across the FOV ($\sim 5\mathrm{mm}$), and an allowance for helmet slippage ($\pm 3\mathrm{mm}$). This would set a minimum exit pupil diameter of 14 mm. A large exit pupil is important for a flight HMD, so the user does not lose the image if the HMD shifts on his head. A value of 12 to 15 mm has been deemed an acceptable value for these applications.

Fig. 5

Diagrams showing the optical (a) and physical (b) eye relief. The optical eye relief is defined as the distance from the last optical element to the exit pupil, where the eye would be placed. The physical eye relief can be less than the optical eye relief if additional structures are present.

The exit pupil is located at a distance called the optical eye relief, which is defined as the distance from the last optical element to the exit pupil. The HMD needs sufficient eye relief in order to allow the user to wear spectacles. This produces a minimum value of 25 mm.⁴³ However, care must be taken with this terminology because in classical optical design the eye relief is measured as the distance along the optical axis from the last optical surface to the actual exit pupil. In most HMDs, the final optical surface in front of the eye may be an angled combiner, which will fold the optical path to get the rest of the optics away from the front of the face. Thus, the actual eye clearance distance (ECD), measured from the face to the closest point of the combiner, may be considerably less. Therefore, it is important that the useable distance from the eye to the first contact point of the HMD optics, the ECD, provides a minimum of 25 mm separation.⁴⁴

There are a number of HMD optical designs^45,46 that have been used over the decades of HMD development. The following descriptions encompass three optical design approaches which are popular and are only representative of the many varied designs that have been implemented.

3.1.

Off-Axis Design

Compared to the coaxial HMD system, the off-axis HMD⁴⁷ can increase the exit pupil and eye relief, be built for an optimal man-machine configuration, simplify the system structure, improve the lighting proportion of the system, and eliminate the ghost image. Similar to other off-axis optical systems, the off-axis HMD involves many complex technical problems. At present, the aberration theory based on the coaxial optical system is difficult to competently guide work of designing an off-axis optical system. Nodal aberration theory has brought fresh hope and new opportunity to design off-axis optical systems.^48–52

According to the nodal aberration theory, the wave aberration expansion through third order is

Eq. (1)

$$\begin{gathered} W=\mathrm{\Delta }{W}_{20}(\overrightarrow{\rho }.\overrightarrow{\rho })+\mathrm{\Delta }{W}_{11}(\overrightarrow{H}.\overrightarrow{\rho })+\sum _j\mathrm{\Delta }{W}_{040j}{(\overrightarrow{\rho }.\overrightarrow{\rho })}^2 \\ +\sum _j{W}_{131j}[(\overrightarrow{H}-{\overrightarrow{\sigma }}_j).\overrightarrow{\rho }](\overrightarrow{\rho }.\overrightarrow{\rho })+\sum _j{W}_{222j}{[(\overrightarrow{H}-{\overrightarrow{\sigma }}_j).\overrightarrow{\rho }]}^2+\sum _j{W}_{220j}[(\overrightarrow{H}-{\overrightarrow{\sigma }}_j).(\overrightarrow{H}-{\overrightarrow{\sigma }}_j)](\overrightarrow{\rho }.\overrightarrow{\rho }) \\ +\sum _j{\mathrm{W}}_{311j}[(\overrightarrow{H}-{\overrightarrow{\sigma }}_j).(\overrightarrow{H}-{\overrightarrow{\sigma }}_j)][(\overrightarrow{H}-{\overrightarrow{\sigma }}_j).\overrightarrow{\rho }], \end{gathered}$$

where the subscript $j$ is the surface number, $W_{\mathrm{klm}}$ are the wave aberration coefficients, $H$ is the vector that locates the image point of interest in the focal plane, $\rho$ is the aperture vector in the exit pupil, and ${\sigma }_j$ is the surface by surface location of the center of the aberration field for each surface, which is a vector residing in the Gaussian image plane.⁵³ In the decentered/tilted optical system, there is a leveraging binodal astigmatism given by⁵⁴

Eq. (2)

$$W_{\text{AST}}=\frac{1}{2}\sum _j{W}_{222j}[{(\overrightarrow{H}-{\overrightarrow{\sigma }}_j)}^2.{\overrightarrow{\rho }}^2].$$

If $W_{222}\ne 0$, them

Eq. (3)

$$W_{\text{AST}}=\frac{1}{2}{W}_{222}[{(\overrightarrow{H}-\overrightarrow{a_{222}})}^2+{\overrightarrow{b_{222}}}^2].{\overrightarrow{\rho }}^2,$$

Eq. (4)

$${\overrightarrow{a}}_{222}\equiv \frac{{\overrightarrow{A}}_{222}}{W_{222}}=\frac{1}{W_{222}}\sum _j(W_{222j}^{(\mathrm{sph})}{\overrightarrow{\sigma }}_j^{(\mathrm{sph})}+W_{222j}^{(a\mathrm{sph})}{\overrightarrow{\sigma }}_j^{(a\mathrm{sph})}),$$

Eq. (5)

$${{\overrightarrow{b}}_{222}}^2\equiv \frac{{{\overrightarrow{B}}_{222}}^2}{W_{222}}-{{\overrightarrow{a}}_{222}}^2=\frac{1}{W_{222}}\sum _j(W_{222j}^{(\mathrm{sph})}{\overrightarrow{\sigma }}_j^{{(\mathrm{sph})}^2}+W_{222j}^{(a\mathrm{sph})}{\overrightarrow{\sigma }}_j^{{(a\mathrm{sph})}^2})-{{\overrightarrow{a}}_{222}}^2.$$

Off-axis catadioptric systems are usually referred to as reflective off-axis systems and may or may not require combiners. As the off-axis angle to the power combiner increases, the induced distortions and aberrations increase rapidly.⁵⁵ An example⁵⁶ of an off-axis catadioptric design with a combiner is shown in Fig. 6. This catadioptric design achieves a 50 deg (H) by 38.5 deg (V) FOV with a 10 mm exit pupil and 60.46 mm eye relief. The display apparatus enables the observation of an image that is flat and clear with visual field at a view angle of 40 deg or more, which ensures a large exit pupil diameter and eye relief. Not only are the lenses tilted, but also there is an asymmetric aspheric element that is used to help correct the asymmetric aberrations caused by the combiner, which is an anamorphic asphere surface and ocular concave mirror.

Fig. 6

Visual display apparatus comprising a decentered correcting optical system.

For the design of compact optics for HMDs, optical designers have determined that DOEs⁵⁷ may be used in place of doublets for the correction of chromatic aberrations because of the strong negative chromatic dispersion of DOEs. The concept is an improvement which utilizes one or more diffractive optical surfaces to provide a system that operates with fewer elements. This makes them simper, lighter, and easier to assemble than the previous systems. Figure 7 shows a head-mounted display utilizing DOEs.⁵⁸ With the diffraction surfaces 58 and 59, the system is simplified further to one having only three spherical and centered lenses.

Fig. 7

Head-mounted display utilizing diffractive optical elements.

Due to the advances in the optical design and manufacturing technology, plastic lenses have been used in HMD design. Chen et al. designed a lightweight HMD⁵⁹ that includes a group of plastic lenses, which is shown in Fig. 8. It is possible to reduce the weight and cost of the display device by using optical-quality plastic elements. Compared with a conventional design, the weight of the relay group is reduced by 60%, which in turn reduces the moment of inertia of the display device. In addition, the plastic elements are located between the glass optical wedge and glass positive-power lens module, which protects them from scratching and other damage.

Fig. 8

Lightweight HMD with plastic lenses.

The primary advantage^60–62 of the off-axis reflective HMD design is that it provides the highest theoretical luminance transfer from the display with the highest see-through vision and increased eye clearances for a given FOV. The primary disadvantages are very complex optical designs, shape distortions, as well as low structural integrity and stability of the reflective surfaces.

3.2.

Design with Freeform Optical Surface

A freeform optical surface is typically defined as a surface that is complicated, irregular, and nonrotationally symmetric.⁶³ These surfaces provide additional degrees of freedom that can lead to improved performances compared to systems that only use conventional optics.⁶⁴ Because of the fabrication possibility of freeform surfaces, the use of freeform optics is becoming more widespread in the optical system design, which has opened new avenues of modern optical research and development.⁶⁵

There is no single universal equation that can describe the geometry of the freeform surfaces and can be described by a myriad of equations including $x\text{-}y$ polynomials, Zernike polynomials, and $\phi$-polynomials. There is a new way to describe freeform surfaces with radial basis functions (RBF): a meshless surface description, which was first applied to optical system design by Cakmakci et al.⁶⁶ They performed research on an off-axis magnifier configuration that can be used in HMDs, which is shown in Fig. 9. The study gives a comparison between a 10th-order anamorphic sphere, $x\text{-}y$ polynomial, 10th-order Zernike polynomial, and linear combination of Gaussians for a system containing the same parameters: $>15\mathrm{mm}$ eye relief, 3 mm exit pupil, and 24 deg (H) by 14.7 deg (V) FOV.

Fig. 9

Optical layout of the off-axis magnifier represented using a linear combination of Gaussians.

If the theory and promotion of application in this field is improved, then the field of optical engineering will benefit. Thus, we design an HMD with a large FOV, large pupil size, and long exit pupil relief through the use of freeform surface described by Gaussian RBF.⁶⁷ An off-axis see-through HMD that is composed of a tilted combiner with an RBF surface representation is achieved. The system, which is shown in Fig. 10, has a 100 mm eye relief, 15 mm pupil, 45 deg (H) by 32 deg (V) FOV, and 60 deg combiner tilt angle.

Fig. 10

An off-axis HMD using freeform surface described by radial basis functions: (a) optical layout and (b) polychromatic MTF plot of the HMD.

The HMD was designed, analyzed, and compared to existing shape descriptors such as an asphere and Zernike polynomial in this paper. From the results, we discovered that the use of a linear combination of Gaussian surfaces on one hand could reduce the number of optical elements, simplify the structure of optical system, and reduce weight. Conversely, it also exhibits higher levels of MTF performances. Specifically, there was one less optical element in the relay lens group compared to an asphere and a 12.1% gain at the spatial frequency of $33\text{line pairs}/\mathrm{mm}$ compared to a Zernike polynomial. The comparison of the properties between the linear combination of Gaussian surfaces and other surface types is listed in Table 1.

Table 1

Comparison between the average tangential and sagittal modulation transfer function (MTF) values and number of relay lens between surface types of asphere, Zernike polynomial, and a linear combination of Gaussians.

Surface type	Average MTF (at $33\mathrm{cyc}/\mathrm{mm}$)	Number of relay lens
Asphere	40.8%	5
Zernike polynomial	55.1%	4
Linear combination of Gaussians	67.2%	4

Compact constraints are the primary motivation for optical designers to use freeform surfaces in HMDs and many astronomical applications.⁶⁸ Togino et al. provided a design⁶⁹ with freeform surfaces, which is shown in Fig. 11. In this design, the first, second, and third surfaces, 3, 4, and 5, are all three-dimensional surfaces defined by

Eq. (6)

$$Z=C_2+C_3\mathrm{y}+C_{4}x+C_5{y}^2+C_{6}yx+C_7{x}^2+C_8{y}^3+C_9{y}^{2}x+C_{10}y{x}^2+C_{11}{x}^3+C_{12}{y}^4+C_{13}{y}^{3}x+C_{14}{y}^2{x}^2+C_{15}y{x}^3+C_{16}{x}^4+C_{17}{y}^5+C_{18}{y}^{4}x+C_{19}{y}^3{x}^2+C_{20}{y}^2{x}^3+C_{21}y{x}^4+C_{22}{x}^5+C_{23}{y}^6+C_{24}{y}^{5}x+C_{25}{y}^4{x}^2+C_{26}{y}^3{x}^3+C_{27}{y}^2{x}^4+C_{28}y{x}^5+C_{29}{x}^6+C_{30}{y}^7+C_{31}{y}^{6}x+C_{32}{y}^5{x}^2+C_{33}{y}^4{x}^3+C_{34}{y}^3{x}^4+C_{35}{y}^2{x}^5+C_{36}y{x}^6+C_{37}{x}^7\dots$$

where $\mathrm{Z}$ is the amount of deviation from a plane tangent to the origin of the surface configuration, and $C_i(i=2,3\dots )$ are coefficients.

Fig. 11

Olympus HMD apparatus comprising a rotationally asymmetric surface.

This design achieves a 40 deg (H) by 30.5 deg (V) FOV with an 8 mm exit pupil. It is very compact and lightweight so that when it is combined with a head-mounted support structure to hold the display in proximity to the eye, only several ounces are added to the total weight. One particular advantageous feature of this design is that a given FOV may be obtained with a relatively small thickness associated with the optical system, thereby producing a lightweight and small volume HMD. Such a display is achieved with excellent optical properties.

As a complicated and nonrotationally symmetric surface, freeform optics can break through the concept of the conventional optical system and be used in a new optical system design.⁷⁰ The benefits of using freeforms in an optical system design are as follows:^65,71,72

• 1. Less optics can be used in the optical system, which decreases the amount of optical surfaces. Since every surface is a reduction of light intensity, e.g., by scattering, a higher throughput for the optical system occurs.

• 2. Less optics also means a reduction in weight and size.

• 3. An improvement in optical quality, e.g., spherical aberration, coma, distortion.

• 4. A more favorable position of the optical components is possible.

Conversely, the disadvantages that come with using freeforms are as follows:^73,74

• 1. Difficulty in determining the optimal freeform representation and location in the optical train.

• 2. Optical tolerance analyses are not yet common practice in optical design packages.

• 3. Difficulty in manufacturing with classical production technologies.

• 4. Difficulty in validating the surface shape.

• 5. Higher difficulty in aligning because of increased degrees of freedom.

• 6. More expensive to manufacture.

3.3.

Design with Holographic Waveguide Technology

In recent years, the rapid development of integrated optics and microelectronics technology has made holographic waveguide technology widely used in image displays,⁷⁵ LCD illumination,^76,77 and optical interconnection.⁷⁸ Holographic waveguide technology is a revolutionary way of designing HMDs. In this method, HMDs can obtain a compact structure, light weight, large exit pupil, and excellent real-world transmission.⁷⁹ Optical waveguide technology relies on two fundamental optical principles:^80,81

• 1. Total internal reflection (TIR): The ability of an optical medium such as glass to completely contain rays of light under certain conditions, thereby allowing the light to propagate freely and efficiently through the medium.

• 2. Diffraction: The ability of regular structures with a periodicity comparable to the wavelength of light to modify the direction of light passing through them.

The holographic optical waveguide display consists of three important parts: the input coupler, waveguide, and hologram as shown in Fig. 12.⁸² The input coupler is the mechanism that conducts the light from a source into the waveguide through TIR. This function can be achieved using a prism, grating, or edge-lighting mechanism. The waveguide for the holographic optical waveguide display is a sheet of transparent material with two surfaces, which are locally parallel and optically polished. The refractive index of a waveguide must be higher than the index of its environment to achieve waveguiding. The coupled wave is confined inside the waveguide through TIR on the waveguide surfaces and propagates along following a zigzag path. The hologram is placed parallel to and immediately in contact with the waveguide. When the hologram is illuminated with the guided wave, the previous recorded holographic image is reconstructed.

Fig. 12

The basic structure of a holographic optical waveguide display.

A basic holographic optical waveguide display configuration for expanding light beams from a small source in two dimensions is shown in Fig. 13.^83–85 It comprises three laterally displaced linear holographic gratings (HGs) that are recorded on a single transparent substrate. The first HG $H_1$ couples the input incident light from the display source into the substrate, traps it through TIR, and directs it toward the second HG $H_2$, which expands the light in one dimension. The second HG $H_2$ intermediate redirects the light distribution toward the much larger third HG $H_3$, which expands light in the other orthogonal direction and decouples it from the substrate outward toward the viewer who can obtain a large uniform light beam.

Fig. 13

Simplified diagram of a holographic optical waveguide display. Holographic waveguide displays use this combination of total internal reflection and diffraction to take a small exit pupil image from a display and expand the exit pupil in the vertical and horizontal axes to produce a large exit pupil display.

Another basic holographic optical waveguide display configuration, which is also suitable for displays, is shown in Fig. 14.^86–88 The configuration comprises two holographic elements: a linear grating (LG) and holographic lens (HL), both of which are recorded on one substrate. The first holographic lens diffracts the incident light from the display source so it will be trapped inside the substrate through TIR, while the second hologram, which is just a linear grating, diffracts the light out from the substrate toward the observer.

Fig. 14

A basic holographic optical waveguide display configuration.

Holographic optical waveguide technology has been applied to provide radically new types of HMDs and HUDs. The first HMD products were the Q-Sight™ family⁸⁹ of HMDs specifically designed for rotary-wing applications, which were developed by BAE Systems, London, Britain. Using patented technology, the Q-Sight™ family employs a revolutionary means of moving light using holographic waveguides. It offers a modular approach to providing pilots with a heads-up/eyes-out capability, while at the same time delivering mission-critical situational awareness with significant improvements in weight, cost, flexibility, simplicity, and optical performance. The basic configuration is shown in Fig. 15, which highlights how the waveguide concept has eliminated the need for complex optics assemblies, while also integrating into standard aviator helmets. Furthermore, Sony Corporation, Tokyo, Japan developed a see-through full color eyewear display using holographic planar waveguides,⁹⁰ which is shown in Fig. 16. Using a 20 deg diagonal FOV, a 120% National Television Systems Committee color gamut, $<0.008\mathrm{\Delta }{u}^{\prime }{v}^{\prime }$ color uniformity, and $>2500\mathrm{cd}/{\mathrm{m}}^2$ brightness were achieved.

Fig. 15

Q-Sight™ 100 display with holographic optical waveguide technology.

Fig. 16

A basic structure of the holographic planar waveguide from Sony Corporation.

We have designed HMDs with holographic optical waveguide technology. The HMDs with 14 deg (H) by 14 deg (V), 4 mm pupil, 30 mm eye relief, and 550 nm operating wavelength were designed. An MTF close to the diffraction limit over the entire FOV was achieved. The maximum RMS spot radius value is 2.9 μm at ($x=-7\mathrm{deg}$, $y=-7\mathrm{deg}$). The magnitude of the maximum distortion occurs at ($x=7\mathrm{deg}$, $y=-2.8\mathrm{deg}$) in the field and is simulated to be $-2.62\%$. The simulation results show that the optical system of HMDs has a low aberration and can provide user clear symbols or video. This HMD, which has good image quality, small volume, and a light weight, can be applied to the next-generation HMD technology. The configuration, MTF curves, and distortion of the HMDs are shown in Fig. 17.

Fig. 17

An HMD based on the holographic planar waveguide. (a) Configuration and (b) MTF curves of the HMD.

4.

Current and Future Avionic Helmet-Mounted Displays

The various militaries across the world have actively pursued the research, development, application, and fleet introduction of a variety of helmet-mounted technologies for over 40 years. There are many types of HMD programs^91,92 that have achieved at least limited fielding and some are still in their research and development phase. While the HMDs have been used by both vehicular-mounted and dismounted warfighters recently, they are predominantly used in rotary- and fixed-wing aircraft platforms.

The next few paragraphs will provide a summary⁹³ of some of the HMDs that are used in rotary- and fixed-wing aircraft platforms and are being fielded today or in the next few years. Table 2 presents a partial summary of the more notable experimental, prototype, fielded, and future HMD fixed-wing programs. Many of these HMDs are depicted in Fig. 18. Many of the programs involve a number of contracts with various commercial HMD developers that play different roles and are also multinational in scope. The country of development listed in Table 2 and ensuing program descriptions is generally based on the initial developmental phase.

Table 2

Summary of selected fixed-wing helmet-mounted display (HMD) programs.

Fig. 18

Selected current and future fixed-wing HMD programs.

Table 3 presents a partial summary of the more notable experimental, prototype, fielded, and future HMD rotary-wing programs. Many of these HMDs are shown in Fig. 19. Many of the programs involve a number of contracts with various commercial HMD developers that play different roles and are also multinational in scope. The country of development listed in Table 3 and ensuing program descriptions is generally based on the initial developmental phase.

Table 3

Summary of selected rotary-wing HMD programs.

Fig. 19

Selected current and future rotary-wing HMD programs.

A complete overview of the HMD would be difficult as there have been hundreds of head tracker and HMD development efforts. This paper introduces three representative HMDs, which include the JHMCS, TopOwl®, and Q-sight™.

5.

Conclusion and Future Research Directions

We presented a review of HMDs focusing on the optical design aspects. Based on the user’s characteristics and applications, we summarized a few optical system parameters that are fundamentally important to the vast majority of designs, especially for avionic HMDs. These include weight, FOV, MTF, exit pupil size, and eye relief.

With the constant requirements for high performance, a greater number of new techniques and components have been applied to the design of HMDs. We gave descriptions that encompass three popular optical design approaches that are only representative of the many varied designs that have been implemented.

Next, we presented a brief synopsis of the more significant HMD programs. The summary lists some of the HMDs that are used in rotary- and fixed-wing aircraft platforms and are being fielded today as well as in the next few years.

The main task of an embedded optical system in avionic HMDs is to provide a clear and bright aiming symbol or video images that can be superimposed on the real world. Further, the main demands of the optical system are a large FOV, large pupil size, long exit pupil relief, clear image, low profile package, lightweight, and small barycenter offset. Thus, the purpose of the optical design is to solve the conflict and seek balance through the use of all kinds of feasible techniques. According to the main demands of an optical system that is incorporated into avionic HMDs, several performance parameters will be prime drivers of future see-through HMD designs.

We believe that the potential use of freeform optics for avionic HMDs is very promising. There are some opportunities for research in terms of determining the optimal freeform representation and location in the optical train. Besides, further foreseeable opportunities in avionic HMD design also include the holographic waveguide technology, which is a novel and exciting approach. It can provide aircrews as well as other users with an enhanced situational awareness in a compact, lightweight, and low-cost product that is simple to install and use. Future developments of the holographic waveguide technology include a large FOV $>40\mathrm{deg}$, high-resolution displays, full color capability, low color crosstalk, and very low profile.