For symbology to be viewed in a see-through helmet-mounted display (HMD) or head-up display (HUD), the luminance of the symbology must be sufficient to distinguish it from the see-through background. This is true whether or not symbology is displayed on a monochromatic or full-color display. When the contrast of the transparent symbology is sufficiently high, the symbology appears as an overlay on the ambient scene. In order for an HMD or HUD to be usable in an operational environment, the luminance requirements must take into consideration the type of displayed imagery (e.g., symbology, situational maps, target sights), the tasks (e.g., targeting, navigation, obstacle avoidance), the operational setting (e.g., day/night, terrain features), additional hardware (e.g., visors, windscreens, laser protection), and other considerations. For a color HMD, symbology color overlaid on an ambient scene should consider luminance and color contrast, which requires information about the spectral content of the landscape or ambient scene. However, optical designers must consider all terrain features when designing a see-through optic as the military can be deployed to any geographical location around the globe; this fact may simplify the development of luminance requirements for see-through, full-color HMDs. The reason that this will simplify the development of a contrast requirement is that luminance contrast, and not color contrast, is the determining factor.
Modeling Helmet-Mounted Display Symbology
As an initial step in defining the luminance requirements for daylight symbology, Harding et al.1 processed simulated images of white symbology (simulating a full-color HMD with all three primary colors displayed) overlaid on selected static natural backgrounds (Fig. 1). Observers evaluated the quality of symbology (Table 1) in “contrast correct” images of symbology overlaid over eight natural backgrounds (an example is shown in Fig. 2), one artificial background (artificial clutter in Fig. 1), and one uniform field. The overlaid symbology shown in Fig. 2 was created by use of a software model1 that applied appropriate luminance scaling to yield a symbology image that has been added to the background image. In this experiment, the complexity of the background image was of paramount importance when determining the minimum luminance requirements for see-through symbology. This finding is in general agreement with other studies23.–4 of transparent text against complex backgrounds. The luminance of the background image was of less importance.
Rating scale and description of ratings given to subjects in Harding et al.1
|Rating||Quality||Description of rating|
|7||Excellent||All letters and symbols are easily seen with high contrast.|
|6||Very good||All letters and symbols are easily seen with good contrast.|
|5||Good||All letters and symbols can be seen with reduced contrast.|
|4||Adequate||All letters and symbols can be deciphered with a little difficulty.|
|3||Poor||Letters and symbols can barely be detected and some letters or symbols are very difficult to see.|
|2||Not adequate||Some of the letters and symbols cannot be seen.|
|1||Not usable||Difficult to recognize that symbology is present.|
For each background scene, 20 symbology images of varying contrast were evaluated by observers. Following a training session, observers judged the quality of symbology for each of the 200 images in three test sessions. All observers were required to pass an intermediate-field acuity test (distance of 22 in.) and a Farnsworth 15-Hue color vision test. There were no gender or age-specific requirements. Twenty volunteer observers were selected for the study, however, for an observer’s data to be included in the final data analyses, critical judgment criteria were established and only data from seven of the observers were selected for final data inclusion (see original paper1 for selection metrics). For each of the 200 images, an average Michaelson contrast was calculated for each image. To make calculations easier, block symbology images were processed in exactly the same fashion as the symbology image. For example, for each of the 20 blocks shown in Fig. 2(b), equaled the average luminance for each of the 20 symbology blocks. Likewise, was the average calculated luminance of 20 areas, of the same size adjacent to them.
The size of each of the background images was . The patch size was . Luminance values were calculated for each subpixel using the following equations:Fig. 3(a)]. The coefficients 0.23, 0.67, and 0.10 provide an average fit to the relative luminance contributions of the subpixels. The luminance of each pixel, , equaled the sum of the subpixel luminances and 1 simulated luminances at the eye ranged from a low of 7 fL for the Ground Clutter 2 image to a high of 6198 fL for the uniform background. To evaluate visual sensitivity differences over the simulated photopic ranges and the actual display output, we used Barten’s model5,6 to calculate static contrast sensitivity for average luminances of 10, 100, 1000, and 10,000 fL [Fig. 3(b)]. Symbology characters were 37.5 arcmin wide with a 7.5 arcmin gap. The gap size (if the gaps were continuous) corresponds to a spatial frequency of . Calculating the contrast threshold range () at reveals an insignificant difference of in the four curves suggesting that photopic simulations can be made at the monitor’s reduced luminance.
The minimum contrast for symbology was judged to be equal to an observer rating of 4 on the seven-point scale, as described in Table 1. Harding et al.1 found that the following equation provided a reasonable fit for describing the luminance required for a minimum contrast rating of 4.0:
When Michaelson contrast was plotted as a function of observer ratings, the curves for different background images were quite disparate (Fig. 5). When background complexity was added as a codeterminant, the plots became much tighter [see Fig. 7(b) in Ref. 1].
Background Clutter in Natural Scenes
Although the natural background images were deemed representative, the imagery was static and not dynamic. During actual flight, flight symbology would appear overlaid over a changing ambient scene. Motion tends to blur or reduce the local variations in an ambient scene (especially in nap of the earth flight), and the background relative to the symbology has a temporal modulation component. At minimal contrast, symbology characters would likely appear recognizable in one instance and perhaps partially disappear in the next instance as the complexity and luminance flow of the ambient image waxed and waned. There is certainly a dynamic feature to the visibility and quality of symbology, and taking motion into account would likely yield more accurate results than simply evaluating static symbology. Analyzing the effect of motion on symbology visibility and quality, however, is beyond the scope of this paper. Since the quality of symbology is based on the visibility of each character or indicator against its background, we expressed the in Eq. (1) as the average percent standard deviation of small patches of background. In this way, the background complexity, as measured by the standard deviation, would always be relevant to the overlaid symbology. We coined the term “local contrast” to describe the contrast between the luminance within a character or indicator divided by the average luminance of adjacent patches that subtended an area equal to about the size of a symbology character.5,7
By setting the maximum pixel luminance in each of the nine images shown in Fig. 1 to 5000 fL, a patch analysis distribution of luminance versus complexity produced scatter grams whose upper envelope could be well described by the following equation:Fig. 4, over 99.9% of the patch complexity data falls on or below the curve from Eq. (2).
Observer Ratings of Symbology
In the original study,1 observers evaluated the quality of symbology in 200 images, 20 images per background image. Average contrasts were calculated for each of the 200 images, and the contrasts varied widely depending on the background images. The average Michaelson contrast was calculated for each of the images and these contrasts were plotted as a function of the average observer ratings. Figure 5 shows the results for all 10 of the background images. Least square power functions were fit to each image set ( based on curves fit to data points at or below an observer rating of 6.5). For each background image, observer ratings of 4, 5, and 6 were calculated from the curve fits. For six of the images, observer ratings of 4 or lower were not observed, and extending the curve fits allowed calculations of the minimum contrast rating (i.e., see data for images “cloud 1,” “cloud 2,” “horizon 2,” “horizon 3,” “horizon 4,” and “uniform” in Fig. 5). To assess the relationship between the contrast required for an observer rating of 4, 5, and 6, observer data for each of the images were averaged and plotted. Figure 6 shows the contrast requirements for each of the 10 images as a function of .
For purposes of this discussion, we will term observer ratings of 4, 5, and 6 as minimum contrast, average contrast, and good contrast, respectively. Linear curves of the same slope were fit to each of the ratings in Fig. 5 and are given below:
Determining Luminance Requirements
As is defined in Eq. (2) as a function of , Michaelson contrasts Eqs. (3) to (5) can be calculated over the range of 1 to 6000 fL by simple substitution. Before we can make these calculations, however, a cap must be placed on the values since at low luminance, the envelope curve is characterized by increasingly high values. These values represent contrast variation in the shadows of most natural images and because of this are likely not readily apparent to observers due to a higher light adaptation level of the visual system. Even the darkest background image, “ground clutter 2,” has an average calculated luminance value of 877 fL based on a 5000-fL peak luminance. At low luminance values, the envelope curve would require contrasts higher than 1.0, which by definition are not achievable. To overcome this difficulty, we capped maximum contrast at 0.8 and this was based on observer ratings. Evaluating Fig. 5 as a basis and noting the good contrast data points, all average contrast values were below 0.8. Using 0.8 as a contrast limit places an upper limit on to about 1.28, 1.07, and 0.76 for minimum, average, and good contrast, respectively. Using this contrast limit, Fig. 7 shows the contrast requirements for the three contrast conditions as a function of .
To calculate the luminance requirement for symbology, ( or ) based on the data, shown in Fig. 7, requires assumptions about the luminance distribution within . Within a background scene, if we assume that pixel luminance values are normally distributed, then the middle luminance () can be calculated as follows:
Using the 0.8 Michaelson contrast cap and setting approximately equal to , then the luminance of symbology can be calculated using the following formula:Fig. 8 shows the luminance requirements for the three contrast conditions.
The three contrast curves merge to a straight line at low luminance values and this is due to the limit placed on , as seen in Fig. 7. We termed this straight line the 0.8 Michaelson contrast line. Each contrast curve peaks at an intermediate luminance level due to the fall-off in with increasing luminance. Of note, the peak luminance values did not change even when the Michaelson contrast was capped at a higher 0.9; thus, we feel confident that this method yields luminance requirements that are trustworthy.
In the original study,1 contrast ratios were calculated from the block symbology images and Eq. (1) provided a reasonable fit to that data for an observer rating of 4.0. Using that equation in association with Eq. (2) yields a peak of 2278 fL. This luminance estimate is about 20% lower than the 2840 fL found using the Michaelson contrast curve fit. The method in which the 1.42 scalar was derived is likely not as accurate as it represents an average of exponential curves fit to the contrast ratios for each background image.
The peak luminance values in Fig. 8 were based on linear curves fit to the data in Fig. 6. When calculating maximum luminance requirements for symbology, it would be better to use curves fit to the maximum contrast data in Fig. 6 rather than using curves fit to the average values. This can be achieved easily by simply scaling the three linear curves upward until all data points are either on or below the straight line fit. Scaling is performed by simply increasing the value of the constant in Eqs. (3) to (5). Table 2 shows the results of this process along with the peak luminance requirements for each contrast condition. For the good contrast condition, peaks at 5470 fL, and this value will form the basis for our discussion about color.
Peak calculated luminance values based on linear curve fits to the data [Eqs. (3) to (5)] as applied to the envelope curve [Eq. (2)]. The peak luminances for the average condition can be observed in the three contrast curve plots in Fig. 8.
|Contrast||Fit to data||Contrast equation Constant||Peak luminance (fL)|
In summary, the 5470-fL peak luminance value from Table 2 was calculated for white symbology against eight natural backgrounds, one artificial background, and one uniform field. This does not take into account transmission characteristics of military hardware even though military hardware was modeled during the collection of the original data. The images produced by the model were evaluated by observers, and calculations were only made on these standalone images with contrast-correct representations of symbology overlaid over each background. Even though a full-color HMD was modeled, only white symbology was evaluated as it represented the highest achievable luminance contrast.
When developing see-through HMD luminance and color contrast requirements, consideration of natural terrain features becomes an obvious concern as background luminances and color add to the luminance and color of the symbology. In choosing symbology colors, each must have sufficient contrast against any background color to be easily distinguishable and recognizable. When considering sufficient contrast of color symbology against natural backgrounds, it may be beneficial to use the luminance requirements for white symbology as a guide. This would allow us to calculate the HMD luminances for various colors that might be used in a see-through HMD. For example, Fig. 9 shows the chromaticity coordinates for a full-color liquid crystal display (LCD) using the 1976 Comission Internationale de l’Eclairage (CIE) chromaticity diagram. For this particular display, the red and green chromaticity coordinates are near the monochromatic border as the spectra have fairly narrow bands at 612 and 544 nm, respectively. This LCD provides a good example of the relative luminances expected in a full-color HMD. Table 3 shows the primary values for each of the colors plotted in Figs. 9 and 10. The relative luminances of the three primaries (red, green, and blue) were chosen by industry such that their summation (with each primary set to the same value) would yield the achromatic grayscale or shades of gray from black (0,0,0) to white (255,255,255). The white value is represented by the middle data point in Fig. 9.
Primary values for each of the colors plotted in Figs. 9 and 10.
Table 4 shows calculated luminances based on the relative luminances of the LCD example. By setting each color equal to the 5470-fL peak luminance requirement derived above, luminances for the other six colors could be calculated. Clearly, some of the values in the table seem exceedingly high. Especially high are those values for blue symbology, where the HMD would have to produce a total luminance output at the eye of 80,196 fL. Of course, these luminance requirements will be reduced when combined with the light transmission factors for aircraft windscreens, visors, and HMD combiners or other lenses (see Sec. 5 for a typical example). Besides the enormous light intensity requirement for blue symbology, blue is a poor choice for symbology. The blue display spectra mostly stimulate the short-wavelength cone whose population in the retina is sparse in comparison to the medium- and long-wavelength cones, and are thus not as suitable for distinguishing small characters.
Calculated required see-through HMD display luminances (fL) for the seven colors shown in Fig. 10. Values were calculated relative to each individual color set equal to the peak 5470 fL luminance value derived above.
|Percentage of white||32.29%||6.82%||74.30%||67.48%||92.95%||25.47%||100.00%|
See-through HMD imagery is additive in that each displayed pixel is a function of the display as well as the ambient scene. As the ambient scene gets brighter or display luminance is reduced, the color of the displayed pixel increasingly takes on the color of the background. Because of this, it is difficult to discount the importance of luminance contrast above all other considerations. In other words, if we define color contrast as a separation between coordinates on a chromaticity diagram, then as luminance contrast is reduced, the color contrast is also reduced as the distance between chromaticity coordinates shrinks.6 This concept is depicted in Fig. 11.
Acquisition specifications for see-through HMDs should take into account the hardware that will be used with the HMD. Compatibility issues aside, windscreens, visors, and the HMD combiner lens will block some of the ambient light reaching the eye, and this light reduction should reduce the overall high luminance requirement.8 For example, helicopter windscreens and tinted visors generally have a light transmission curve that is generally flat across the visible spectrum. If a tinted visor is likely to be worn during daylight hours, then the transmissivity of the windscreens and visors should be taken into account. In addition, HMD combiner lens coatings provide some attenuation of ambient light transmission. For flat transmission spectra, assuming a 75% transmission of an aircraft windscreen, a 15% transmission of a tinted visor, and a 40% transmission of the HMD would yield a total light transmission of 4.5%. Applying this transmission to the 5470 fL value calculated above results in a luminance requirement of 246 fL for symbology. If red symbology was needed in a full-color system, the total output of the HMD would equal 966 fL, calculated from Table 4 above.
HMD transmission spectra are normally notched at one or more HMD emission spectra peaks (e.g., at 543 nm for the P-43 phosphor used in the U.S. Army’s AH-64 Apache HMD). This is done to improve contrast of the emissions against the ambient scene. Certainly, notch filters reduce the overall transmissivity of the combiner lens, which will improve contrast. However, under certain circumstances, it has little effect. For instance, the example shown in Fig. 11 would not be improved by a typical narrow-band notch filter as the emission spectra is green and the background is composed of red and blue light.
The contrast requirements provided here were based on observer ratings of the quality of symbology in static imagery.1 The imagery consisting of eight natural scenes, a uniform field, and a background image composed of artificial clutter was superimposed with symbology over a rather large range of contrast conditions. Ratings of the quality of the symbology were used to generate equations for minimum, average, and good contrast as a function of luminance and background clutter. These equations were then applied to an envelope curve that describes possible background clutter as a function of daylight luminance.5,7 The results of this application produced daylight luminance requirements for minimum, average, and good contrast. Using the good contrast value of 5470 fL produced the results in Table 4, which describes the luminance requirements for symbology of different colors (symbology consisting of the three primary colors and their combinations).
The results in Table 4 were based on the emission spectra of a typical LCD. As manufacturers will normally adjust the contributions of the three primaries, at their maxima, to produce white light of a given color temperature, similarly scaled contributions from the three primaries will likely be used regardless of the light source.
The question that likely arises from these results is how robust is the luminance requirement determined in this paper. The imagery that observers evaluated was well-controlled, with a calibrated monitor and luminances calculated based on photometric assessments. The imagery was static and dynamic imagery may somewhat reduce the luminance requirement but will not likely increase it. What may increase the luminance requirement is displaying imagery other than symbology (e.g., sensor imagery, synthetic imagery, tactical maps). Aviators are very familiar with the symbology set used in their aircraft. They know the function and position of each element, and this knowledge assists them in reading the symbology when contrast conditions may be poor. On the other hand, other imagery has an unknown quality about it and to correctly decipher information content requires good contrast conditions. In a preliminary investigation, Harding et al.9 found that, indeed, map imagery required greater contrast to interpret, especially against a cloud image (high luminance, low clutter), where map imagery needed about twice as much luminance contrast compared to symbology. Against high-cluttered backgrounds, symbology and maps equally required high contrast. Of course, this is a rather coarse estimate based on preliminary findings. We do, however, feel fairly confident that for HMD symbology, the data provided here should assist in establishing general guidelines for HMD luminance performance requirements.
Envelope Curve and Luminance Distributions within Natural Scenes
Equation (2) represents peak as a function of luminances in natural scenes. We used this curve to derive the daylight luminance requirements for see-through displays. The equation provided a limit or envelope curve where 99% of the data plotted in Fig. 4 fell on or below the curve. The data from the artificial clutter image were largely responsible for setting the boundaries of Eq. (2). Based on a limited sample size of nine images, should we trust the curve to clearly reveal the limits of over the expected daylight luminance range? To answer the question of background clutter, we chose two natural images that were composed of extremely high contrasts and high spatial frequencies. The images are shown in Fig. 12. The complexity of the image, , is plotted as a function of scaled luminance. As stated earlier, Eq. (2) provides a reasonable envelope containing over 99% of the complexity values from the two images.
Equation (2) also assumes that 6000 fL is a reasonable peak luminance to use when determining operational light output requirements of HMDs. This value is an upper range compromise of the many background luminances encountered across fixed- and rotary-wing military flight environments. HMD imagery must be viewable against such varied backgrounds as grass, dirt, sand, water, snow, clouds, and sky.10 While values in excess of 30,000 fL can be encountered (i.e., white cumulus clouds), in the high-altitude, fixed-wing environment, a 10,000 fL maximum luminance value representing a clear sky is frequently used as a representative background luminance in HMD calculations;11 for the rotary-wing environment, 6000 fL (representative of a bright overcast sky) is typically used.
Limitations of Using Static Imagery
The background images used to evaluate the quality of overlaid symbology (Fig. 1) were static, i.e., no relative motion between the symbology and the selected background. However, in real-world HMDs, symbology is viewed against pilotage imagery, either synthetic or real-world (i.e., provided by a sensor mounted on the aircraft or pilot’s head). In such scenarios, the symbology is viewed against a moving background, resulting in varying degrees of relative motion between the symbology and the background scene. In the simplest of these scenarios, the symbology may be viewed against a uniform sky or water background, which, having no spatial content, produces no perceived relative motion, and can be considered equivalent to the static case. However, in most flight scenarios, the background is a complex spatial scene and relative motion is present between the symbology and background.
HMD symbology may result in greater relative symbology motion than HUD symbology. Because the symbology block the real-world background scene, they are perceived as being nearer than the background, even though they are collimated. The motion of the symbology is perceived as faster even though it moves at the same rate as the real world.12
It has long been known that relative symbology/background motion in HUDs can degrade retinal image contrast of either the symbology or the background scene (depending on viewer fixation), although performance effects are complex, depending on viewer task, symbology characteristics (e.g., luminance, shape, and size) and location, and background characteristics (e.g., clutter, luminance, and contrast).13,14
Limitations of Using White Symbology to Determine Daylight Luminance Requirements
Although luminance and contrast requirements for the legibility of white symbology have been and are a continuing topic of investigation, such requirements for color symbology in HMDs have not been pursued as vigorously.1516.–17 Metrics defining color contrast (and hence luminance) are more complicated than those presented previously where the contrast refers only to differences in luminance. Color contrast metrics must include differences in chromaticity as well as luminance. And, it is not as straight forward to transform chromatic differences into just noticeable differences (JNDs) in a perceived color space. This is due to a number of reasons. First, color is perceptually a multidimensional variable. The chromatic aspect, or hue, is qualitative and two-dimensional, consisting of a blue–yellow axis and a red–green axis. Second, the dimensions of saturation and brightness, as well as other factors, such as the size and shape of a stimulus, affect the perceived color and color differences.18 As an example, an orange band against a grey background will appear “brighter” than an identical blue or green band at the same saturation value; even though a grayscale conversion may demonstrate equal luminances.19
Displaying Symbology versus Imagery in a See-Through Helmet Mounted Display
Defining the luminance requirements for a color display using Table 4, for example, presupposes that the display was developed to yield a white balance when the red, green, and blue primaries are all set to their maximum level (e.g., 255 in an 8 bit-per-color display). Displays that are white balanced reproduce imagery that best mimics real-world colors. If the function of the HMD is to only produce symbology, proper white balancing may not be necessary. In laser-based scanning HMDs, for example, the short wavelength laser could be driven by a much higher voltage to offset the low sensitivity of the visual system to lower wavelength light. In this instance, the red and green lasers need not require a higher driver voltage thus, the high luminances shown in Table 4 for white symbology could be significantly reduced. However, using commercial display systems (e.g., LCDs, OLEDs) as sources in an HMD, you will most likely encounter displays that are purposely set to achieve proper white balancing.
This research was supported in part by an appointment to the Postgraduate Research Participation Program and the Knowledge Preservation Program at the U.S. Army Aeromedical Research Laboratory administered by the Oak Ridge Institute for Science and Education through an interagency agreement between the U.S. Department of Energy and the U.S. Army Medical Research and Materiel Command.
Thomas H. Harding received his PhD in 1977 from Purdue University in visual neurosciences. Since 1979, his career has been exclusively at the U.S. Army Aeromedical Research Laboratory, where he has held several key research and leadership positions. In terms of his association with SPIE, his research has mainly focused on helmet-mounted displays and other advanced optical systems with particular emphasis on design considerations, image quality metrics, operational performance, and visual perception.
Clarence E. Rash is retired from the U.S. Army Areomedical Research Laboratory. He holds BS/MS degrees in physics from Old Dominion University. He has published 300+ papers in the field of displays. He served for 25 years as HMD Session, Conference, and Conference Track Chair for the SPIE Display Symposium. He is the editor of Helmet-Mounted Displays: Design Issues for Rotary-Wing Aircraft and Helmet-Mounted Displays: Sensation, Perception, and Cognition Issues.