1.

Introduction

For symbology to be viewed in a see-through helmet-mounted display (HMD) or head-up display (HUD), the luminance of the symbology must be sufficient to distinguish it from the see-through background. This is true whether or not symbology is displayed on a monochromatic or full-color display. When the contrast of the transparent symbology is sufficiently high, the symbology appears as an overlay on the ambient scene. In order for an HMD or HUD to be usable in an operational environment, the luminance requirements must take into consideration the type of displayed imagery (e.g., symbology, situational maps, target sights), the tasks (e.g., targeting, navigation, obstacle avoidance), the operational setting (e.g., day/night, terrain features), additional hardware (e.g., visors, windscreens, laser protection), and other considerations. For a color HMD, symbology color overlaid on an ambient scene should consider luminance and color contrast, which requires information about the spectral content of the landscape or ambient scene. However, optical designers must consider all terrain features when designing a see-through optic as the military can be deployed to any geographical location around the globe; this fact may simplify the development of luminance requirements for see-through, full-color HMDs. The reason that this will simplify the development of a contrast requirement is that luminance contrast, and not color contrast, is the determining factor.

2.

Modeling Helmet-Mounted Display Symbology

As an initial step in defining the luminance requirements for daylight symbology, Harding et al.¹ processed simulated images of white symbology (simulating a full-color HMD with all three primary colors displayed) overlaid on selected static natural backgrounds (Fig. 1). Observers evaluated the quality of symbology (Table 1) in “contrast correct” images of symbology overlaid over eight natural backgrounds (an example is shown in Fig. 2), one artificial background (artificial clutter in Fig. 1), and one uniform field. The overlaid symbology shown in Fig. 2 was created by use of a software model¹ that applied appropriate luminance scaling to yield a symbology image that has been added to the background image. In this experiment, the complexity of the background image was of paramount importance when determining the minimum luminance requirements for see-through symbology. This finding is in general agreement with other studies^2–4 of transparent text against complex backgrounds. The luminance of the background image was of less importance.

Fig. 1

Nine of the ten images used to evaluate the quality of overlaid symbology. The tenth image was a uniform field. Image from Harding et al.¹

Table 1

Rating scale and description of ratings given to subjects in Harding et al.1

Fig. 2

Horizon 1 background image with (a) overlaid symbology and (b) block symbology image of same contrast. The average observer rating of the symbology in this image was 5.0, all characters could be seen with reduced contrast. With a display gamma of 2.2, the average Michaelson contrast was 0.41.

For each background scene, 20 symbology images of varying contrast were evaluated by observers. Following a training session, observers judged the quality of symbology for each of the 200 images in three test sessions. All observers were required to pass an intermediate-field acuity test (distance of 22 in.) and a Farnsworth 15-Hue color vision test. There were no gender or age-specific requirements. Twenty volunteer observers were selected for the study, however, for an observer’s data to be included in the final data analyses, critical judgment criteria were established and only data from seven of the observers were selected for final data inclusion (see original paper¹ for selection metrics). For each of the 200 images, an average Michaelson contrast $[(L_{\max }-L_{\min })/(L_{\max }+L_{\min })]$ was calculated for each image. To make calculations easier, block symbology images were processed in exactly the same fashion as the symbology image. For example, for each of the 20 blocks shown in Fig. 2(b), $L_{\max }$ equaled the average luminance for each of the 20 symbology blocks. Likewise, $L_{\min }$ was the average calculated luminance of 20 areas, of the same size adjacent to them.

The size of each of the background images was $640\times 480\text{pixels}$. The patch size was $50\times 50\text{pixels}$. Luminance values were calculated for each subpixel using the following equations:

Fig. 3

(a) Gamma of Sony Trinitron monitor used in original study1. (b) Calculated visual contrast sensitivity as a function of four luminance levels, using the Barten model.^5,6

The minimum contrast for symbology was judged to be equal to an observer rating of 4 on the seven-point scale, as described in Table 1. Harding et al.¹ found that the following equation provided a reasonable fit for describing the luminance required for a minimum contrast rating of 4.0:

When Michaelson contrast was plotted as a function of observer ratings, the curves for different background images were quite disparate (Fig. 5). When background complexity was added as a codeterminant, the plots became much tighter [see Fig. 7(b) in Ref. 1].

2.1.

Background Clutter in Natural Scenes

Although the natural background images were deemed representative, the imagery was static and not dynamic. During actual flight, flight symbology would appear overlaid over a changing ambient scene. Motion tends to blur or reduce the local variations in an ambient scene (especially in nap of the earth flight), and the background relative to the symbology has a temporal modulation component. At minimal contrast, symbology characters would likely appear recognizable in one instance and perhaps partially disappear in the next instance as the complexity and luminance flow of the ambient image waxed and waned. There is certainly a dynamic feature to the visibility and quality of symbology, and taking motion into account would likely yield more accurate results than simply evaluating static symbology. Analyzing the effect of motion on symbology visibility and quality, however, is beyond the scope of this paper. Since the quality of symbology is based on the visibility of each character or indicator against its background, we expressed the ${\mathrm{B}}_{\mathrm{SD}}$ in Eq. (1) as the average percent standard deviation of small patches of background. In this way, the background complexity, as measured by the standard deviation, would always be relevant to the overlaid symbology. We coined the term “local contrast” to describe the contrast between the luminance within a character or indicator divided by the average luminance of adjacent patches that subtended an area equal to about the size of a symbology character.^5,7

By setting the maximum pixel luminance in each of the nine images shown in Fig. 1 to 5000 fL, a patch analysis distribution of luminance versus complexity produced scatter grams whose upper envelope could be well described by the following equation:

Eq. (2)

$$\text{Complexity or}{\mathrm{B}}_{\mathrm{SD}}=100[-1.0\ln (L_{\mathrm{B}})+8.7]\text{over the range}1\text{to}6000\mathrm{fL},$$

where $\ln (L_{\mathrm{B}})$ is the natural logarithm of the background luminance in fL and complexity is in percent standard deviation. Below about 2200 fL, the limiting equation yields percent standard deviation measures that are greater than 100%. Above 6000 fL, the curve produces a negative complexity value. However, over a large photopic range of 1 to 6,000 fL, the approximation is reasonable. Of the data plotted in Fig. 4, over 99.9% of the patch complexity data falls on or below the curve from Eq. (2).

Fig. 4

Scatter plot of patch analysis of each of the photos shown in Fig. 1. The solid curve is from Eq. (2). Image adapted from Harding et al.⁷

2.2.

Observer Ratings of Symbology

In the original study,¹ observers evaluated the quality of symbology in 200 images, 20 images per background image. Average contrasts were calculated for each of the 200 images, and the contrasts varied widely depending on the background images. The average Michaelson contrast was calculated for each of the images and these contrasts were plotted as a function of the average observer ratings. Figure 5 shows the results for all 10 of the background images. Least square power functions were fit to each image set ($R^2>95\%$ based on curves fit to data points at or below an observer rating of 6.5). For each background image, observer ratings of 4, 5, and 6 were calculated from the curve fits. For six of the images, observer ratings of 4 or lower were not observed, and extending the curve fits allowed calculations of the minimum contrast rating (i.e., see data for images “cloud 1,” “cloud 2,” “horizon 2,” “horizon 3,” “horizon 4,” and “uniform” in Fig. 5). To assess the relationship between the contrast required for an observer rating of 4, 5, and 6, observer data for each of the images were averaged and plotted. Figure 6 shows the contrast requirements for each of the 10 images as a function of ${\mathrm{B}}_{\mathrm{SD}}$.

Fig. 5

Contrast plotted as a function of observer ratings for the 10 background images.¹ The solid curves are power functions fit to the data.

Fig. 6

Contrast requirements as a function of ${\mathrm{B}}_{\mathrm{SD}}$ for observer ratings of 4, 5, and 6. Each set of data has 10 data points representing the 10 background images.¹ The straight lines are linear fits to each of the observer ratings.

For purposes of this discussion, we will term observer ratings of 4, 5, and 6 as minimum contrast, average contrast, and good contrast, respectively. Linear curves of the same slope were fit to each of the ratings in Fig. 5 and are given below:

Eq. (3)

$$\text{Minimum contrast}=0.06+0.58{\mathrm{B}}_{\mathrm{SD}}, R^2=87.8\%,$$

Eq. (4)

$$\text{Average contrast}=0.18+0.58{\mathrm{B}}_{\mathrm{SD}}, R^2=89.3\%,$$

Eq. (5)

$$\text{Good contrast}=0.36+0.58{\mathrm{B}}_{\mathrm{SD}}, R^2=83.6\%.$$

3.

Determining Luminance Requirements

As ${\mathrm{B}}_{\mathrm{SD}}$ is defined in Eq. (2) as a function of $L_{\mathrm{B}}$, Michaelson contrasts Eqs. (3) to (5) can be calculated over the range of 1 to 6000 fL by simple substitution. Before we can make these calculations, however, a cap must be placed on the ${\mathrm{B}}_{\mathrm{SD}}$ values since at low luminance, the envelope curve is characterized by increasingly high ${\mathrm{B}}_{\mathrm{SD}}$ values. These values represent contrast variation in the shadows of most natural images and because of this are likely not readily apparent to observers due to a higher light adaptation level of the visual system. Even the darkest background image, “ground clutter 2,” has an average calculated luminance value of 877 fL based on a 5000-fL peak luminance. At low luminance values, the envelope curve would require contrasts higher than 1.0, which by definition are not achievable. To overcome this difficulty, we capped maximum contrast at 0.8 and this was based on observer ratings. Evaluating Fig. 5 as a basis and noting the good contrast data points, all average contrast values were below 0.8. Using 0.8 as a contrast limit places an upper limit on ${\mathrm{B}}_{\mathrm{SD}}$ to about 1.28, 1.07, and 0.76 for minimum, average, and good contrast, respectively. Using this contrast limit, Fig. 7 shows the contrast requirements for the three contrast conditions as a function of $L_{\mathrm{B}}$.

Fig. 7

Michaelson contrast calculations as a function of $L_{\mathrm{B}}$ based on the envelope curve [Eq. (2)].

To calculate the luminance requirement for symbology, ($L_{\mathrm{sym}}$ or $L_{\max }-L_{\min }$) based on the data, shown in Fig. 7, requires assumptions about the luminance distribution within $L_{\mathrm{B}}$. Within a background scene, if we assume that pixel luminance values are normally distributed, then the middle luminance ($L_{\mathrm{mid}}$) can be calculated as follows:

Using the 0.8 Michaelson contrast cap and setting $L_{\mathrm{mid}}$ approximately equal to $L_{\mathrm{B}}$, then the luminance of symbology can be calculated using the following formula:

Fig. 8

Symbology luminance required for minimum, average, and good contrast based on the linear curve fits [Eqs. (3)–(5)] to the data of Fig. 4.

The three contrast curves merge to a straight line at low luminance values and this is due to the limit placed on ${\mathrm{B}}_{\mathrm{SD}}$, as seen in Fig. 7. We termed this straight line the 0.8 Michaelson contrast line. Each contrast curve peaks at an intermediate luminance level due to the fall-off in ${\mathrm{B}}_{\mathrm{SD}}$ with increasing luminance. Of note, the peak luminance values did not change even when the Michaelson contrast was capped at a higher 0.9; thus, we feel confident that this method yields luminance requirements that are trustworthy.

In the original study,¹ contrast ratios were calculated from the block symbology images and Eq. (1) provided a reasonable fit to that data for an observer rating of 4.0. Using that equation in association with Eq. (2) yields a peak $L_{\mathrm{sym}}$ of 2278 fL. This luminance estimate is about 20% lower than the 2840 fL found using the Michaelson contrast curve fit. The method in which the 1.42 scalar was derived is likely not as accurate as it represents an average of exponential curves fit to the contrast ratios for each background image.

The peak luminance values in Fig. 8 were based on linear curves fit to the data in Fig. 6. When calculating maximum luminance requirements for symbology, it would be better to use curves fit to the maximum contrast data in Fig. 6 rather than using curves fit to the average values. This can be achieved easily by simply scaling the three linear curves upward until all data points are either on or below the straight line fit. Scaling is performed by simply increasing the value of the constant in Eqs. (3) to (5). Table 2 shows the results of this process along with the peak $L_{\mathrm{sym}}$ luminance requirements for each contrast condition. For the good contrast condition, $L_{\mathrm{sym}}$ peaks at 5470 fL, and this value will form the basis for our discussion about color.

Table 2

Peak calculated luminance values based on linear curve fits to the data [Eqs. (3) to (5)] as applied to the envelope curve [Eq. (2)]. The peak luminances for the average condition can be observed in the three contrast curve plots in Fig. 8.

4.

Color Symbology

In summary, the 5470-fL peak luminance value from Table 2 was calculated for white symbology against eight natural backgrounds, one artificial background, and one uniform field. This does not take into account transmission characteristics of military hardware even though military hardware was modeled during the collection of the original data. The images produced by the model were evaluated by observers, and calculations were only made on these standalone images with contrast-correct representations of symbology overlaid over each background. Even though a full-color HMD was modeled, only white symbology was evaluated as it represented the highest achievable luminance contrast.

When developing see-through HMD luminance and color contrast requirements, consideration of natural terrain features becomes an obvious concern as background luminances and color add to the luminance and color of the symbology. In choosing symbology colors, each must have sufficient contrast against any background color to be easily distinguishable and recognizable. When considering sufficient contrast of color symbology against natural backgrounds, it may be beneficial to use the luminance requirements for white symbology as a guide. This would allow us to calculate the HMD luminances for various colors that might be used in a see-through HMD. For example, Fig. 9 shows the chromaticity coordinates for a full-color liquid crystal display (LCD) using the 1976 Comission Internationale de l’Eclairage (CIE) chromaticity diagram. For this particular display, the red and green chromaticity coordinates are near the monochromatic border as the spectra have fairly narrow bands at 612 and 544 nm, respectively. This LCD provides a good example of the relative luminances expected in a full-color HMD. Table 3 shows the primary values for each of the colors plotted in Figs. 9 and 10. The relative luminances of the three primaries (red, green, and blue) were chosen by industry such that their summation (with each primary set to the same value) would yield the achromatic grayscale or shades of gray from black (0,0,0) to white (255,255,255). The white value is represented by the middle data point in Fig. 9.

Fig. 9

1976 CIE chromaticity diagram of the measured color space of a sample full-color LCD. The triangle connects the three primaries (red, green, and blue). The intermediate points between the primaries are yellow, cyan, and magenta. The point inside the triangle represents the white point with all three primaries set to 255.

Table 3

Primary values for each of the colors plotted in Figs. 9 and 10.

Fig. 10

Measured luminances of the colors plotted in Fig. 8.

Table 4 shows calculated luminances based on the relative luminances of the LCD example. By setting each color equal to the 5470-fL peak luminance requirement derived above, luminances for the other six colors could be calculated. Clearly, some of the values in the table seem exceedingly high. Especially high are those values for blue symbology, where the HMD would have to produce a total luminance output at the eye of 80,196 fL. Of course, these luminance requirements will be reduced when combined with the light transmission factors for aircraft windscreens, visors, and HMD combiners or other lenses (see Sec. 5 for a typical example). Besides the enormous light intensity requirement for blue symbology, blue is a poor choice for symbology. The blue display spectra mostly stimulate the short-wavelength cone whose population in the retina is sparse in comparison to the medium- and long-wavelength cones, and are thus not as suitable for distinguishing small characters.

Table 4

Calculated required see-through HMD display luminances (fL) for the seven colors shown in Fig. 10. Values were calculated relative to each individual color set equal to the peak 5470 fL luminance value derived above.

See-through HMD imagery is additive in that each displayed pixel is a function of the display as well as the ambient scene. As the ambient scene gets brighter or display luminance is reduced, the color of the displayed pixel increasingly takes on the color of the background. Because of this, it is difficult to discount the importance of luminance contrast above all other considerations. In other words, if we define color contrast as a separation between coordinates on a chromaticity diagram, then as luminance contrast is reduced, the color contrast is also reduced as the distance between chromaticity coordinates shrinks.⁶ This concept is depicted in Fig. 11.

Fig. 11

(a) Progressive color chart of transparent green targets against a magenta background. (b) The chromaticity distance on the 1976 CIE diagram between target and background is plotted as a function of luminance contrast. As target luminance is reduced, so is color contrast for transparent targets. (c) 1976 CIE chromaticity coordinates of the transparent green targets shown in (a).

5.

Hardware Considerations

Acquisition specifications for see-through HMDs should take into account the hardware that will be used with the HMD. Compatibility issues aside, windscreens, visors, and the HMD combiner lens will block some of the ambient light reaching the eye, and this light reduction should reduce the overall high luminance requirement.⁸ For example, helicopter windscreens and tinted visors generally have a light transmission curve that is generally flat across the visible spectrum. If a tinted visor is likely to be worn during daylight hours, then the transmissivity of the windscreens and visors should be taken into account. In addition, HMD combiner lens coatings provide some attenuation of ambient light transmission. For flat transmission spectra, assuming a 75% transmission of an aircraft windscreen, a 15% transmission of a tinted visor, and a 40% transmission of the HMD would yield a total light transmission of 4.5%. Applying this transmission to the 5470 fL value calculated above results in a luminance requirement of 246 fL for symbology. If red symbology was needed in a full-color system, the total output of the HMD would equal 966 fL, calculated from Table 4 above.

HMD transmission spectra are normally notched at one or more HMD emission spectra peaks (e.g., at 543 nm for the P-43 phosphor used in the U.S. Army’s AH-64 Apache HMD). This is done to improve contrast of the emissions against the ambient scene. Certainly, notch filters reduce the overall transmissivity of the combiner lens, which will improve contrast. However, under certain circumstances, it has little effect. For instance, the example shown in Fig. 11 would not be improved by a typical narrow-band notch filter as the emission spectra is green and the background is composed of red and blue light.

6.

Discussion

The contrast requirements provided here were based on observer ratings of the quality of symbology in static imagery.¹ The imagery consisting of eight natural scenes, a uniform field, and a background image composed of artificial clutter was superimposed with symbology over a rather large range of contrast conditions. Ratings of the quality of the symbology were used to generate equations for minimum, average, and good contrast as a function of luminance and background clutter. These equations were then applied to an envelope curve that describes possible background clutter as a function of daylight luminance.^5,7 The results of this application produced daylight luminance requirements for minimum, average, and good contrast. Using the good contrast value of 5470 fL produced the results in Table 4, which describes the luminance requirements for symbology of different colors (symbology consisting of the three primary colors and their combinations).

The results in Table 4 were based on the emission spectra of a typical LCD. As manufacturers will normally adjust the contributions of the three primaries, at their maxima, to produce white light of a given color temperature, similarly scaled contributions from the three primaries will likely be used regardless of the light source.

The question that likely arises from these results is how robust is the luminance requirement determined in this paper. The imagery that observers evaluated was well-controlled, with a calibrated monitor and luminances calculated based on photometric assessments. The imagery was static and dynamic imagery may somewhat reduce the luminance requirement but will not likely increase it. What may increase the luminance requirement is displaying imagery other than symbology (e.g., sensor imagery, synthetic imagery, tactical maps). Aviators are very familiar with the symbology set used in their aircraft. They know the function and position of each element, and this knowledge assists them in reading the symbology when contrast conditions may be poor. On the other hand, other imagery has an unknown quality about it and to correctly decipher information content requires good contrast conditions. In a preliminary investigation, Harding et al.⁹ found that, indeed, map imagery required greater contrast to interpret, especially against a cloud image (high luminance, low clutter), where map imagery needed about twice as much luminance contrast compared to symbology. Against high-cluttered backgrounds, symbology and maps equally required high contrast. Of course, this is a rather coarse estimate based on preliminary findings. We do, however, feel fairly confident that for HMD symbology, the data provided here should assist in establishing general guidelines for HMD luminance performance requirements.

6.1.

${\mathrm{B}}_{\mathrm{SD}}$ Envelope Curve and Luminance Distributions within Natural Scenes

Equation (2) represents peak ${\mathrm{B}}_{\mathrm{SD}}$ as a function of luminances in natural scenes. We used this curve to derive the daylight luminance requirements for see-through displays. The equation provided a limit or envelope curve where 99% of the data plotted in Fig. 4 fell on or below the curve. The data from the artificial clutter image were largely responsible for setting the boundaries of Eq. (2). Based on a limited sample size of nine images, should we trust the curve to clearly reveal the limits of ${\mathrm{B}}_{\mathrm{SD}}$ over the expected daylight luminance range? To answer the question of background clutter, we chose two natural images that were composed of extremely high contrasts and high spatial frequencies. The images are shown in Fig. 12. The complexity of the image, ${\mathrm{B}}_{\mathrm{SD}}$, is plotted as a function of scaled luminance. As stated earlier, Eq. (2) provides a reasonable envelope containing over 99% of the complexity values from the two images.

Fig. 12

(a and b) Trees silhouetted with sunlight peering through composed of high contrast and high spatial frequencies. The highest luminance pixel in each image was set to 5000 fL and the image scaled appropriately. (c and d) The image was divided into 10 by 10 patches and the average luminance and standard determined of each patch is plotted in the scatter grams. The red line in each scatter gram is a plot of the envelope function [Eq. (2)].

Equation (2) also assumes that 6000 fL is a reasonable peak luminance to use when determining operational light output requirements of HMDs. This value is an upper range compromise of the many background luminances encountered across fixed- and rotary-wing military flight environments. HMD imagery must be viewable against such varied backgrounds as grass, dirt, sand, water, snow, clouds, and sky.¹⁰ While values in excess of 30,000 fL can be encountered (i.e., white cumulus clouds), in the high-altitude, fixed-wing environment, a 10,000 fL maximum luminance value representing a clear sky is frequently used as a representative background luminance in HMD calculations;¹¹ for the rotary-wing environment, 6000 fL (representative of a bright overcast sky) is typically used.

7.

Disclaimer

The views, opinions, and/or findings contained in this paper are those of the authors and should not be construed as an official Department of the Army position, policy, or decision unless so designated by other official documentation.