A linear systems model has been developed for determining resolution capabilities provided by any shadow mask (SM) CRT-observer configuration. A square wave model is used to simulate both the sampling of the SM and the spatial phase difference between the displayed image and the SM holes. Due to the spatial phase differences for a selected SM CRT-observer configuration, distributions of calculated modulation depths are obtained. This analysis supports the notion that to meaningfully specify and measure the resolution of SM CRTs, characteristic of the luminance profiles prior to SM sampling must be obtained. To this end efficient measurement techniques were developed that provide accurate modulation depth values of the unsampled luminance profiles from the sampled luminance profiles obtained at the CRT surface.

The contrast of Liquid Crystal Displays (LCD) was measured under both point source and diffuse illumination. It appears that diffuse lighting is more appropriate to the measurement of visual performance of reflective or transflective displays. The design of integrating spheres for the simulation of diffuse illumination is discussed. Concerning the representation of visual quantities as a function of viewing direction, the stereographic projection is more convenient than the standard pseudo polar plot.

Increasing the refresh frequency of a video display terminal above the line frequency can create character jitter due to interference between the line frequency and refresh frequency. We measured threshold for character jitter on high luminance, positive polarity displays for a range of differences between line and refresh frequency (difference frequencies or DFs). The results indicate an average threshold of about 17 arc sec for a range of DFs. The function relating threshold amplitude to DF is independent of line frequency and closely resembles the results of displacement threshold studies. This suggests that the detection of character jitter is very similar to the hyperacuity task of detecting the minimal displacement of a single line. This implies that jitter threshold will be effected in the same fashion by such variables as blur, luminance and presence or absence of reference lines.

Spatial adaptation, in the form of a frequency-specific reduction in contrast sensitivity, can occur when the visual system is exposed to certain stimuli. We employed vertical sinusoidal test gratings to investigate adaptation to the horizontal structure of text presented on a standard video display terminal. The parameters of the contrast sensitivity test were selected, on the basis of waveform analysis of spatial luminance scans of the text stimulus. We found. that subjects exhibited a small, but significant, frequency-specific adaptation consistent with the spatial frequency spectrum of the stimulus. Theoretical and practical significance of this finding are discussed.

An algorithm is described in part I that determines the focus quality of an image by measuring the maximum brightness gradient present in the image. Measurement on actual television pictures are correlated with subjective focus evaluation. The excellent correlation between measurement and subjective evaluation indicates the usefulness of this simple algorithm. Part II reports on the subjective evaluation of 20 actual television pictures using a five point grading scale. The main purpose was to determine if subjects could successfully evaluate focus quality on the grading scale and if so, what parameters are used by subjects in the evaluation. This can prove the validity of the measurements. Multidimensional scaling was used to determine dimensions used in the evaluation. Results show that only one dimension is used by subjects and that they are able to evaluate pictures very accurately on a five point grading scale. The results obtained here serve to prove that the objective evaluation is valid.

Transformation of computer graphics images to the RS170A television format by a new digital video standards conversion technique is described. Nominal resolution of incoming images is 1280x1024 pixels. Output image resolution is 762x483 pixels. To avoid aliasing effects, images are low-pass filtered prior to resampling in the horizontal and vertical directions. A range of '1000-line' images with different dimensions and timing parameters may be accommodated by adjustment of stored coefficients through the interactive user interface.

A psychological hierarchy model of human vision(1)(2) suggests that the visual signals are processed in a serial manner from lower to higher stages: that is "sensation" - "perception" - "emotion." For designing a future television system, it is important to find out what kinds of physical factors affect the "emotion" experienced by an observer in front of the display. This paper describes the psychological effects induced by the sharpness of the picture. The subjective picture quality was evaluated for the same pictures with five different levels of sharpness. The experiment was performed on two kinds of printed pictures: (A) a woman's face, and (B) a town corner. From these experiments, it was found that the amount of high-frequency peaking (physical value of the sharpness) which psychologically gives the best picture quality, differs between pictures (A) and (B). That is, the optimum picture sharpness differs depending on the picture content. From these results, we have concluded that the psychophysical sharpness of the picture is not only determined at the stage of "perception" (e.g., resolution or signal to noise ratio, which everyone can judge immediately), but also at the stage of "emotion" (e.g., sensation of reality or beauty).

The paper discusses the complexity of color vision in humans, considering the main aspects involved: the physical aspect, the psychophysical aspect, the physiological aspect and the psychological aspect. The meanings of the term color associated to each such aspect (asfor example, color stimulus, color valence, neural color signal and color percept) are introduced. Some types of color defective vision, relevant for color display users, are indicated. The methods to generate color stimuli in modern display devices, employing different technologies, are compared.

Brightness contrast is an important factor in perceptual image quality. In the case of a TV system it is dominantly determined by the luminance reproduction function. Subjective quality was found to be optimal for non-linear luminance transfer and closely related to (subjective) brightness contrast. Quality is also related to sharpness, which in turn depends on the non-linear transfer. Fortunately, in the neighbourhood of the optimum, quality is dominantly determined by brightness contrast. It will be shown that brightness contrast is not adequately characterized by the luminance contrast ratio, which is defined as the ratio of the luminance extremes. Alternative and more suitable contrast measures in terms of luminance are discussed.

After a short survey of some other measures of perceived image quality, the recently proposed square root integral (SQRI) is described. Special attention is paid to the way in which this metric takes the effect of various display parameters into account. Experimental data on subjective image quality at varying resolution, addressability, luminance and display size are compared with predictions by the square root integral. From the comparison it appears that there is a linear relation between subjective image quality and SQRI value.

The contrast sensitivity function (CSF) is widely used to specify the sensitivity of the visual system. Each point of the CSF specifies the amount of contrast needed to detect a sinusoidal grating of a given spatial frequency. This paper describes a set of five mathematically related visual patterns, called "multipoles," that should replace the CSF for measuring visual performance. The five patterns (ramp, edge, line, dipole and quadrupole) are localized in space rather than being spread out as sinusoidal gratings. The multipole sensitivity of the visual system provides an alternative characterization that complements the CSF in addition to offering several advantages. This paper provides an overview of the properties and uses of the multipole stimuli. This paper is largely a summary of several unpublished manuscripts with excerpts from them. Derivations and full references are omitted here. Please write me if you would like the full manuscripts.

For the purpose of display design, it is important to know the Characteristics of the human observers viewing the displays. As one such characteristic, the temporal response of human vision was measured under unadapted conditions from absolute threshold to the highest feasible intensities, under controlled conditions which minimized flicker adaptation. Flicker was visible to more than 100 Hz at high intensities and conformed to the Ferry-Porter law over a 5-6 decade range. Flicker sensitivity increased about twice as fast in the periphery as the central foveola. The simple relation of the Ferry-Porter law can thus be used to characterize the human temporal response for almost any display requirements.

Although there are many papers on the threshold detection of flicker, suprathreshold flicker effects have been relatively unstudied. In this paper, we present a psychophysical matching technique for measuring the magnitude of above-threshold flicker. We have used this technique to measure perceived flicker on CRT displays. Over the range of temporal modulations and luminances studied, we find that suprathreshold perceived flicker increases linearly with log pixel luminance.

A "quick look" visual model, a kind of standard observer in software, is being developed to estimate the appearance of new display designs before prototypes are built. It operates on images also stored in software. It is assumed that the majority of display design flaws and technology artefacts can be identified in representations of early visual processing, and insight obtained into very local to global (supra-threshold) brightness distributions. Cognitive aspects are not considered because it seems that poor acceptance of technology and design is only weakly coupled to image content.

In this paper, a new visual model of brightness perception is introduced to resolve apparent incompatibilities among Fechner's logarithmic relation, Stevens' power law response, and psychophysical and electrophysiological Km•I/(I+S) experimental results. The proposed visual model, which is based upon combining and extending two existing ones, is shown to predict these results successfully. Thus, a unification of three apparently conflicting visual laws emerges and the versatility and validity of the new visual model is demonstrated.

Digital processing of black and white images has received most attention during the last 25 years, and has led to various algorithms for the enhancement, smoothing, and zooming of images. Due to the decreasing cost and increasing availability of color image acquisition and display equipment along with the computing systems of higher performance, it is expected that the digital processing of color images will rapidly grow in demand. In this paper we review our experiences with the processing of color images. In particular, algorithms for the contrast enhancement, smoothing and zooming of color images are investigated. The effects of applying these methods in color coordinates other than RGB are examined and considerations with respect to performance, time efficiency and compatibility with some popular algorithms developed for monochrome images are discussed.

Coherent medical ultrasound images include high contrast speckle artifact. Low contrast, slowly varying image components, such as subtle tissue reflectivity changes due to a diffuse disease state, are difficult to perceive under such conditions. Physicians do not trust smoothed images followed by contrast stretching because of loss of other detail which is considered important. However, the color perception of human observers is ideally suited to the lower spatial frequencies. This paper discusses a technique in which color tinting is used to enhance the low spatial frequency information content of a medical ultrasound image. The resulting images present detail as reduced contrast luminance and slowly varying local average values as hue or chroma.

Since viewing conditions greatly affect the perception of an image, the coding algorithm must take into account the viewing conditions and the human visual perception mechanisms. If the coded image and the uncoded input image are indistinguishable in a casual viewing session lasting 20 seconds, then the coding algorithm is said to have attained Transparent quality. The smallest bit rate at which such transparent quality is achieved is called the perceptual entropy. Starting with a low bit rate vector quantizer which does not offer transparent quality encoding, we explore a technique with which perceptually relevant regions can be carefully encoded to achieve transparent and high quality reproductions. The bit rates associated with this technique are in the range of 1 to 3.5 bits/pixel and these can be regarded as an upper bound to the perceptual entropy.

Many image compression techniques involve segmentation of a gray level image. With such techniques, information is extracted that describes the regions in the segmented image, and this information is then used to form a coded version of the image. In this paper we present a region-growing-based segmentation technique that incorporates human visual system properties, and describe the use of this technique in image compression. We also discuss the effect of requantizing a segmented image. Requantization of a segmented image is useful because it can lead to a reduction in the number of bits required to code the description of the regions in the segmented image. This results in a lower data rate. We show that the number of gray levels in a segmented image can be reduced by a factor of at least twelve, without noticeable degradation in the quality of the segmented image. This result is attributable to human visual system properties having to do with contrast sensitivity, and to the fact that requantization of a segmented image does not usually reduce significantly the number of distinct segments in the image. In addition, in this paper we explore the relationship between the number of segments in an image, and the extent of requantization possible before noticeable degradation occurs in the image. Finally, we discuss the impact of the above results on image compression algorithms, and present some experimental results.

We show that when mean-square error is used to determine the performance of image compression algorithms, in particular vector quantization algorithms, the meansquare error measurement is dependent upon the data type of the digitized images. When using vector quantization the possibility exists for encoding images of one type with code books of another type, we show that this cross-encoding has an adverse effect on performance. Thus, when making comparative evaluations of different vector quantization compression techniques one must be careful to document the data type used in both the code book and the test image data. We also show that when mean-square error measurements are made in the perceptual space of a human visual model, the distortion measurements correlate more with subjective image evaluation than when the distortions are calculated in other spaces. We use a monochrome visual model to improve the quality of vector quantized images, but our preliminary results indicate that in general, the performance of the model is dependent upon the type of data and the coding method used.

We discuss the significance of masking effects for source coding of video signals without visible impairments. A new nonlinear spatiotemporal model of human threshold vision is proposed. Linearization yields the space-time-varying w-model. The model predicts both spatial and temporal masking effects accurately. Maximum bit-rate savings by irrelevancy reduction according to the w-model are evaluated for natural test pictures on the basis of the Shannon Lower Bound of rate distortion theory. Maximum bit-rate savings due to masking are below 0.5 bit/sample in the average. Typically 1/3 of the masking gain is due to spatial masking, the rest is due to the presence of dark and bright areas in the picture, where the visibility of noise is reduced. Gains due to temporal masking are significant only in the first 100 ms after a scene cut.

Efficient representation of images for human use requires an understanding of how the brain processes and represents visual information. Spatial imagery is represented in the brain in the receptive fields of visual neurons. Models of these neurons lead to models of image fidelity, and to digital implementations of these neural codes. This approach will be illustrated by two example codes. The advantages and difficulties of this approach will be discussed.

Image compression schemes abound with little work which compares their bit-rate performance based on subjective fidelity measures. Statistical measures of image fidelity, such as squared error measures, do not necessarily correspond to subjective measures of image fidelity. Most previous comparisons of compression techniques have been based on these statistical measures. We used a psychophysical method to estimate, for a number of compression techniques, a threshold bit-rate yielding a criterion level of performance in discriminating original and compressed images. The compression techniques studied include Block Truncation, Laplacian Pyramid, Block Discrete Cosine Transform, with and without a human visual system scaling, and Cortex Transform coders.

A model for the perception of distortions in pictures is suggested. It consists of two main parts: an adaptive input stage realized as a ROG (Ratio of Gaussian) pyramid also suited for applications in image coding and computer vision, and a further decomposition by orientation selective filters including a saturating nonlinearity acting at each point of the filter outputs. The output values for each point of each filter are regarded as feature vector of the internal representation of the input picture. The difference between the internal representations of original and distorted picture is evaluated as norm of the difference vector. Due to local nonlinearities this operation explains periodic and aperiodic masking effects.

The visual contrast sensitivity function (CSF) has found increasing use in image compression as new algorithms optimize the display-observer interface in order to reduce the bit rate and increase the perceived image quality. In most compression algorithms, increasing the quantization intervals reduces the bit rate at the expense of introducing more quantization error, a potential image quality degradation. The CSF can be used to distribute this error as a function of spatial frequency such that it is undetectable by the human observer. Thus, instead of being mathematically lossless, the compression algorithm can be designed to be visually lossless, with the advantage of a significantly reduced bit rate. However, the CSF is strongly affected by image noise, changing in both shape and peak sensitivity. This work describes a model of the CSF that includes these changes as a function of image noise level by using the concepts of internal visual noise, and tests this model in the context of image compression with an observer study.

Neural-network-like models of receptor position learning and interpolation function learning are being developed as models of how the human nervous system might handle the problems of keeping track of the receptor positions and interpolating the image between receptors. These models may also be of interest to designers of image processing systems desiring the advantages of a retina-like image sampling array.

The sequential sampling, transmission, and rendering of images has been studied in three related contexts: image transmission, vision, and graphics. The solutions developed in these contexts can be viewed as (different) specializations of a single adaptive sampling scheme. We present an analysis of this general scheme, and show how to specialize it. Our basic assumptions are that we can identify an original image (sometimes a model, sometimes the real world), a transmitter which can directly access the original, and a receiver interested in creating a reconstructed image. Different methods are motivated by different goals and constraints, especially the amount of knowledge that the transmitter is allowed to use (but not communicate). In the image synthesis problem, the process which evaluates the image function at a sample point can be viewed as the transmitter. The process which assembles an image from the individual point samples can be viewed as the receiver. In the progressive transmission problem, the transmitter selects or encodes values from a digital image. The receiver reconstructs increasingly accurate approximations of the image, as the samples arrive. These problems share three subproblems: selection of efficient sampling patterns, methods to adaptively control the sample rate, and filters for image reconstruction. In this paper, we explore several related methods of handling these problems, based on different constraints, and different weighting of the goals.

A quarter of a century ago I introduced two paradigms into psychology which in the intervening years have had a direct impact on the psychobiology of early vision and an indirect one on artificial intelligence (AI or machine vision). The first, the computer-generated random-dot stereogram (RDS) paradigm (Julesz, 1960) at its very inception posed a strategic question both for AI and neurophysiology. The finding that stereoscopic depth perception (stereopsis) is possible without the many enigmatic cues of monocular form recognition - as assumed previously - demonstrated that stereopsis with its basic problem of finding matches between corresponding random aggregates of dots in the left and right visual fields became ripe for modeling. Indeed, the binocular matching problem of stereopsis opened up an entire field of study, eventually leading to the computational models of David Marr (1982) and his coworkers. The fusion of RDS had an even greater impact on neurophysiologists - including Hubel and Wiesel (1962) - who realized that stereopsis must occur at an early stage, and can be studied easier than form perception. This insight recently culminated in the studies by Gian Poggio (1984) who found binocular-disparity - tuned neurons in the input stage to the visual cortex (layer IVB in V1) in the monkey that were selectively triggered by dynamic RDS. Thus the first paradigm led to a strategic insight: that with stereoscopic vision there is no camouflage, and as such was advantageous for our primate ancestors to evolve the cortical machinery of stereoscopic vision to capture camouflaged prey (insects) at a standstill. Amazingly, although stereopsis evolved relatively late in primates, it captured the very input stages of the visual cortex. (For a detailed review, see Julesz, 1986a)

Why does the mammalian visual system represent information as it does? If we assume that visual systems have evolved to cope with the natural environment then we might expect the coding properties of the visual system to be related to the statistical structure of our environment. Indeed, images of the natural environment do not have random statistics. The first-order statistics (e.g., distribution of pixel values) and second-order statistics (e.g., power spectra) of natural images have been discussed previously and they bear important relations to visual coding. Statistics higher than second-order are difficult to measure but provide crucial information about the image. For example, it can be shown that the lines and edges found in natural images are a function of these higher-order statistics. In this paper, these higher-order statistics will be discussed in relation to the coding properties of the mammalian visual system. It is suggested that the spatial parameters of the cortical 'filters' (e.g., bandwidths of simple and complex cells) are closely related to these higher-order statistics. In particular, it will be shown that the spatial non-linearities shown by cortical complex cells provide the early visual system with the information required to learn about these statistics.

Perceptual grouping was studied psychophysically with spatially periodic visual patterns. Different two-dimensional arrangements of identical elements (squares, diamonds, or disks) were used as test stimuli. At an exposure duration of 33 ms, the perceived orientational structure was found to depend upon the orientations of the most visible spatial frequency components. If the fundamental frequency was high (5 cpd), the perceived orientations were independent of the shape of the elements. If the fundamental spatial frequency was low (.5 cpd), the shape of the elements determined the perceived orientational structure. The results also show some response bias which is discussed.

We present a new paradigm for studying motion perception. This approach is based on a class of stimuli that we devised for testing the relative strength of stimulus attributes (luminance, color, spatial frequency, orientation, binocular disparity, etc.) in eliciting motion perception by correspondence matching. In this class of stimuli, different attributes are matched simultaneously in the spatio-temporal domain in a systematic, algorithmic manner which allows each attribute to produce motion in an arbitrary direction (if, of course, it is a token for movement perception), independently of the other attributes. This results in animation sequences in which many different motion paths may co-exist, each path due to a different attribute. Such an arrangement allows a direct comparison of the strength of attributes in eliciting movement. Results from psychophysical experiments based on our paradigm can be used to develop complex motion detection models for machine vision systems which attempt to approximate human performance. Similar methods are discussed for studying stereopsis mechanisms.

Several recent studies have proposed models of local motion detection for visual perception using spatiotemporal filters. However, these models are limited their motion perception capacity by the use of only one spatial frequency channel. But in the human visual system, it has been shown by various psychophysical studies that there are interactions between spatial frequency channels. In this paper, therefore, we propose one simple approach to introducing channel interactions into local motion detection models. In our method, true velocity is given as the intersection of lines representing the possible solutions on the velocity plane. Scalar motion sensors presented by Watson and Ahumada are used to get each line. Channel interactions are incorporated by plotting lines from multiple spatial frequency channels on the same velocity plane and finding their intersection. The effectiveness of these procedures was demonstrated with moving random dot patterns.

Using non-invasive psychophysical techniques, the effect of image degradation by induced optical blur was examined with respect to static versus dynamic target thresholds for central and peripheral visual field locations. Increased blur tended to raise thresholds centrally and have a lesser effect peripherally. In addition increased blur tended to have less effect upon dynamic than static thresholds.

By the mid 1990's, high definition television (HDTV) is likely to be widely available to the American public. Also by this time, construction of the Space Station is expected to have begun. NASA's statutory mandate includes being able to transmit information, often in the form of video images, from the space station for distribution to the media for public viewing. Eight-bit encoding of full-bandwidth digitized HDTV images for transmission would require a bit-rate of about 1.2 Gigabits. The available bit-rate, however, is a factor of 10 less than this. Thus, bit-rate reduction schemes will be needed that compress the image, without impairing its quality. This requires a good understanding of the basic psychophysics of image sampling, in both spatial and temporal domains. In the spatial domain, we have conducted a series of two-alternative forced choice experiments to compare the sharpness of televised moving images sampled using cardinal or diagonal patterns, with reference images of varying resolution. Considerable savings in bit-rate transmissions is possible if the same apparent sharpness can be obtained with diagonal as with cardinal sampling. Studies are underway to determine the optimal sampling patterns and pre-and-post filtering characteristics to maximize sharpness with minimum sampling artifacts. Digital transmission also requires images that are sampled in time. Such sampling as well as scan conversion between different frame rates, can produce motion artifacts, such as "jutter". In the laboratory, we have studied discriminability of jutter using the drifting sine-wave gratings with known spatial and temporal characteristics.

1.1. Three functions of color vision. Setting aside the complex psychological effects of color, related to esthetics, fashion, and mood, three relatively basic functions of color vision, which can be examined scientifically, are discernable. (1) With the eye in a given state of adaptation, color vision allows the perception of signals that otherwise would be below threshold, and therefore lost to perception. Evidence for this comes from a variety of two-color threshold experiments. (2) Visible contours can be maintained by color differences alone, regardless of the relative radiances of the two parts of the field whose junction defines the border. For achromatic vision, contour disappears at the isoluminant point. (3) Color specifies what seems to be an absolute property of a surface, one that enhances its recognizability and allows a clearer separation and classification of non-contiguous elements in the visual field.

This paper addresses strategies for the selection of fixed palettes of visually articulated colors. To access a computer's color generating capacity, of whatever magnitude, we need to work within some kind of logical display of its color possibilities. Also, in many display applications, the palette of colors is limited. The usual solution is to span the gamut of the display device with uniformly quantized increments. Since these increments are uniform with respect to the color process rather than with respect to the human visual system, the resulting palettes tend to appear non-uniform. Our approach is to start with a visually efficient color space, the Logical Visual Display, which is a refinement of the Munsell system, with its approximate visual uniform quantization. The orthogonal nature of this space makes it easy to adjust the relative resolution of all dimensions of visually perceived color. The division of our display space into planes of constant equal lightness value is advantageous to color mixing and dithering. Colors thus organized are the basic building blocks necessary for facile specification of communicative color sensation. We have found that there is little computational overhead for using these color palettes, since efficient search strategies, such as binary trees, can be independent of uniform, or symmetrical, incremental quantization.

Motivated by an increase in aesthetic value and information content, an increasing number of daily activities once performed in monochrome are being converted to color. As a result, the display and printing of color images is becoming an increasingly important topic. An often overlooked limitation of the color display or printing process is that of color gamut mismatch. We investigate both clipping and compression techniques for compensation of one type of color gamut mismatch. Image independent and image dependent algorithms using both linear and nonlinear compression techniques are considered. The lightness, saturation, and hue attributes of color are varied among the techniques in order to better understand in which attributes gamut mismatch errors are least perceptible.

The purpose of this study is: first, to measure the properties of human vision under conditions that are subject to "chromatic adaptation"; second, to compare the human visual properties to a number of different Retinex models. In particular, the experiments study Gray-world properties of Mondrians. Models using averages of radiance or Gray-world assumptions do not show good correlation with observers. Nonlinear, reset models of lightness normalize each pixel to the maxima in each waveband. Triplets of normalized lightness do show good correlation with observer color matches.

In this paper we compare the merits of two standards for the correlated color temperature of reference white for video display units, one standard at 6500 K and the other at 9300 K. The choice of 6500 K evolved from the N.T.S.C. specification for studio television receivers based on Illuminant C -- an "average daylight." The assumption of the N.T.S.C. was that, if equal-signal white is made to have the chromaticity of a reference illuminant, then colors rendered by the television systems should approximate the colors actually seen under the reference illuminant. Given this assumption, average daylight seemed to be the most natural adaptation state and hence acceptable for a white point. On the other hand, many manufacturers of home television receivers and VDU graphics displays chose 9300 K because the visual efficiency of the prevailing blue phosphors was greater than that of red or green phosphors, because viewers did not object to a blue color bias, and because 9300 K was bluer than any of the prevailing ambient illuminations. In the years since the original adoption of the 9300 standard, these facts underwent some change, suggesting a re-evaluation of the 9300 K standard.

A model is described that combines features of both classical and recent theories of color vision and visual adaptation. Neural signals from cone receptors activate gain control mechanisms, which cause neural attenuations to become proportionally greater as receptor signals grow. The attenuated signals are fed to three mechanisms of one of two "opponent colors" systems, and the resulting signals, which are compressed at a final stage, are used to either encode brightness and color appearance or to mediate discriminations and detections of lights. The model provides reasonable accounts of a very diverse body of data.