To develop an ultrahigh-definition television (UHDTV) camera-with a resolution 16 times higher than that of HDTV
resolution and a frame rate of 60 Hz (progressive)-a compact and high-mobility UHDTV camera using a 33M-pixel
CMOS image sensor to provide single-chip color imaging was developed. The sensor has a Bayer color-filter array
(CFA), and its output signal format is compatible with the conventional UHDTV camera that uses four 8M-pixel image
sensors. The theoretical MTF characteristics of the single-chip camera and a conventional four-8M-pixel CMOS camera
were first calculated. A new technique for Bayer CFA demosaicing used for the single-chip UHDTV camera was then
evaluated. Finally, a pick-up system for single-chip imaging with a 33M-pixel color CMOS image sensor was
measured. The measurement results show that the resolution of this is equivalent to or surpasses that of the conventional
four-8M-pixel CMOS camera. The possibility of a practical compact UHDTV camera that makes use of single-chip
color imaging was thereby confirmed.
We propose a method for measuring the multidirectional modulation transfer function (MTF) of digital image
acquisition devices using a Siemens star. There are two leading methods for measuring the MTF: the slanted-edge
method and the modulated Siemens star method. The former measures the MTF in the horizontal or vertical spatial
frequency based on the line spread function (LSF) derived from the edge profile of a slanted knife-edge image. The
latter measures the multidirectional MTF using a pattern circumferentially modulated with continuous gray levels. Our
method measures the multidirectional MTF using the multidirectional knife-edges of a Siemens star, which is a simple
binary image consisting of radial spokes. The vertical edge of the Siemens star is slightly slanted so that the
multidirectional edge profiles are obtained in super-resolution. A portion image consisting of the knife-edge is selected
in each direction and rotated so that the knife-edge stands upright with a slight tilt. Along the edge slope detected by
fitting a cumulative distribution function to the pixel levels, the pixels are projected onto the horizontal axis, forming the
edge profile. The resulting multidirectional MTF computed from the edge profiles is in excellent agreement with that
measured by the modulated Siemens star method.
We have been developing an ultra high definition television (UHDTV) system with a 7,680 horizontal by 4,320 vertical pixel resolution and a 60 Hz frame rate. This system, which is called Super Hi-vision (SHV), is expected to serve the next generation of broadcasting services. We have just completed the world's first imaging equipment that is capable of capturing video at a full SHV resolution. In designing this equipment, we decided to develop three new devices, taking into account the camera performance and the ease of implementation. First, we developed a 33-megapixel CMOS image sensor. Its pixel size of 3.8 &mgr;m sq. retained the dynamic range of the sensor above 60 dB even with a 3-transistor pixel structure. Second, a fixed focal length lens was developed to create an adequate MTF right up to the limiting resolution of the sensor. Third, we developed a signal-processing device capable of handling 72 Gbps signals and cascading boards to expand the process. SHV images with a modulation of 20% at the Nyquist frequency were obtained by using these three key technologies.
We are currently researching a next-generation broadcasting system named "Super Hi-Vision", which provides images
with much higher resolution and quality than a high definition television, and are developing displays for this system. We
have developed a projector which can display the high-resolution images of Super Hi-Vision (7680x4320 pixels) and has
a high dynamic range of 1.1 million to 1. This projector features a serial combination of two modulation blocks: the first
block for chrominance modulation and the second block for luminance modulation. While a conventional projector has one
modulation block that contains three devices to modulate red, green, and blue light, our projector has another modulation
block that contains a device for luminance to further modulate the combined RGB modulated light. This configuration
enables the projector to output high-resolution color images by combining three low-resolution devices for chrominance
modulation and one high-resolution device for luminance modulation. As a high-resolution device, we have newly developed
a 1.75-inch liquid crystal on silicon device with 8192x4320 pixels. In addition, the dynamic range is dramatically
improved because this dual modulation scheme minimizes the black levels in projected images. We demonstrated that our
projector displays Super Hi-Vision color images and that it has an extremely high dynamic range of 1.1 million to 1 and a
fine 10-bit tone reproduction.
Achieving ultimate visual realness of natural images on a display requires high resolution, so that artifacts due to finite
image resolution are undetectable. An image resolution of 30 cycles/degree (cpd) or one pixel/arc-minute is often used as
the criterion for viewing conditions when assessing displayed image quality. It is reasoned that if the pixel size is smaller
than the separable angle of normal vision (20/20), the pixel structure is invisible and doesn't negatively affect image
quality. However, it is not clear whether 30 cpd resolution is adequate to prevent seeing artifacts, especially for observers
with better than 20/20 vision. We conducted experiments to find the threshold resolution of natural images and its
dependence on visual acuity. Three objects were used; each object was presented 60 times at 5 resolutions (19.5, 26, 39,
52, or 78 cpd) next to the same image at a resolution of 156 cpd. Forty-five observers with visual acuity of 20/20 or
better were asked to make a forced-choice distinction between the image pair in regard to resolution. Each observer
indicated which image of the pair appeared at a higher resolution. The results show that the mean resolution for 75%
correct responses for each of the visual acuity groups increased from more than 30 cpd as visual acuity increased and
reached a plateau at 40-50 cpd at -0.3 logMAR.
We developed an experimental single chip color HDTV video image acquisition system with 8M-pixel CMOS
image sensor. The imager has 3840 (H) × 2160 (V) effective pixels and built-in analog-to-digital converters, and its
frame rate is 60-fps with progressive scanning. The MTF characteristic we measured with this system on luminance
signal in horizontal direction was about 45% on 800 TV lines. This MTF was better than conventional three-pickup
broadcasting cameras, therefore the enhancement gain (the "enhancement area" in MTF) of the 8M single-chip HDTV
system was about a half of the three-pickup cameras. We also measured the color characteristics and corrected the color
gamut using matrix gain on primary colors. We set the color correction target similar to that of three-pickup color
cameras in order to use multiple cameras to shoot for broadcasting, where all cameras are controlled in the same manner.
The color error between the single-chip system and three-pickup cameras after the correction became 2.7, which could
be useful in practice.
Image resolution is one of the important factors for visual realness. We performed subjective assessments to examine
the realness of images at six different resolutions, ranging from 19.5 cpd (cycles per degree) to 156 cpd. A paired-comparison
procedure was used to quantify the realness of six images versus each other or versus the real object. Three
objects were used. Both real objects and images were viewed through a synopter, which removed horizontal disparity
and presented the same image to both eyes. Sixty-five observers were asked to choose the viewed image which was
closer to the real object and appeared to be there naturally for each pair of stimuli selected from the group of six images
and the real object. It was undisclosed to the observers that real objects were included in the stimuli. The paired
comparison data were analyzed using the Bradley-Terry model. The results indicated that realness of an image increased
as the image resolution increased up to about 40-50 cpd, which corresponded to the discrimination threshold calculated
based on the observers' visual acuity, and reached a plateau above this threshold.
We have developed color camera for an 8k x 4k-pixel ultrahigh-definition video system, which is called Super Hi- Vision, with a 5x zoom lens and a signal-processing system incorporating a function for real-time lateral chromatic aberration correction. The chromatic aberration of the lens degrades color image resolution. So in order to develop a compact zoom lens consistent with ultrahigh-resolution characteristics, we incorporated a real-time correction function in the signal-processing system. The signal-processing system has eight memory tables to store the correction data at eight focal length points on the blue and red channels. When the focal length data is inputted from the lens control units, the relevant correction data are interpolated from two of eights correction data tables. This system performs geometrical conversion on both channels using this correction data. This paper describes that the correction function can successfully reduce the lateral chromatic aberration, to an amount small enough to ensure the desired image resolution was achieved over the entire range of the lens in real time.
We have developed an experimental single-chip color HDTV image acquisition system using 8M-pixel CMOS image sensor. The sensor has 3840 × 2160 effective pixels and is progressively scanned at 60 frames per second. We describe the color filter array and interpolation method to improve image quality with a high-pixel-count single-chip sensor. We also describe an experimental image acquisition system we used to measured spatial frequency characteristics in the horizontal direction. The results indicate good prospects for achieving a high quality single chip HDTV camera that reduces pseudo signals and maintains high spatial frequency characteristics within the frequency band for HDTV.
When designing a system capable of capturing and displaying three-dimensional (3-D) moving images in real time by the integral imaging (II) method, one challenge is to eliminate pseudoscopic images. To overcome this problem, we propose a simple system with an array of three convex lenses. This paper first describes by geometrical optics the lateral magnification of the elemental optics and expansion of an elemental image, confirming that the elemental optics satisfies the conditions under which pseudoscopic images can be avoided. By the II method, adjacent elemental images must not overlap, a condition also satisfied by the proposed optical system. Next, the paper describes an experiment carried out to acquire and display 3-D images. The real-time system we have constructed comprises elemental optics array with 54(H) x 59(V) elements, a CCD camera to capture a group of elemental images created by the lens array, and a liquid crystal panel to display these images. The experiment results confirm that the system produces orthoscopic images in real time and so is effective for
real-time application of the II method.
We propose a system to calculate the spatial distortion in 3-D images based on the shooting, display, and viewing conditions. It can be used to predict the extent of the perceived puppet-theater effect and the cardboard effect. The magnitude of the spatial distortion and the extent of the puppet-theater and cardboard effects are displayed using a space grid whose size can be estimated based on the objects' depths, calculated from the binocular parallax of the acquired stereoscopic images. This system can also be used to predict excessive binocular parallax and excessive parallax distribution. Several cases in which puppet-theater and cardboard effects are expected to be produced are presented. We also demonstrate how the proposed system might be used to predict ratings of naturalness and quality of depth.
The problems associated with watching stereoscopic HDTV have been classified into three groups, one of which is how natural/unnatural stereoscopic pictures look to viewers. It is known that the shooting and viewing conditions affect the depth of a stereoscopic image, and this depth distortion is a major factor influencing the viewer's stereoscopic perception. The second group concerns the visual comfort/discomfort. Visual discomfort is caused by the difficulty of fusing left and right images because of excessive binocular parallax and its temporal changes. We have studied how visual comfort is affected by the range of parallax distribution and temporal parallax changes. The results show that stereoscopic images are comfortable to view for an angular parallax of up to about 60 minutes and that visual comfort is achieved if discontinuous temporal changes are angle of 60 minutes or less. The third group concerns visual fatigue that a viewer experiences after viewing a stereoscopic HDTV program, which is thought to be mainly caused by the mismatch between the eyes' convergence and accommodation. We confirmed that, after observing stereoscopic images for about an hour, the fusion range diminishes and the viewer's visual functions deteriorate as a result.
The relationship between visual comfort and parallax distribution for stereoscopic HDTV has been studied. In this study, we first examined a method for measuring this parallax distribution. As it is important to understand the characteristics of the distribution in a frame or temporal changes of the characteristics, rather than having detailed information on the parallax at every point, we propose a method to measure the parallax based on the phase correlation. It includes a way of reducing the measurement error depending on the phase correlation method. The method was used to measure stereoscopic HDTV images with good results. Secondly, we conducted a subjective evaluation test of visual comfort and sense of presence using 48 different stereoscopic HDTV pictures, and compared the results with the parallax distributions in these pictures measured by the proposed method. The comparison showed that the range of parallax distribution and the average parallax distribution significantly affect visual comfort when viewing stereoscopic HDTV images. It is also suggested that the range of parallax distribution in many of the images that were judged comfortable to view is located within approximate 0.3Diopter.
This paper describes a laser telecine which has been developed for transferring film images into HDTV signals with high picture quality. In this equipment the film image is directly scanned by a laser beam concentrated onto a small spot with high intensity so that both high resolution and a high S/N ratio can be obtained. This equipment has already been delivered to the broadcasting station and are being used to create HDTV programs using 35mm films.
This paper describes a high-picture-quality laser film recorder developed for HDTV movie production applications. This equipment can directly record HDTV pictures in realtime onto a low-sensitivity fine grain color film using three laser beams of red green and blue. A practical model has already been developed introduced into production studios and is being used for movie production. 1 .