Systematic and effective color management for displays on mobile devices is becoming increasingly more challenging.
A list of the main challenges includes (a) significant differences in display technologies, (b) significant display color
response and tone response variability, (c) significant display color gamut variability, (d) significant content gamut
variability, (e) mixing content with different color gamuts, (f) Significant variability in viewing conditions and (g)
significant mobile display power consumption. This paper will provide general descriptions of the above challenges,
their characteristics and complexities.
Picture adjustment is referred to those adjustments that affect the four main subjective perceptual image attributes: Hue,
Saturation, Brightness (sometimes called Intensity) and Contrast--HSIC adjustments. The common method used for this
type of adjustments in a display processing pipe is based on YCbCr color space and a 3x4 color adjustment matrix.
Picture adjustments based on this method, however, leads to multiple problems such as adjusting one attribute leads to
degradation of other attributes. As an alternative, other color spaces such as HSV can be used to generate more
consistent and effective picture adjustments. In this paper, the results of a comparative performance analysis between the
two methods based on YCbCr and HSV color spaces (for HSIC adjustments) are presented.
Cell-phone display performance (in terms of color quality and optical efficiency) has become a critical factor in creating
a positive user experience. As a result, there is a significant amount of effort by cell-phone OEMs to provide a more
competitive display solution. This effort is focused on using different display technologies (with significantly different
color characteristics) and more sophisticated display processors. In this paper, the results of a mobile-display
comparative performance analysis are presented. Three cell-phones from major OEMs are selected and their display
performances are measured and quantified. Comparative performance analysis is done using display characteristics such
as display color gamut size, RGB-channels crosstalk, RGB tone responses, gray tracking performance, color accuracy,
and optical efficiency.
LCD displays exhibit significant amount of variability in their tone-responses, color responses and backlight-modulation
responses. LCD display characterization and calibration using a spectrometer or a color meter, however, leads to two
basic deficiencies: (a) It can only generate calibration data based on a single spot on the display (usually at panel center);
and (b) It generally takes a significant amount of time to do the required measurement. As a result, a fast and efficient
system for a full LCD display characterization and calibration is required. Herein, a system based on a 3CCD
calorimetrically-calibrated camera is presented which can be used for full characterization and calibration of LCD
displays. The camera can provide full tri-stimulus measurements in real time. To achieve high-degree of accuracy,
colorimetric calibration of camera is carried out based on spectral method.
Recently, 3D displays and videos have generated a lot of interest in the consumer electronics industry. To make
3D capture and playback popular and practical, a user friendly playback interface is desirable. Towards this end,
we built a real time software 3D video player. The 3D video player displays user captured 3D videos, provides
for various 3D specific image processing functions and ensures a pleasant viewing experience. Moreover, the
player enables user interactivity by providing digital zoom and pan functionalities. This real time 3D player was
implemented on the GPU using CUDA and OpenGL. The player provides user interactive 3D video playback.
Stereo images are first read by the player from a fast drive and rectified. Further processing of the images
determines the optimal convergence point in the 3D scene to reduce eye strain. The rationale for this convergence
point selection takes into account scene depth and display geometry. The first step in this processing chain is
identifying keypoints by detecting vertical edges within the left image. Regions surrounding reliable keypoints
are then located on the right image through the use of block matching. The difference in the positions between
the corresponding regions in the left and right images are then used to calculate disparity. The extrema of
the disparity histogram gives the scene disparity range. The left and right images are shifted based upon the
calculated range, in order to place the desired region of the 3D scene at convergence. All the above computations
are performed on one CPU thread which calls CUDA functions. Image upsampling and shifting is performed in
response to user zoom and pan. The player also consists of a CPU display thread, which uses OpenGL rendering
(quad buffers). This also gathers user input for digital zoom and pan and sends them to the processing thread.
Putting high quality and easy-to-use 3D technology into the hands of regular consumers has become a recent
challenge as interest in 3D technology has grown. Making 3D technology appealing to the average user requires
that it be made fully automatic and foolproof. Designing a fully automatic 3D capture and display system
requires: 1) identifying critical 3D technology issues like camera positioning, disparity control rationale, and
screen geometry dependency, 2) designing methodology to automatically control them. Implementing 3D capture
functionality on phone cameras necessitates designing algorithms to fit within the processing capabilities of the
device. Various constraints like sensor position tolerances, sensor 3A tolerances, post-processing, 3D video
resolution and frame rate should be carefully considered for their influence on 3D experience. Issues with
migrating functions such as zoom and pan from the 2D usage model (both during capture and display) to 3D
needs to be resolved to insure the highest level of user experience. It is also very important that the 3D usage
scenario (including interactions between the user and the capture/display device) is carefully considered. Finally,
both the processing power of the device and the practicality of the scheme needs to be taken into account while
designing the calibration and processing methodology.
KEYWORDS: LCDs, RGB color model, Calibration, Gadolinium, Error analysis, Data modeling, LED displays, Light emitting diodes, CRTs, Analytical research
LCD displays exhibit significant color crosstalks between their red, green and blue channels (more or less depending on
the type of LCD technology). This problem, if it is not addressed properly, leads to (a) a significant color errors in the
rendered images on LCD displays and (b) a significant gray tracking problem. The traditional method for addressing this
problem has been to use a 3x3 color correction matrix in the display processing pipe. Experimental data clearly shows
that this linear model for color correction is not sufficient to address color crosstalk problem in LCD displays. Herein, it
is proposed to use higher order polynomials for color correction in the display processing pipe. This paper presents
detailed experimental results and comparative analysis on using polynomial models with different orders for color
correction.
The MIPI standard has adopted DPCM compression for RAW data images streamed from mobile cameras. This
DPCM is line based and uses either a simple 1 or 2 pixel predictor. In this paper, we analyze the DPCM
compression performance as MTF degradation. To test this scheme's performance, we generated Siemens star
images and binarized them to 2-level images. These two intensity values where chosen such that their intensity
difference corresponds to those pixel differences which result in largest relative errors in the DPCM compressor.
(E.g. a pixel transition from 0 to 4095 corresponds to an error of 6 between the DPCM compressed value and
the original pixel value). The DPCM scheme introduces different amounts of error based on the pixel difference.
We passed these modified Siemens star chart images to this compressor and compared the compressed images
with the original images using IT3 MTF response plots for slanted edges. Further, we discuss the PSF influence
on DPCM error and its propagation through the image processing pipe.
KEYWORDS: Image compression, Visualization, Video compression, Video, Computer programming, Data compression, Reconstruction algorithms, Data processing, Transform theory, RGB color model
We describe design of a low-complexity lossless and near-lossless image compression system with random access,
suitable for embedded memory compression applications. This system employs a block-based DPCM coder using
variable-length encoding for the residual. As part of this design, we propose to use non-prefix (one-to-one) codes for
coding of residuals, and show that they offer improvements in compression performance compared to conventional
techniques, such as Golomb-Rice and Huffman codes.
Significant sensitivity variations among cell-phone camera modules have been observed. As a result, for an effective and
reliable white balancing, per-module RGB-ratios calibration/estimation under various illumination conditions is
required. Herein, a new technique is proposed which minimizes/simplifies RGB-ratios calibration/estimation process.
The proposed method could be based on either direct image capture or spectral numerical processing--the latter is shown
to be more flexible and accurate.
Certain feedback loop based algorithms contained in an image processing engine, such as auto white balance, auto
exposure or auto focus, are best designed and evaluated within a real-time framework due to strong requirements of
close study of the dynamics present. Furthermore, the development process entails the usual flexibility associated with
any software module implementation, such as the ability to dump debugging information or placement of break points in
the code. In addition, the end deployment platform is not usually available during the design process, while tuning of the
above mentioned algorithms must encompass particularities of each individual target sensor. We explore in this paper a
real-time hardware-software solution that addresses all the requirements mentioned before and functions on a non-real
time operating system (Windows). Moreover we exemplify and quantify the hard deadlines required by such a feedback
control loop algorithm and illustrate how they are supported in our implementation.
Color noise in the form of clusters of color non-uniformity is a major negative quality factor in color images. This type
of noise is significantly more pronounced in CMOS cameras with increasingly smaller pixel sizes (e.g., 1.75μm and
1.4μm pixel sizes). This paper identifies and quantifies temporal noise as the main factor for this type of noise. As well,
it is shown how differences in R/G/B responses and as well possible presence of R/G/B-response non-linearity can
exacerbate color-blotch noise. Furthermore, it is shown how run-time averaging can effectively remove this noise (to a
large extent) from a color image-if capture condition permits.
Chrominance noise appears as low frequency colored blotches throughout an image, especially in darker flat areas. The effect is more pronounced in lower light levels where the characteristic features are observed as irregularly shaped clusters of colored pixels that vary anywhere from 15 to 25 pixels across. This paper proposes a novel, simple and intuitive method of reducing chrominance noise in processed images while minimizing color bleeding artifacts. The approach is based on a hybrid multi scale spatial dual tree adaptive wavelet filter in hue-saturation-value color space. Results are provided in terms of comparisons on real images between the proposed method and another state of the art method.
Image blur due to handshake is a significant problem for cell-phone cameras. A set of new handshake characteristics are
established using a high-frame-rate image capture and processing system. Based on these newly established handshake
characteristics, an efficient, effective and inexpensive method for minimizing image blur due to handshake is proposed.
The results of applying the proposed method under different scene conditions are presented.
This paper describes the framework used in one of the pilot studies run under the I3A CPIQ initiative to quantify overall
image quality in cell-phone cameras. The framework is based on a multivariate formalism which tries to predict overall
image quality from individual image quality attributes and was validated in a CPIQ pilot program. The pilot study
focuses on image quality distortions introduced in the optical path of a cell-phone camera, which may or may not be
corrected in the image processing path. The assumption is that the captured image used is JPEG compressed and the cellphone
camera is set to 'auto' mode. As the used framework requires that the individual attributes to be relatively
perceptually orthogonal, in the pilot study, the attributes used are lens geometric distortion (LGD) and lateral chromatic
aberrations (LCA). The goal of this paper is to present the framework of this pilot project starting with the definition of
the individual attributes, up to their quantification in JNDs of quality, a requirement of the multivariate formalism,
therefore both objective and subjective evaluations were used. A major distinction in the objective part from the 'DSC
imaging world' is that the LCA/LGD distortions found in cell-phone cameras, rarely exhibit radial behavior, therefore a
radial mapping/modeling cannot be used in this case.
Bilateral filtering is an effective technique for reducing image noise while preserving edge content. The filter kernel is constructed based on two criteria of neighboring pixels, namely photometric resemblance and geometric proximity. The Euclidean distance is used as a metric for the photometric portion of the kernel in the classic definition of the filter. We illustrate in this paper a simplified method for calculating the Euclidean distance metric which reduces the computational complexity of the filter. Furthermore, we generalize the idea of bilateral filtering by linking the filter processing parameters to the noise profile of a CMOS image sensor and present a simple method for tuning the performance of the filter.
Cell-phone cameras generally use mini lenses that are wide-angle and fixed-focal length (4-6 mm) with a fixed aperture (usually f/2.8). As a result, these mini lenses have very short hyper-focal lengths (e.g., the estimated hyper-focal length for a 3.1-MP cell-phone camera module with a 5.6-mm mini lens is only about 109 cm which covers focused-object distances from about 55 cm to infinity). This combination of optical characteristics can be used effectively to achieve: (a) a faster process for auto-focusing based on a small number of pre-defined non-uniform lens-position intervals; and (b) a depth map generation (coarse or fine depending on the number of focus regions of interest--ROIs) which can be used for different image capture/processing operations such as flash/no-flash decision-making. The above two processes were implemented, tested and validated under different lighting conditions and scene contents.
KEYWORDS: Cameras, High dynamic range imaging, Sensors, Image processing, CMOS sensors, Imaging systems, Image restoration, Optical filtering, Signal to noise ratio, Algorithm development
Most cell-phone cameras today use CMOS sensors with higher and higher pixel counts, which in turn, results in smaller pixel sizes. To achieve good performance in current technologies, pixel structures are fairy complicated. Increasing complexity in pixel structure, coupled with optical constraints specific to cell-phone cameras, results in non-uniform light response over the pixel array. A cell-phone camera sensor module typically has a light-falloff of -40% from center relative to an edge. This high fall-off usually has non-radial spatial distribution making lens fall-off corrections
complicated. The standard method of reducing light fall-off is linear (i.e. multiplicative gain), resulting in close to a ~2x peripheral gain and a corrected image with lower dynamic range. To address this issue, a novel idea is explored where the fall-off is used to increase the dynamic range of the captured image. As a typical lens fall-off needs a gain of up to 2x centre vs edge, the fall-off can be thought of as a 2D neutral density filter which allows up to 2x more light to be sensed towards the periphery of the sensor. The proposed solution uses a 2D scaled down gain map to correct the fall-off. For each pixel, using the gain map, an inflection point is calculated which is used to estimate the associated pixel transfer characteristic which is linear up to the inflection point and then becomes logarithmic.
KEYWORDS: Image processing, Signal to noise ratio, Sensors, RGB color model, Image sensors, Cameras, Signal processing, Color image processing, Interference (communication), Optical filters
An image processing path typically involves color correction or white balance resulting in higher than unity color gains. A gain higher than unity increases the noise in that respective channel, and therefore degrades the SNR performance of the input signal. If the input signal does not have enough SNR to accommodate the extra gain, the resultant color image has increased color noise. This is the usual case for color processing in cell phone cameras, which have sensors with limited SNR and high color crosstalk. This phenomenon degrades images more as illuminants differ from D65. In addition, the incomplete information for clipped pixels often results in unsightly artifacts during color processing. To correct this dual problem, we investigate the use of under unity color gains, which, by increasing the exposure of the sensor, would improve the resultant SNR of the color corrected image. The proposed method preserves the appearance of clipped pixels and the overall luminance of the image, while applying the appropriate color gains.
As CMOS imaging technology advances, sensor to sensor differences increase, creating an increasing need for
individual, per sensor, calibration. Traditionally, the cell-phone market has a low tolerance for complex per unit
calibration. This paper proposes an algorithm that eliminates the need for a complex test environment and does not
require a manufacturing based calibration on a per phone basis. The algorithm locates "bad pixels", pixels with light
response characteristics out of the mean range of the values specified by the manufacturer in terms of light response. It
uses several images captured from a sensor without using a mechanical shutter or predefined scenes. The implementation
that follows uses two blocks: a dynamic detection block (local area based) and a static correction block (location table
based). The dynamic block fills the location table of the static block using clustering techniques. The result of the
algorithm is a list of coordinates containing the location of the found 'bad pixels'. An example is given of how this
method can be applied to several different cell-phone CMOS sensors.
The classical bilateral filter smoothes images and preserves edges using a nonlinear combination of surrounding pixels. Our modified bilateral filter advances this approach by sharpening edges as well. This method uses geometrical and photometric distance to select pixels for combined low and high pass filtering. It also uses a simple window filter to reduce computational complexity.
KEYWORDS: Colorimetry, RGB color model, Cameras, Image sensors, Sensors, Signal to noise ratio, Color reproduction, Reflectivity, Visualization, Image processing
The chromaticity of an acquired image reconstructed from a Bayer pattern image sensor is heavily dependent on the scene illuminant and needs color corrections to match human visual perception. This paper presents a method to 'white balance' an image that is computationally inexpensive for hardware implementation, has reasonable accuracy without the need of storing the full image, and is aligned to the current technical development of the field. The proposed method introduces the use of a 2D chromaticity diagram of the image to extract information about the resultant scene reflectance. It assumes that the presence of low-saturated colors in the scene will increase the probability of retrieving accurate scene color information.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.