We propose a frequency domain scheme for obtaining subpixel estimates of interframe motion for arbitrary shaped regions. Our scheme is based on phase correlation and the shape adaptive discrete Fourier transform and one of its key features is that it does not require extrapolation, which requires extra complexity and can dilute the accuracy of the estimated motion parameters. We demonstrate that our method outperforms in terms of subpixel accuracy and motion-compensated prediction error both conventional phase correlation but also shape adaptive techniques operating in the frequency domain and requiring extrapolation.

The method of paired comparison based on Thurstone's case V of his law of comparative judgments is often used as a psychophysical method to derive interval scales of perceptual qualities in imaging applications. However, methods for determining confidence intervals and critical distances for significant differences have been elusive, leading some to abandon the simple analysis provided by Thurstone's formulation. Monte Carlo simulations of paired comparison experiments were performed in order to derive an empirical formula for determining error. The results show that the variation in the distribution of experimental results can be well predicted as a function of stimulus number and the number of observations. Using these results, confidence intervals and critical values for comparisons can be made using traditional statistical methods.

The authors propose a technique for performing real-time correction of the gain nonuniformity at output taps in an area-parallel-type frame transfer CCD (FT-CCD) by incorporating a reference optical signal. This technique activates a reference light source every vertical blanking (V-blanking) period, during which time the image sensor is shielded from incident light, and uses that output signal level to automatically adjust the gain of each block. For this method, a color-separation prism with reference light sources has been developed. In addition, to obtain a reference signal from the reference light source emitting for about 11% of the V-blanking period, a new method of accumulating the electron signal generated by the reference light source is described for the CCD. The results of a camera experiment using an 8-million-pixel CCD are described, and the compensation accuracy of this method is discussed.

We propose a system to calculate the spatial distortion in 3-D images based on the shooting, display, and viewing conditions. It can be used to predict the extent of the perceived puppet-theater effect and the cardboard effect. The magnitude of the spatial distortion and the extent of the puppet-theater and cardboard effects are displayed using a space grid whose size can be estimated based on the objects' depths, calculated from the binocular parallax of the acquired stereoscopic images. This system can also be used to predict excessive binocular parallax and excessive parallax distribution. Several cases in which puppet-theater and cardboard effects are expected to be produced are presented. We also demonstrate how the proposed system might be used to predict ratings of naturalness and quality of depth.

Color filter array (CFA) demosaicking is an essential process for restoring full-color images from incomplete color samples acquired by single-sensor digital cameras. We present two main contributions to CFA demosaicking. First, we analyze the causes of two main types of CFA demosaicking artifacts and examine the schemes that are effective in suppressing them, respectively. Second, by combining and extending the core merits of the schemes examined, we compose a new CFA demosaicking algorithm to suppress as many demosaicking artifacts as possible and obtain full-color images of high quality. Experiments using a large variety of test images have been conducted, and the results show that the proposed method outperforms the existing state-of-the-art methods both visually and in terms of peak signal-to-noise ratio.

The design of a gamut mapping algorithm (GMA) is always a compromise between preserving different competing aspects such as color, contrast, and lightness. A natural requirement of a GMA is that the algorithmic treatment of this competition has to avoid any additional artifacts such as discontinuities or loss of contrast. In this work, several common gamut mapping algorithms are studied from this aspect, resulting in the observation that problems with geometric discontinuities are widespread. For the assessment of the phenomena induced by local mapping properties, an algorithmic test is developed and applied. This new test supports both the quality check of existing as well as the development of new GMAs.

A visual method for characterizing cathode ray tube (CRT) displays and finding corresponding colors is presented. The method is motivated by the considerable expense of measuring instruments and the consideration of color appearance factors. In the experiment, a set of color chips in the natural color system (NCS) color atlas, an illuminant, and a CRT are used. 487 sample pairs of RGB (CRT) and HVC, which are the attributes of NCS color chips, are obtained by experimentally comparing softcopies and hardcopies in the office environment in the sense of best color matching to nine observers. Both the spectral data of the color chips and the colorimetric values of the CRT samples are measured. In addition, 12 error back-propagation (BP) neural networks are trained to help the realization of the transformation from HVC to RGB for data generalization. The current results show that the displays on the CRT can match the chips in color perception well.

We describe an image codec based on subband decomposition and vector quantization techniques. Biorthogonal wavelets are used for image decomposition, while encoding relies on multistage, multiclass vector quantization with a tree partitioning algorithm. The codec is evaluated for different images, and the results are compared with the popular set partitioning in hierarchical trees (SPIHT) image codec and with the new JPEG 2000 standard. The experimental evaluation suggests that the proposed codec is competitive with SPIHT and JPEG 2000 for all bit rates, especially for images with medium to high detail content. The proposed codec also provides an embedded bitstream with an asymmetric codec complexity, in that the decoder is less complex than the encoder.

We present two new techniques for reducing blocking artifacts in motion compensated frames. These techniques can be applied generally to motion compensated frames obtained by employing any partitioning schemes. In the first technique, the weighting coefficients of the overlapped block motion compensation are modeled with fuzzy relations. These fuzzy relations are formed based on two fuzzy sets, which define the closeness of a given pixel to the block boundaries in the horizontal and vertical directions, respectively. The second technique uses the similarity between a set of pixels inside a partitioned region and that on the borders to modify the border pixels. Simulations conducted on several benchmark video sequences indicate that the proposed techniques can improve the subjective quality of the frames as well as reduce the prediction error.

We present a technique to robustly transmit regions of interest in the JPEG2000 framework. The technique assumes a prioritized region of interest coding and optimally assigns channel protection to the coded data according to the importance of every packet in the final bit stream. The mean energy of the transform coefficients contained in a packet, and the distance of a packet from the region of interest, determine the importance of every packet. Channel protection is achieved by means of a concatenation of a cyclic redundancy check outer coder and an inner rate-compatible convolutional coder. Simulation results performed over a Rayleigh-fading channel show an improvement in the visual quality of the reconstructed images.

A robust, invisible watermarking scheme is proposed for digital images, where the watermark is embedded using the block-based lapped orthogonal transform (LOT). The embedding process follows a spread spectrum watermarking approach. In contrast to the use of transforms such as discrete cosine transform, our LOT watermarking scheme allows larger watermark embedding energy while maintaining the same level of subjective invisibility. In particular, the use of LOT reduces block artifacts caused by the insertion of the watermark in a block-by-block manner, hence obtaining a better balance between invisibility and robustness. Moreover, we use a human visual system (HVS) model to adaptively adjust the energy of the watermark during embedding. In our HVS model, each block is categorized into one of four classes (texture, fine-texture, edge, and plain-area) by using a feature known as the texture masking energy. Blocks with edges are also classified according to the edge direction. The block classification is used to adjust the watermark embedding parameters for each block.

We present a new watermarking scheme for joint ownership verification of digital images. In the proposed scheme, the watermarking key is held by a trust third party (TTP), and it is kept secret to all co-owners using Shamir's secret sharing scheme. Each owner is assigned a share of the watermarking key and a distinct detection watermark. The scheme can provide individual and joint watermark detections, and the robustness of individual watermark detections does not decrease with the increase of co-owners. Experimental results and analysis demonstrate that the scheme is effective for joint ownership verification and is robust to common image processing operations.

We propose a nonextensive information-theoretic measure called Jensen-Tsallis divergence, which may be defined between any arbitrary number of probability distributions, and we analyze its main theoretical properties. Using the theory of majorization, we also derive its upper bounds performance. To gain further insight into the robustness and the application of the Jensen-Tsallis divergence measure in imaging, we provide some numerical experiments to show the power of this entopic measure in image edge detection.

We discuss how an orthogonalized fractal decoding algorithm can be used to derive a fractal decomposition of the encoded image into a dc and a set of ac components that characterize the edge information in the image. The dc component can be characterized by its histogram. The ac components provide a model for the texture, and the distribution of each ac component can be well represented by a generalized Gaussian density (GGD), which can be efficiently characterized by two GGD parameters. Such characterization of the ac components is exploited in this work for texture image retrieval, and experimental results show that our proposed indexing technique based on our novel fractal dc and ac signature provides superior retrieval performance compared to current fractal indexing techniques. Moreover, the performance of our approach is comparable to the state of the art wavelet-based method at a fraction of the retrieval time.

We describe a texture description algorithm, designed in the wavelets domain, to reduce the dimension of an existing texture-based feature vector and improve on the existing texture description algorithm in terms of both effectiveness and efficiency. We first demonstrate that the estimated texture directions at different wavelet decomposition scales are very similar. Thus, a new texture description with explicit direction representation can be constructed to improve discrimination capability. Second, we propose a subband integration scheme to further improve the texture description and achieve robustness to rotation of texture patterns. Third, a range of successful texture description elements developed in the pixel domain are applied to the LL subband and added to the texture descriptor for further enhancement of the proposed algorithm. Extensive testing, benchmarked by the existing techniques, shows that the proposed algorithm not only reduces the sensitivity of retrieval to image texture rotation, but also improves the retrieval accuracy.

One of the main issues of face recognition is to extract features from face images, which include both local and global features. We present a novel method to perform feature fusion at the feature level. First, global features are extracted by principal component analysis (PCA), while local features are obtained by active appearance model (AAM) and Gabor wavelet transform (GWT). Second, two types of features are fused by weighted concatenation. Finally, Euclidean and feature distances of fused features are applied to carry out a nearest neighbor classifier. The method is evaluated by the recognition rates and computation cost over two face image databases [AR (created by A. Martinez and R. Benavente) and SJTU-IPPR (Shanghai JiaoTong University-Institute of Image Processing and Pattern Recognition)]. Compared with PCA and elastic bunch graph matching (EBGM), the presented method is more effective. Though the recognition rate of the presented method is not as good as nonlinear feature combination (NFC), low computation cost is its superiority. In addition, experimental results show that the novel method is robust to variations over time, expression, illumination, and pose to a certain extent.

We present an efficient and reliable algorithm to detect human faces in an image under different lighting conditions. In our algorithm, skin-colored pixels are identified using a region-based approach, which can provide more reliable skin color segmentation under various lighting conditions. In addition, to compensate for extreme lighting conditions, a color compensation scheme is proposed, and the distributions of the skin-color components under various illuminations are modeled by means of the maximum-likelihood method. With the skin-color regions detected, a ratio method is proposed to determine the possible positions of the eyes in the image. Two eye candidates form a possible face region, which is then verified as a face or not by means of a two-stage procedure with an eigenmask. Finally, the face boundary region of a face candidate is further verified by a probabilistic approach to reduce the chance of false alarms. Experimental results based on the HHI MPEG-7 face database, the AR face database, and the CMU pose, illumination, and expression (PIE) database show that this face detection algorithm is efficient and reliable under different lighting conditions and facial expressions.

Map images are composed of semantic layers depicted in arbitrary color. Color separation is often needed to divide the image into layers for storage and processing. Separation can result in severe artifacts because of the overlapping of the layers. In this work, we introduce a technique to restore the original semantic layers after the color separation. The proposed restoration technique improves compression performance of the reconstructed layers in comparison to the corrupted ones when compressed by lossless algorithms such as International Communication Unit (ITU) Group 4 (TIFF G4), Portable Network Graphics (PNG), Joint Bi-level Image experts Group (JBIG), and context tree method. The resulting technique also provides good visual quality of the reconstructed image layers, and can therefore be applied for selective layer removal/extraction in other map processing applications, e.g., area measurement.

We present an image processing algorithm customized for high-speed, real-time inspection of pavement cracking. In the algorithm, a pavement image is divided into grid cells of 8×8 pixels, and each cell is classified as a noncrack or crack cell using the grayscale information of the border pixels. Whether a crack cell can be regarded as a basic element (or seed) depends on its contrast to the neighboring cells. A number of crack seeds can be called a crack cluster if they fall on a linear string. A crack cluster corresponds to a dark strip in the original image that may or may not be a section of a real crack. Additional conditions to verify a crack cluster include the requirements in the contrast, width, and length of the strip. If verified crack clusters are oriented in similar directions, they will be joined to become one crack. Because many operations are performed on crack seeds rather than on the original image, crack detection can be executed simultaneously when the frame grabber is forming a new image, permitting real-time, online pavement surveys. The trial test results show a good repeatability and accuracy when multiple surveys were conducted at different driving conditions.

We present a classification work performed on industrial parts using artificial vision, a support vector machine (SVM), boosting, and a combination of classifiers. The object to be controlled is a coated heater used in television sets. Our project consists of detecting anomalies under manufacturer production, as well as in classifying the anomalies among 20 listed categories. Manufacturer specifications require a minimum of ten inspections per second without a decrease in the quality of the produced parts. This problem is addressed by using a classification system relying on real-time machine vision. To fulfill both real-time and quality constraints, three classification algorithms and a tree-based classification method are compared. The first one, hyperrectangle based, proves to be well adapted for real-time constraints. The second one is based on the Adaboost algorithm, and the third one, based on SVM, has a better power of generalization. Finally, a decision tree allowing improving classification performances is presented.

We propose an approach using artificial neural networks to classify masses in mammograms as malignant or benign. Single-layer and multilayer perceptron networks are used in a study on perceptron topologies and training procedures for pattern classification of breast masses. The contours of a set of 111 regions on mammograms related to breast masses and tumors are manually delineated and represented by polygonal models for shape analysis. Ribbons of pixels are extracted around the boundaries of a subset of 57 masses by dilating and eroding the contours. Three shape factors, three measures of edge sharpness, and 14 texture features based on gray-level co-occurrence matrices of the pixels in the ribbons are computed. Several combinations of the features are used with perceptrons of varying topology and training procedures for the classification of benign masses and malignant tumors. The results are compared in terms of the area Az under the receiver operating characteristics curve. Values of Az up to 0.99 are obtained with the shape factors and texture features. However, only feature sets that included at least one shape factor provide consistently high performance with respect to variations in network topology and training.

An application of an unsupervised self-organizing neural network—the minimal free energy vector quantization neural network—is reported for the detection and characterization of breast lesions in dynamic contrast-enhanced magnetic resonance imaging (MRI). This technique enables the extraction of spatial and temporal features of dynamic MRI data stemming from patients with confirmed lesion diagnosis. By revealing regional properties of contrast-agent uptake, characterized by subtle differences of signal amplitude and dynamics, this method provides both a set of prototypical time series and a corresponding set of cluster assignment maps, which further provides a segmentation with regard to identification and regional subclassification of pathological breast tissue lesions. We present three different segmentation methods for the evaluation of signal intensity time courses for the differential diagnosis of enhancing lesions in breast MRI. Starting from the conventional methodology, we proceed by introducing the separate concepts of threshold segmentation and cluster analysis based on the minimal free energy vector quantization neural network, and in the last step by combining those two concepts. The results suggest that the minimal free energy vector quantization neural network has the potential to increase the diagnostic accuracy of MRI mammography by improving sensitivity without reduction of specificity.

A newly developed grammar-based lossless source coding theory and its implementation was proposed in 1999 and 2000, respectively, by Yang and Kieffer. The code first transforms the original data sequence into an irreducible context-free grammar, which is then compressed using arithmetic coding. In the study of grammar-based coding for mammography applications, we encountered two issues: processing time and limited number of single-character grammar G variables. For the first issue, we discover a feature that can simplify the matching subsequence search in the irreducible grammar transform process. Using this discovery, an extended grammar code technique is proposed and the processing time of the grammar code can be significantly reduced. For the second issue, we propose to use double-character symbols to increase the number of grammar variables. Under the condition that all the G variables have the same probability of being used, our analysis shows that the double- and single-character approaches have the same compression rates. By using the methods proposed, we show that the grammar code can outperform three other schemes: Lempel-Ziv-Welch (LZW), arithmetic, and Huffman on compression ratio, and has similar error tolerance capabilities as LZW coding under similar circumstances.