Some classical filters, such as bilateral filter, nonlocal means (NLM) filter, and locally adaptive regression kernel, have proven to have a good performance in image denoising. However, there is one shortcoming, i.e., they cannot control the denoising strength very well. As for synthetic aperture radar (SAR) images, due to the special multiplicative noise, the denoising process becomes more complicated. A diffusion iterative filter can enhance the performance of a kernel in the denoising process but will lose some latent details and important targets from the underlying SAR image. In contrast, an iterative boosting filter can preserve these latent details of the SAR image well, although the improvement of the kernel performance is not very desired. By adopting the advantages of diffusion and boosting of the two iterative methods, an adaptive iterative risk estimator minimum mean square error (Min-MSE) method is proposed, which is mainly based on the Min-MSE to adaptively get the optimal iterative method and the corresponding optimal iterative number. The analysis of the experimental results and the comparison with some other state of the art methods demonstrate that our proposed method can improve the performance of an NLM filter and effectively suppress the SAR image speckle.

Three-dimensional (3-D) visual comfort assessment (VCA) is a particularly important and challenging topic, which involves automatically predicting the degree of visual comfort in line with human subjective judgment. State-of-the-art VCA models typically focus on minimizing the distance between predicted visual comfort scores and subjective mean opinion scores (MOSs) by training a regression model. However, obtaining precise MOSs is often expensive and time-consuming, which greatly constrains the extension of existing MOS-aware VCA models. This study is inspired by the fact that humans tend to conduct a preference judgment between two stereoscopic images in terms of visual comfort. We propose to train a robust VCA model on a set of preference labels instead of MOSs. The preference label, representing the relative visual comfort of preference stereoscopic image pairs (PSIPs), is generally precise and can be obtained at much lower cost compared with MOS. More specifically, some representative stereoscopic images are first selected to generate the PSIP training set. Then, we use a support vector machine to learn a preference classification model by taking a differential feature vector and the corresponding preference label of each PSIP as input. Finally, given a testing sample, by considering a full-round paired comparison with all the selected representative stereoscopic images, the visual comfort score can be estimated via a simple linear mapping strategy. Experimental results on our newly built 3-D image database demonstrate that the proposed method can achieve a better performance compared with the models trained on MOSs.

Hyperspectral imaging (HSI) based sensing devices were utilized to develop nondestructive, rapid, and low cost analytical strategies finalized to detect and characterize materials constituting demolition waste. In detail, HSI was applied for quality control of high-grade recycled aggregates obtained from end-of-life concrete. The described HSI quality control approach is based on the utilization of a platform working in the near-infrared range (1000–1700 nm). The acquired hyperspectral images were analyzed by applying different chemometric methods: principal component analysis for data exploration and partial least-square-discriminant analysis to build classification models. Results showed that it is possible to recognize the recycled aggregates from different contaminants (e.g., brick, gypsum, plastic, wood, foam, and so on), allowing the quality control of the recycled flow stream.

Texture and color are important features in analyzing images of natural scenes. Fractal descriptors based on quaternion Fourier transform and local polynomial regression are proposed for color texture image analysis. First, considering the relation between the power spectrum and frequency in the quaternion Fourier transform domain, we proposed fractal dimensions of a color image using a fast quaternion Fourier transform. Second, local polynomial regression is applied to estimate log(spectrum)–log(frequency) curve, which is not usually linear in a natural texture image. Finally, a local polynomial regression curve is defined as fractal descriptors for the color image classification problem with multiclasses. The experimental results show that our proposed approach is more effective than other color texture analysis methods both in the correct classification rate and the duration.

The increasing demand to develop a palmprint biometric system with a low-error rate has prompted scientists to use multispectral imaging to overcome the limits of the techniques that act in visible light. In order to improve the accuracy of multispectral palmprint recognition, we explore two level fusions: pixel and the feature level fusion approaches. The former is based on a maximum selection rule, which combines discriminating information from different spectral bands of discrete wavelet transform of multispectral images. The latter operates the fusion of features extracted from subimages. We propose to use both approaches for statistical and energy distribution analysis of the finite ridgelet transform coefficients, for the sake of their simplicity and low-computational complexity. Once the feature vectors are obtained, we perform a robust classification to identify/verify individuals with both approaches. The effectiveness of the proposed methods is evaluated on several classifiers for binary and multiclass cases. The experimental results conducted on Chinese Academy of Sciences Institute of Automation and Hong Kong Polytechnic University databases show that the proposed approaches ensure, respectively, an accuracy rate of 100% and 99.79%. A comparative study has revealed that our approach outperforms or at least equals the performances of the state-of-the-art multispectral palmprint recognition methods.

A method for palmprint verification and identification, which is robust to blurred and occluded palmprints is proposed. An algorithm is proposed for region of interest (ROI) extraction that aligns each palmprint ROI on the same position. First-level decomposition of the ROI using a Haar wavelet gives an approximation ROI (AROI) and reduces computational overhead. Local phase quantization (LPQ) based on quantization of phase information by local Fourier transform is applied on AROI to obtain blur invariant palmprint features. The LPQ image is divided into M×M nonoverlapping blocks, and histograms of these blocks are considered as features. The entropy of each block is used to identify it as occluded or nonoccluded. The average chi-square distance between corresponding nonoccluded blocks is calculated for measuring similarity. Experiments performed on PolyU 2D, CASIA, IITD, and IIITDMJ palmprint databases show that the proposed system is independent of acquisition device. Moreover, to justify the robustness of the proposed system, experiments are performed with variable amounts of blur and occlusion present in test palmprints. The performance of the proposed system is compared with several local feature-based palmprint recognition techniques on normal as well as occluded palmprints.

Efficient registration across pose is the most challenging research area for accurate recognition of human face images. Authors have discussed a tool that has been developed for registration of face images across poses using the nose tip of the face images. The nose tip has been considered here because of its stability in situations such as variations in pose, expression, etc. The aim of this investigation is to develop a face registration tool called “Register-My-Face” with a working methodology in all the three directions, namely yaw, pitch, and roll. This tool has been developed for three-dimensional (3-D) face registration, which is inspired by analyzing the “depth values” of face range images. The registration of the face is done using a geometrical technique which is based on computing the corresponding rotation in three orthogonal directions. The advantages of the designed tool are that it does not need any training phase for accurate detection of the nose tip, and this method can handle large pose variations, including 90 deg pose variations about the Y-axis in both the positive and negative directions. The method that has been followed to develop this tool is also independent of facial expression, occlusion and illumination variations. Moreover, it quickly detects the nose tip because it does not need to process the entire face surface, but only requires the isolated nose region. The tool has been integrated with three different databases; GavabDB, Bosphorus, and Frav3D, and the investigation highlights the robustness of the tool. Additionally, for exploring the performance of the tool, a SIMULINK model for hardware interface is also developed with a discrete solver and is tested on two different configuration setups and executed in two different execution modes with two simulation stop timings 10.0 and 1.0. This model can proceed according to the algorithm with a minimum of 6.640 s to register an unregistered raw 3-D face scan input image from the Frav3D database. Accuracies of the nose region of 98.87%, 94.44%, and 98.08% for the Frav3D, GavabDB, and Bosphorus databases, respectively, are observed. For nose tip detection, the success rates are 98.91% for the Frav3D database, 98.74% for the GavabDB database, and 96.03% for the Bosphorus database. Based on the success rate of nose tip detection, the registration process is implemented on three databases. Registration accuracy, computed between a neutral and the registered range face image for the Frav3D database is 87.5% for GavabDB and 89.87% for Bosphorus database, and the rate of success is 70.23%.

This paper reports the development of a spectral reconstruction technique for predicting hyperspectral images from RGB color images and classifying food-borne pathogens in agar plates using reconstructed hyperspectral images. The six representative non-O157 Shiga-toxin producing Escherichia coli (STEC) serogroups (O26, O45, O103, O111, O121, and O145) grown on Rainbow agar plates were used for the study. A line-scan pushbroom hyperspectral imaging spectrometer was used to scan full reflectance spectra of pure non-O157 STEC cultures in the visible and near-infrared spectral range from 400 to 1000 nm. RGB color images were generated by simulation from hyperspectral images. Polynomial multivariate least-squares regression analysis was used to reconstruct hyperspectral images from RGB color images. The mean R-squared value for hyperspectral image reconstruction was ∼0.98 in the spectral range between 400 and 700 nm for linear, quadratic, and cubic polynomial regression models. The accuracy of the hyperspectral image classification algorithm based on k-nearest neighbors algorithm of principal component scores was validated to be 92% with the test set (99% with the original hyperspectral images). The results of the study suggested that color-based hyperspectral imaging would be feasible without much loss of prediction accuracy compared to true hyperspectral imaging.

In human-centric technologies, skin segmentation of body parts is a prerequisite for high-level processing. The traditional method of skin detection is pixel-wise detection coupled with morphological operations. Pixel-wise methods usually generate a number of false samples and outlier skin pixels, which can make it difficult for morphological operations to provide satisfactory results in complex scenarios. Furthermore, in many cases only a coarse region is required (e.g., the bounding-box of the face) rather than detailed pixel-wise labeling. A patch-wise skin segmentation method is proposed based on deep neural networks. Our method treats image patches as processing units instead of pixels, which directly exploits the spatial information of pixels in the detection stage rather than using morphological operations on isolated pixels after detection. An image patch dataset is built and deep skin models (DSMs) are trained based on the new dataset. Trained DSMs are then integrated into a sliding window framework to segment skin regions of the human body parts. Experiments on standard benchmarks demonstrate that DSMs provide more explicit skin region of interest candidates than pixel-wise methods in complex scenarios, and achieve competitive performance on pixel-wise skin detection.

In sketch-based image retrieval (SBIR) systems, representing photo-realistic images by their strong edges is an intuitive and effective way to bridge the appearance gap between sketches and photo-realistic images. However, noisy edges and missing edges usually enlarge the appearance gap and significantly degrade retrieval performance. To alleviate the impact, we formalize the matching task between the sketches and extracted edges of photo-realistic images as a partial matching problem. We treat the sketches and extracted edges as a set of line segments which serve as the basis for better shape description and partial matching. We propose a new descriptor, structure point, to represent sketches and the extracted edges. Based on the structure point, a decompose-and-assemble hierarchical matching algorithm is developed to match sketches and extracted edges. Observing that false matches can degrade performance, we introduce a spatial constraint to filter the false matches. We have tested the proposed framework on public datasets and a new dataset of three million images which we recently collected for the SBIR evaluation purpose. We compared our method with the state-of-the-art methods (SHoG and GF-HOG). The experimental results show that our framework significantly improves retrieval performance.

Background subtraction is an important task for various computer vision applications. The task becomes more critical when the background scene contains more variations, such as swaying trees and abruptly changing lighting conditions. Recently, robust principal component analysis (RPCA) has been shown to be a very efficient framework for moving-object detection. However, due to its batch optimization process, high-dimensional data need to be processed. As a result, computational complexity, lack of features, weak performance, real-time processing, and memory issues arise in traditional RPCA-based approaches. To handle these, a background subtraction algorithm robust against global illumination changes via online robust PCA (OR-PCA) using multiple features together with continuous constraints, such as Markov random field (MRF), is presented. OR-PCA with automatic parameter estimation using multiple features improves the background subtraction accuracy and computation time, making it attractive for real-time systems. Moreover, the application of MRF to the foreground mask exploits structural information to improve the segmentation results. In addition, global illumination changes in scenes are tackled by using sum of the difference of similarity measure among features, followed by a parameter update process using a low-rank, multiple features model. Evaluation using challenging datasets demonstrated that the proposed scheme is a top performer for a wide range of complex background scenes.

Iris segmentation is the first and critical step in an iris recognition system. A robust iris segmentation algorithm based on the self-adaptive Chan and Vese (SACV) level set model is proposed. First, the process of constructing the SACV model based on analyses of the corresponding requirements for the CV model is described when it is applied on iris segmentation. Second, the coarse segmentation of pupil and iris, which are localized based on image pixel gray information, is used as the initial contour of the SACV model. Third, the interference factors, such as eyelashes and eyelids, are detected and evaluated simultaneously to generate the interference degree and then the related parameter of SACV is set according to the interference degree. Finally, SACV is used to conduct the final fine segmentation of the pupil and iris. Experiments on four public iris image databases (e.g., CASIA-V1, CASIA-V3 Interval, CASIA-V3 Lamp, and MMU-V1) demonstrate the segmentation accuracy performance of the proposed algorithm, and at the same time, the proposed algorithm also displays robust performance in noisy situations, such as Gaussian, Poisson, salt-and-pepper, and speckle noises. Moreover, comparisons with the well-known methods further show that our algorithm can segment iris images more accurately.

In order to remove the mixed noise of Gaussian noise and impulse noise from video data using sparse and low-rank decomposition algorithms based on S_1/2 norm, a nonlocal video denoising scheme is proposed. By grouping similar patches in both spatial and temporal domains, the problem of removing noise is converted into a sparse and low-rank decomposition problem. Two algorithms were designed to recover the low-rank components from a noisy video based on the related theories of L_1/2 regularization and S_1/2 norm. Algorithm 1 is based on the combination of S_1/2 norm and L_1/2 norm. Algorithm 2 is based on the combination of S_1/2 norm and L₁ norm. The resulting L_1/2 norm and S_1/2 norm related minimization problem can be effectively overcome by some recently developed numerical methods. Through the numerical simulation experiment, the effectiveness and precision of the two algorithms in sparse and low-rank component decomposition problems were verified. Video denoising experimental results show that the proposed scheme can effectively remove the Gaussian noise and random-valued impulse noise from the video. The denoising effect by the two algorithms, no matter the visual quality and objective evaluation criteria, were higher than that of the state-of-the-art video denoising algorithm.

We present a technique of three-dimensional (3-D) differential imaging by the way of incoherent digital holography. We demonstrate that the method is suitable for fluorescence microscopy without the need for unfavorable scanning. We acquire the complex optical field of an incoherent volumetric scene at various times and calculate a complex difference hologram representing only the information, which has changed throughout the volumetric space during the time intervals in-between. We first demonstrate the advanced capability of self-interference incoherent digital holography combined with difference holography to track 3-D changes in a broadband, unfiltered, sunlit scene containing macroscopic continuous objects. This case is particularly remarkable due to the exceptionally short temporal coherence length and excessive build-up of noninterfering source points. We then demonstrate the ease of adaptation to the versatile, functional imaging of fluorescence microscopy.

The process of image quality improvement through super-resolution methods is still a gray area in the field of biometric identification. This paper proposes a scheme for fingerprint super-resolution using ridge orientation-based clustered coupled sparse dictionaries. The training image patches are clustered into groups based on dominant orientation and corresponding coupled subdictionaries are learned for each low- and high-resolution patch groups. While reconstructing the image, the minimum residue error criterion is used for choosing a subdictionary for a particular patch. In the final step, back projection is applied to eliminate the discrepancy in the estimate due to noise or inaccuracy in sparse representation. The performance evaluation of the proposed method is accomplished in terms of peak signal-to-noise ratio and structural similarity index. A filter bank-based fingerprint matcher is used for evaluating the performance of the proposed method in terms of matching accuracy. Our experimental results show that the new method achieves better results in comparison with other methods and will establish itself for improving performances of fingerprint-identification systems.

Binocular stereo matching is one of the core algorithms in stereo vision. The edge-aware filter-based cost aggregation methods can produce precise disparity maps on the famous Middlebury benchmark (indoor). However, they perform poorly on the KITTI vision benchmark (outdoor) because frontal-parallel surfaces are assumed in the filter-based methods. We propose a new cost aggregation algorithm which discards the frontal-parallel surfaces assumption. The proposed algorithm performs like optimizing an energy function via dynamic programming. The proposed energy function integrates the pairwise smooth energy by the edge-aware filtering approach, which makes the proposed method adapt to slanted surfaces. The proposed algorithm not only outperforms the edge-aware filter-based local methods on the Middlebury benchmark but also performs well on the KITTI vision benchmark.

Scene categorization is an important research topic in computer vision. Although various approaches have been proposed to solve this problem, as far as the authors know, there is little work concerning details in scene images. This paper proposes to manipulate details in scene images to improve scene categorization performance. Specifically, a scene image is first partitioned into super-pixels, and these segments are evaluated to see whether their details need to be enhanced or not. Then, image segments are accordingly processed. After that, scale invariant feature transform features are extracted from processed images for image representation and categorization. Extensive experiments are conducted on the 15-class scene dataset and 67-class indoor scene dataset. Experimental results showed that enhancing details in scene images appropriately can effectively improve scene categorization performance. Furthermore, fusing information from details and structures of scene images leads to state-of-the-art results.

Image denoising approaches that learn spatially adaptive dictionaries from the observed noisy image have gathered a lot of attention in the past decade. These methods rely on the hypothesis that patches from the underlying clean image can be expressed as sparse linear combinations of these dictionary vectors (bases). We present a framework for inferring an orthonormal set of dictionary vectors using orthogonal locality preserving projection (OLPP). This ensures that patches that are similar in the noisy image should produce similar coefficients when projected in the OLPP domain. Unlike other projection methods, the locality preserving property of OLPP automatically groups similar patches together during inference of the basis. Hence, only one global orthonormal basis suffices to sparsely represent patches from a large subimage or a large portion of the image. The proposed amalgamation of the sparsity and global dictionary make the current approach more suitable for an image denoising task with reduced computational complexity. Experiments on several benchmark datasets made it clear that the proposed method is capable of preserving fine textures while denoising an image, on par with or surpassing several state-of-the-art methods for gray-scale and color images.

Passive forensics is increasing in significance due to the availability of various software tools that can be used to alter original content without visible traces and the increasing public awareness of such tampering. Many passive image tamper detection techniques have been proposed in the literature, some of which use feature extraction methods for tamper detection and localization. This work proposes a flexible methodology for detecting cloning in images based on the use of feature detectors. We determine whether a particular match is the result of a cloning event by clustering the matches using k-means clustering and using a support vector machine to classify the clusters. This descriptor–agnostic approach allows us to combine the results of multiple feature descriptors, increasing the potential number of keypoints in the cloned region. Results using maximally stable extremal regions’ features, speeded up robust features, and scale-invariant feature transform show a very significant improvement over the state of the art, particularly when different descriptors are combined. A statistical filtering step is also proposed, increasing the homogeneity of the clusters and thereby improving the results. Finally, our methodology uses an adaptive technique for independently selecting the optimal k value for each image, allowing our method to work well when there are multiple cloned regions. We also show that our methodology works well when the training and testing datasets are mismatched.

In image deblurring problems, both local and nonlocal regularization priors are well studied. Local regularization prior assumes piecewise smoothness and transform-based sparsity, while the nonlocal one exploits self-similarity of images. We proposed a mixed regularization model which incorporates the advantages of both local adaptive sparsity prior and nonlocal sparsity prior resulting from the nonlocal self-similarity, and thus encourages a solution to simultaneously express both the local and nonlocal natures of images. The deblurring problem with mixed regularization can be transformed into a constrained optimization problem with separable structure via the variable splitting. Then this constrained optimization problem is solved by the alternating direction method of multipliers. Experimental results with a set of images under varying conditions demonstrate that the proposed method achieves the state-of-the-art deblurring performance.

The problem of human pose estimation in still images is considered. Most previous works predicted the pose directly with either local deformable models or a global mixture representation in the pose space. We argue that this process of pose estimation can be divided into different stages. We propose a new two-stage framework for human pose estimation. In the pre-estimation stage, there are three steps: upper body detection, model category estimation for the upper body, and full model selection for pose estimation. A new method based on pairwise scores of the upper body is proposed for upper body detection. In the estimation stage, we address the problem of a variety of human poses and activities. The upper body-based multiple mixture parts (MMP) model is proposed. This model not only joins different mixture models together, but can also analyze activities with complex kinematic structures. The model is compared with the state-of-the-art. The experimental results demonstrate the effectiveness of the MMP model.