Most of the methods surveyed for detecting moving objects through image processing in general purpose processors (GPPs) presented an approach that did not allow successful implementation in field programmable gate array (FPGA). The correct classification of the pixels in these methods is directly related to the need for many resources in terms of memory due to the complex mechanism to get the reference image. Additionally, almost all the methods analyzed used only conventional techniques of digital image processing (DIP). Thus, we propose an approach that combines conventional techniques of DIP with fuzzy integral in a parallel processing aiming to improve the detection of moving vehicles. Hence, the proposed decision-making system seeks not to compromise results in terms of pixel classification even by implementing a simple modeling mechanism. Results in terms of occupied resource, maximum operating frequency, and pixel classification are verified in low-cost FPGA-based implementation. The proposed system processes the images in real-time and presents better pixel classification than just fuzzy decision-making.

Underwater images suffer from various degradation factors, such as blur, haze, color degradation, and marine snow. Marine snow is a type of noise, caused mostly by biological particles that fall into the ocean bottom, and which impedes proper object detection in underwater vision. A method for real-time marine snow removal from underwater color and monochrome video is presented. It is based on the proposed marine snow model, spatiotemporal patch analysis, and three-dimensional median filtering. The method was evaluated on a number of real underwater sequences endowed with the hand-annotated ground-truth data which were made available from the Internet. As shown by the experiments, the method attains high accuracy and performs in real time.

Illumination estimation is an important part of color constancy. It is known that estimating the scene illumination color is an ill-posed problem, and no method can be considered as universal. To select an optimal method for a particular image or scene, some combination methods based on characteristic similarity were proposed. These methods found that the images having similar characteristics can use the same method as an optimal technique. Although the combination methods based on image characteristics have worked well in some scenes, the accuracy of these methods is limited to the accuracy of the set of the unitary method. However, we assume that images with similar characteristics have similar scene illumination color and propose an illumination estimation method which is based on image characteristics. According to the characteristics of each image, we search K images whose characteristics are similar to the image from the image dataset, and the standard illumination color of the K selected images is known. Then, we use the weighted average method to estimate the new image’s illumination color by combining the standard illumination color of the K selected images. The experimental results show that the proposed method outperforms other state-of-the-art methods.

A low-resolution depth map can be upsampled through the guidance from the registered high-resolution color image. This type of method is so-called guided depth map upsampling. Among the existing methods based on Markov random field (MRF), either data-driven or model-based prior is adopted to construct the regularization term. The data-driven prior can implicitly reveal the relation between color-depth image pair by training on external data. The model-based prior provides the anisotropic smoothness constraint guided by high-resolution color image. These types of priors can complement each other to solve the ambiguity in guided depth map upsampling. An MRF-based approach is proposed that takes both of them into account to regularize the depth map. Based on analysis sparse coding, the data-driven prior is defined by joint cosparsity on the vectors transformed from color-depth patches using the pair of learned operators. It is based on the assumption that the cosupports of such bimodal image structures computed by the operators are aligned. The edge inconsistency measurement is explicitly calculated, which is embedded into the model-based prior. It can significantly mitigate texture-copying artifacts. The experimental results on Middlebury datasets demonstrate the validity of the proposed method that outperforms seven state-of-the-art approaches.

We propose a random-forest scheme, namely joint maximum purity forest (JMPF), for classification, clustering, and regression tasks. In the JMPF scheme, the original feature space is transformed into a compactly preclustered feature space, via a trained rotation matrix. The rotation matrix is obtained through an iterative quantization process, where the input data, inclined to different classes, is clustered to the respective vertices of the feature space with maximum purity. In the feature space, orthogonal hyperplanes, which are employed at the split nodes of the decision trees in a random forest, can effectively tackle the clustering problems. We evaluated our proposed method on public benchmark datasets for regression and classification tasks, and experiments showed that JMPF remarkably outperforms other state-of-the-art random-forest-based approaches. Furthermore, we applied JMPF to image super-resolution (SR) specifically because the transformed, compact features are more discriminative to the clustering-regression scheme. Experimental results on several public datasets also showed that the JMPF-based image SR scheme is consistently superior to recent state-of-the-art image SR algorithms.

The paper presents an original hybrid image reconstruction algorithm ART-TVS for few-views computed tomography of strongly absorbing media. It is based on the well-known algebraic reconstruction technique (ART), regularization of interim results through minimization of the total variation norm (TV-regularization), and a method of adaptive segmentation, which is a modernization of the known region growing algorithm. It is shown that the ART-TVS algorithm does not give stripe artifacts even if the number of views is very small (eight or less). ART-TVS reconstruction results for two numerical models of metal shells are compared with those obtained with the ART-TV algorithm (ART with TV-regularization and without adaptive segmentation), the iterative Potts minimization algorithm (IPMA), and our MART-AP algorithm (multiplicative ART with a priori information) we developed earlier for few-views discrete tomography. It is shown that ART-TVS outperforms ART-TV and IPMA and is comparable with MART-AP in reconstruction accuracy. Also, ART-TVS converges markedly faster than IRMA in cases where strongly underdetermined systems are treated. The algorithm we propose also demonstrates quite satisfactory resistance to projection data noise that is inherent in tomography of strongly absorbing media.

Convolutional neural networks have been recently shown to have the highest accuracy for single-image super-resolution reconstruction. A multibranch prediction and selection network that can gradually reconstruct robust images in multiple scales is proposed. This endeavor is achieved through a network structure with two interacting subnetworks: one is a deep cascaded, multibranch prediction network (DCMBPN) and another is a deep block-selection network (DBSN). In particular, in each cascade, DCMBPN predicts multiple reconstructed images progressively with its special multibranch and cascaded structure. DBSN then adaptively selects the predicted confident blocks from these reconstructed images. Our method does not require traditional interpolation methods to upsample the image as a preprocessing step. It, thus, greatly reduces the computational complexity. We use Euclidean and perception loss functions in each branch to obtain two high-quality reconstructions. In addition, for the cascade structure, our network can achieve reconstructions in different scales, such as 1.5  ×  , 2  ×  , 2.5  ×  , 3  ×  , 3.5  ×  , and 4  ×  . Extensive quantitative and qualitative evaluations on benchmark datasets show that the proposed algorithm performs favorably against the state-of-the-art methods in terms of accuracy and visual improvement.

Designing effective local descriptors is crucial for many computer vision tasks such as image matching and patch verification. We propose a convolutional neural network (CNN)-based local descriptor named DHNet with a considerate sampling strategy and a dedicated loss function. By considerate sampling, both the closest nonmatching sample and the farther matching sample can be obtained for effectively training a discriminative model. In addition, an improved triplet loss is designed by adding a constraint that limits the absolute distance for the closest nonmatching pair. Based on hard samples and the constraint, our lightweight CNN can quickly generate local descriptors with enhanced intraclass compactness and interclass separation. Experimental results show that our method significantly outperforms the state-of-the-art methods in terms of strong discrimination ability, as evidenced by a considerable performance improvement on several benchmarks.

Visual saliency is used to guide the coding tree unit (CTU)-level bit allocation process in high-efficiency video coding (HEVC) to improve the visual quality. At first, a saliency detection algorithm is proposed. With the detected saliency map, the distortion of each CTU is weighted by the corresponding saliency, so that the distortion of the salient areas (SAs) is more critical. Then, the optimal bit allocation problem constrained by the picture-level target bits and minimum weighted distortion is built. Numerical method is used to solve the bit allocation problem. Finally, bits reallocation and the rate distortion optimization parameters clipping methods are presented to tackle the bits error accumulation problem. Experiment results show that quality gaining in SAs is up to 0.8658 dB and the gaining of saliency weighted peak signal-to-noise ratio is up to 1.0318 dB.

A learning-based multibranch deep probability network is proposed to estimate the illuminated color of the light source in a scene, commonly referred to as color constancy. The method consists of two coupled subnetworks, which are the deep multibranch illumination estimating network (DMBEN) and deep probability computing network (DPN). The one branch of DMBEN estimates the global illuminant through pooling layer and fully connected layer, whereas the other branch is built as an end-to-end residuals network (Res-net) to evaluate the local illumination. The other adjoint subnetwork DPN separately computes the probabilities that results of DMBEN are similar to the ground truth, then determines the better estimation according to the two probabilities under a new criterion. The results of extensive experiments on Color Checker and NUS 8-Camera datasets show that the proposed approach is superior to the state-of-the-art methods both in efficiency and effectiveness.

Many existing methods effectively remove impulse noise from images, but they usually perform well only in a certain noise level (i.e., either low or high contamination). An impulse noise detection technique that is based on mean shift filtering and an effective vector filtering method that is based on channel suppression processing are proposed. The proposed denoising method excellently suppresses impulse noise in color images and performs well in both low and medium noise contamination cases. First, the noise detection method is performed to divide image pixels into noisy and possibly noise-free ones. Then, for the noisy pixels detected, according to the contamination levels, different denoising strategies are employed. For slightly corrupted images, the proposed channel-suppressed vector filter is performed, whereas for medially and highly contaminated images, a total variation denoising technique is applied, followed by a further impulse noise detection and channel-suppressed filtering. Finally, for the possible noise-free pixels, fine noise detection and channel-suppressed filtering are performed. Extensive simulation results exhibit the validity of the proposed solution by showing clear performance improvements over other widely used color image filtering methods.

Two-dimensional discriminant locality preserving projection (2DDLPP) is an effective method for image feature extraction. However, original 2DDLPP is based solely on the Euclidean distance, which is sensitive to noises and illumination changes in images. To overcome this drawback, we propose a method named nuclear norm-based two-dimensional discriminant locality preserving projection (NN2DDLPP). In NN2DDLPP, two optimal neighbor graphs are first built. Then the nuclear norm-based between-class scatter and within-class scatter are defined. Finally, in order to obtain an optimal projection matrix, the ratio of between-class scatter to within-class scatter is maximized. Using nuclear norm metric and labeled information, NN2DDLPP can both efficiently extract the discriminative features and improve the robustness to illumination changes and noises. Experiments carried out on several different face image databases validate that NN2DDLPP is efficacious for face recognition and better than other related works.

The second version of the high-efficiency video coding standard introduced a scalable extension, which is called scalable high-efficiency video coding. It is necessary to design appropriate rate controllers for this extension based on the specific application requirements. According to delay and buffering constraints, rate control algorithms are divided into constant bit rate and variable bit rate (VBR) algorithms. Our focus is on high-delay applications like streaming and broadcast for which VBR controllers are suitable. Our proposed VBR controller utilizes a fuzzy logic-based algorithm to control long-term average bit rate while preventing buffer overflow or underflow. Targeting perceptual quality, the proposed algorithm has been implemented at the group of pictures level to minimize unnecessary fluctuations of quantization parameter and consequently minimizing video quality fluctuations. Experimental results show a high performance in terms of rate control and quality of the compressed video for the proposed algorithm.

This paper proposes a saliency detection method based on the aggregated Wasserstein distance. A multidimensional Gaussian mixture model is used to model the superpixels, whereby the color information of different color spaces is combined. To overcome the lack of the closed-form solution for the Gaussian mixture model, we employ the aggregated Wasserstein distance to measure the perceptual similarity between different superpixels. The saliency value is then calculated from two aspects. First, the global saliency is computed through all the superpixels in the image using the aggregated Wasserstein distance. Second, the local saliency is computed in a lower range with the same measure. Finally, a saliency map is obtained by combining the two types of saliencies, and is filtered by spectral clustering all the superpixels. The experimental results show that the proposed method outperforms 11 recent exact algorithms on three widely used open datasets.

The detection of foreign substances in transparent-bottled liquid is a key procedure in production. An improved algorithm based on the combination of guided filter and visual background extractor (ViBe) is proposed for the detection of foreign substances in transparent-bottled turbid liquid on the market. In the stage of image preprocessing, the image enhancement is carried out by the guided filter and then the liquid region is detected by the Hough transform. Finally, the area of the connected region is determined for the image processed by the ViBe algorithm to remove the tiny noise in detection images. The simulation results show that the proposed algorithm can not only detect foreign substances in transparent liquid but also detect foreign substances in the turbid liquid.

Perceptual quality assessment of a three-dimensional (3-D) image is one of the most important tasks in various applications such as 3-D image coding, processing, enhancement, and monitoring system. But objective quality assessment of 3-D images is still a challenging task. Especially, blind quality assessment of 3-D images encounters an arduous challenge due to lack of prior information about the original images. To solve this problem, we propose a blind 3-D images’ quality evaluator by simulating binocular rivalry and orientation responses of the human visual system. As a main technical contribution of this research, both the low- and high-level binocular rivalry responses (BRR), as well as binocular orientation-tuned (BOT) responses, are considered for blind quality assessment of 3-D images. Specifically, the self-similarity of the BRR and BOT responses is extracted from the distorted 3-D images, which will change in the presence of distortions. Subsequently, all quality-aware features are mapped to subjective quality scores of the distorted 3-D images via using support vector regression. The performance of our algorithm is evaluated over two popular LIVE 3-D phase I and phase II databases and shown to be competitive with the state-of-the-art algorithms.