The present special section on Video Surveillance and Transportation Imaging was motivated by ubiquity of video surveillance in our everyday environment and transportation venues. With the advent of low-cost/high-performance video sensors, imaging platforms, and computational equipment, it has become increasingly possible to process video streams in real-time on affordable cloud computational servers, desktop systems, and various hand-held mobile devices. Major applications of these technologies span the spectrum from surveillance, transportation, remote sensing, social media, sports, retail, and biomedical to name a few.

We design and implement a robust vehicle classification system based on pan-tilt-zoom cameras. We introduce a simple but effective camera-invariant feature to describe the intrinsic shape patterns of vehicles. The introduced feature can be directly extracted from two-dimensional images, eliminating the need for complicated three-dimensional template fitting used in existing vehicle classification systems. Also, we introduce a prevalent sparse model to make the discriminative learning procedures robust to noise. Experimental results on practical highways show that the proposed system could achieve promising results on vehicle classification in real time.

The large volume of vehicles on the road has created new challenges for agencies responsible for law enforcement and public safety. Such agencies utilize visual surveillance technology to assist monitoring of vehicles from a remote location. These surveillance systems typically require trained human operators. Consequently, they are prone to human errors due to fatigue or diverted attention caused by excess information. Thus a need exists for an automated system that can analyze the surveillance videos and extract important information. This information would be used to detect occurrence of “anomalous” events. We propose a visual surveillance system designed to function in the above-mentioned manner. The system observes vehicular traffic from a standoff range and extracts information about the vehicles. This information includes vehicle type, make, tire size, and its trajectory. Based on this information, the system checks for anomalies in the vehicles’ appearance and/or motion. We describe analysis methods for obtaining the vehicle information from two cameras placed in an orthogonal configuration and for classifying the vehicles using these observations. We present the results of applying these methods on traffic videos. Our proposed system can be deployed for traffic monitoring (on highways/intersections) or infrastructure protection (at check points).

Damaged tires cause many fatal car accidents that claim lives worldwide. The use of a regular digital camera that can generate JPEG images, so as to become aware of damaged tires, is proposed. The digital camera repeatedly takes pictures of the car’s tires. When there is a drastic change in a particular block of JPEG, the value of many of its frequency coefficients will be high and it will be compressed into many more bits; therefore, if the image is overly large, an embedded computer system will turn on a noticeable light-emitting diode in the car. The driver will be aware of the damaged tire before a severe accident would possibly happen.

A new context-aware scale-invariant feature transform (CASIFT) approach is proposed, which is designed for the use in traffic sign recognition (TSR) systems. The following issues remain in previous works in which SIFT is used for matching or recognition: (1) SIFT is unable to provide color information; (2) SIFT only focuses on local features while ignoring the distribution of global shapes; (3) the template with the maximum number of matching points selected as the final result is instable, especially for images with simple patterns; and (4) SIFT is liable to result in errors when different images share the same local features. In order to resolve these problems, a new CASIFT approach is proposed. The contributions of the work are as follows: (1) color angular patterns are used to provide the color distinguishing information; (2) a CASIFT which effectively combines local and global information is proposed; and (3) a method for computing the similarity between two images is proposed, which focuses on the distribution of the matching points, rather than using the traditional SIFT approach of selecting the template with maximum number of matching points as the final result. The proposed approach is particularly effective in dealing with traffic signs which have rich colors and varied global shape distribution. Experiments are performed to validate the effectiveness of the proposed approach in TSR systems, and the experimental results are satisfying even for images containing traffic signs that have been rotated, damaged, altered in color, have undergone affine transformations, or images which were photographed under different weather or illumination conditions.

We propose a supervised approach to detect falls in a home environment using an optimized descriptor adapted to real-time tasks. We introduce a realistic dataset of 222 videos, a new metric allowing evaluation of fall detection performance in a video stream, and an automatically optimized set of spatio-temporal descriptors which fed a supervised classifier. We build the initial spatio-temporal descriptor named STHF using several combinations of transformations of geometrical features (height and width of human body bounding box, the user’s trajectory with her/his orientation, projection histograms, and moments of orders 0, 1, and 2). We study the combinations of usual transformations of the features (Fourier transform, wavelet transform, first and second derivatives), and we show experimentally that it is possible to achieve high performance using support vector machine and Adaboost classifiers. Automatic feature selection allows to show that the best tradeoff between classification performance and processing time is obtained by combining the original low-level features with their first derivative. Hence, we evaluate the robustness of the fall detection regarding location changes. We propose a realistic and pragmatic protocol that enables performance to be improved by updating the training in the current location with normal activities records.

A low-complex, low bit rate, energy-efficient image compression algorithm explicitly designed for resource-constrained visual sensor networks applied for surveillance, battle field, habitat monitoring, etc. is presented, where voluminous amount of image data has to be communicated over a bandwidth-limited wireless medium. The proposed method overcomes the energy limitation of individual nodes and is investigated in terms of image quality, entropy, processing time, overall energy consumption, and system lifetime. This algorithm is highly energy efficient and extremely fast since it applies energy-aware zonal binary discrete cosine transform (DCT) that computes only the few required significant coefficients and codes them using enhanced complementary Golomb Rice code without using any floating point operations. Experiments are performed using the Atmel Atmega128 and MSP430 processors to measure the resultant energy savings. Simulation results show that the proposed energy-aware fast zonal transform consumes only 0.3% of energy needed by conventional DCT. This algorithm consumes only 6% of energy needed by Independent JPEG Group (fast) version, and it suits for embedded systems requiring low power consumption. The proposed scheme is unique since it significantly enhances the lifetime of the camera sensor node and the network without any need for distributed processing as was traditionally required in existing algorithms.

Recently, visual tracking has been formulated as a classification problem whose task is to detect the object from the scene with a binary classifier. Boosting based online feature selection methods, which adopt the classifier to appearance changes by choosing the most discriminative features, have been demonstrated to be effective for visual tracking. A major problem of such online feature selection methods is that an inaccurate classifier may give imprecise tracking windows. Tracking error accumulates when the tracker trains the classifier with misaligned samples and finally leads to drifting. Separability-maximum boosting (SMBoost), an alternative form of AdaBoost which characterizes the separability between the object and the scene by their means and covariance matrices, is proposed. SMBoost only needs the means and covariance matrices during training and can be easily adopted to online learning problems by estimating the statistics incrementally. Experiment on UCI machine learning datasets shows that SMBoost is as accurate as offline AdaBoost, and significantly outperforms Oza’s online boosting. Accurate classifier stabilizes the tracker on challenging video sequences. Empirical results also demonstrate improvements in term of tracking precision and speed, comparing ours to those state-of-the-art ones.

Urban parking management is receiving significant attention due to its potential to reduce traffic congestion, fuel consumption, and emissions. Real-time parking occupancy detection is a critical component of on-street parking management systems, where occupancy information is relayed to drivers via smart phone apps, radio, Internet, on-road signs, or global positioning system auxiliary signals. Video-based parking occupancy detection systems can provide a cost-effective solution to the sensing task while providing additional functionality for traffic law enforcement and surveillance. We present a video-based on-street parking occupancy detection system that can operate in real time. Our system accounts for the inherent challenges that exist in on-street parking settings, including illumination changes, rain, shadows, occlusions, and camera motion. Our method utilizes several components from video processing and computer vision for motion detection, background subtraction, and vehicle detection. We also present three traffic law enforcement applications: parking angle violation detection, parking boundary violation detection, and exclusion zone violation detection, which can be integrated into the parking occupancy cameras as a value-added option. Our experimental results show that the proposed parking occupancy detection method performs in real-time at 5  frames/s and achieves better than 90% detection accuracy across several days of videos captured in a busy street block under various weather conditions such as sunny, cloudy, and rainy, among others.

We present an adaptive feature extraction and annotation algorithm for articulating traffic events from surveillance cameras. We use approximate median filter for moving object detection, motion energy image and convex hull for lane detection, and adaptive proportion models for vehicle classification. It is found that our approach outperforms three-dimensional modeling and scale-independent feature transformation algorithms in terms of robustness. The multiresolution-based video codec algorithm enables a quality-of-service-aware video streaming according to the data traffic. Furthermore, our empirical data shows that it is feasible to use the metadata to facilitate the real-time communication between an infrastructure and a vehicle for safer and more efficient traffic control.

A new superresolution algorithm is proposed to reconstruct a high-resolution license plate image from a set of low-resolution camera images. The reconstruction methodology is based on the discrete algebraic reconstruction technique (DART), a recently developed reconstruction method. While DART has already been successfully applied in tomographic imaging, it has not yet been transferred to the field of camera imaging. DART is introduced for camera imaging through a demonstration of how prior knowledge of the colors of the license plate can be directly exploited during the reconstruction of a high-resolution image from a set of low-resolution images. Simulation experiments show that DART can reconstruct images with superior quality compared to conventional reconstruction methods.

Visual traffic surveillance using computer vision techniques can be noninvasive, automated, and cost effective. Traffic surveillance systems with the ability to detect, count, and classify vehicles can be employed in gathering traffic statistics and achieving better traffic control in intelligent transportation systems. However, vehicle classification poses a difficult problem as vehicles have high intraclass variation and relatively low interclass variation. Five different object recognition techniques are investigated: principal component analysis (PCA)+difference from vehicle space, PCA+difference in vehicle space, PCA+support vector machine, linear discriminant analysis, and constellation-based modeling applied to the problem of vehicle classification. Three of the techniques that performed well were incorporated into a unified traffic surveillance system for online classification of vehicles, which uses tracking results to improve the classification accuracy. To evaluate the accuracy of the system, 31 min of traffic video containing multilane traffic intersection was processed. It was possible to achieve classification accuracy as high as 90.49% while classifying correctly tracked vehicles into four classes: cars, SUVs/vans, pickup trucks, and buses/semis. While processing a video, our system also recorded important traffic parameters such as the appearance, speed, trajectory of a vehicle, etc. This information was later used in a search assistant tool to find interesting traffic events.

Recent emerging trends for automatic behavior analysis and understanding from infrastructure video are reviewed. Research has shifted from high-resolution estimation of vehicle state and instead, pushed machine learning approaches to extract meaningful patterns in aggregates in an unsupervised fashion. These patterns represent priors on observable motion, which can be utilized to describe a scene, answer behavior questions such as where is a vehicle going, how many vehicles are performing the same action, and to detect an abnormal event. The review focuses on two main methods for scene description, trajectory clustering and topic modeling. Example applications that utilize the behavioral modeling techniques are also presented. In addition, the most popular public datasets for behavioral analysis are presented. Discussion and comment on future directions in the field are also provided.

In port surveillance, video-based monitoring is a valuable supplement to a radar system by helping to detect smaller ships in the shadow of a larger ship and with the possibility to detect nonmetal ships. Therefore, automatic video-based ship detection is an important research area for security control in port regions. An approach that automatically detects moving ships in port surveillance videos with robustness for occlusions is presented. In our approach, important elements from the visual, spatial, and temporal features of the scene are used to create a model of the contextual information and perform a motion saliency analysis. We model the context of the scene by first segmenting the video frame and contextually labeling the segments, such as water, vegetation, etc. Then, based on the assumption that each object has its own motion, labeled segments are merged into individual semantic regions even when occlusions occur. The context is finally modeled to help locating the candidate ships by exploring semantic relations between ships and context, spatial adjacency and size constraints of different regions. Additionally, we assume that the ship moves with a significant speed compared to its surroundings. As a result, ships are detected by checking motion saliency for candidate ships according to the predefined criteria. We compare this approach with the conventional technique for object classification based on support vector machine. Experiments are carried out with real-life surveillance videos, where the obtained results outperform two recent algorithms and show the accuracy and robustness of the proposed ship detection approach. The inherent simplicity of our algorithmic subsystems enables real-time operation of our proposal in embedded video surveillance, such as port surveillance systems based on moving, nonstatic cameras.

A system that can automatically annotate surveillance video in a manner useful for locating a person with a given description of clothing is presented. Each human is annotated based on two appearance features: primary colors of clothes and the presence of text/logos on clothes. The annotation occurs after a robust foreground extraction stage employing a modified Gaussian mixture model-based approach. The proposed pipeline consists of a preprocessing stage where color appearance of an image is improved using a color constancy algorithm. In order to annotate color information for human clothes, we use the color histogram feature in HSV space and find local maxima to extract dominant colors for different parts of a segmented human object. To detect text/logos on clothes, we begin with the extraction of connected components of enhanced horizontal, vertical, and diagonal edges in the frames. These candidate regions are classified as text or nontext on the basis of their local energy-based shape histogram features. Further, to detect humans, a novel technique has been proposed that uses contourlet transform-based local binary pattern (CLBP) features. In the proposed method, we extract the uniform direction invariant LBP feature descriptor for contourlet transformed high-pass subimages from vertical and diagonal directional bands. In the final stage, extracted CLBP descriptors are classified by a trained support vector machine. Experimental results illustrate the superiority of our method on large-scale surveillance video data.

Video surveillance is used extensively in intelligent transportation systems to enforce laws, collect tolls, and regularize traffic flow. Benefits to society include reduced fuel consumption and emissions, improved safety, and reduced traffic congestion. These video cameras installed at traffic lights, highways, toll booths, etc., continuously capture video and hence generate a vast amount of data that are stored in large databases. The captured video is typically compressed before being transmitted and/or stored. While all the archived information is present in the compressed video, most current applications operate on uncompressed video. The aim is to improve the efficiency of processing by utilizing features of the compression process and the compressed video stream. Key methods that are employed involve intelligent selection of reference frames (I-frames) and exploitation of the compression motion vectors. Although specific applications in the transportation imaging domain are presented, the methods proposed here can generally impact the ability to mine vast amounts of video data for usable information in many diverse settings. Applications presented include rapid search for target vehicles (Amber Alert, Silver Alert, stolen car, etc.), vehicle counting, stop sign/light enforcement, and vehicle speed estimation.

The use of contextual information can significantly aid scene understanding of surveillance video. Just detecting people and tracking them does not provide sufficient information to detect situations that require operator attention. We propose a proof-of-concept system that uses several sources of contextual information to improve scene understanding in surveillance video. The focus is on two scenarios that represent common video surveillance situations, parking lot surveillance and crowd monitoring. In the first scenario, a pan–tilt–zoom (PTZ) camera tracking system is developed for parking lot surveillance. Context is provided by the traffic sign recognition system to localize regular and handicapped parking spot signs as well as license plates. The PTZ algorithm has the ability to selectively detect and track persons based on scene context. In the second scenario, a group analysis algorithm is introduced to detect groups of people. Contextual information is provided by traffic sign recognition and region labeling algorithms and exploited for behavior understanding. In both scenarios, decision engines are used to interpret and classify the output of the subsystems and if necessary raise operator alerts. We show that using context information enables the automated analysis of complicated scenarios that were previously not possible using conventional moving object classification techniques.

Although the traditional bag-of-words model has shown promising results for human action recognition, in the feature coding phase, the ambiguous features from different body parts are still difficult to distinguish. Furthermore, it also suffers from serious representation error. We propose an innovative coding strategy called position and locality constrained soft coding (PLSC) to overcome these limitations. PLSC uses the feature position in a human oriented region of interest (ROI) to distinguish the ambiguous features. We first construct a subdictionary for each feature by selecting the bases from their spatial neighbor in human ROI. Then, a modified soft coding with locality constraint is adopted to alleviate the quantization error and preserve the manifold structure of features. This novel coding algorithm increases both the representation accuracy and discriminative power with low computational cost. The human action recognition experimental results on KTH, Weizmann, and UCF sports datasets show that PLSC can achieve a better performance than previous competing feature coding methods.

We focus on vision-based hand activity analysis in the vehicular domain. The study is motivated by the overarching goal of understanding driver behavior, in particular as it relates to attentiveness and risk. First, the unique advantages and challenges for a nonintrusive, vision-based solution are reviewed. Next, two approaches for hand activity analysis, one relying on static (appearance only) cues and another on dynamic (motion) cues, are compared. The motion-cue-based hand detection uses temporally accumulated edges in order to maintain the most reliable and relevant motion information. The accumulated image is fitted with ellipses in order to produce the location of the hands. The method is used to identify three hand activity classes: (1) two hands on the wheel, (2) hand on the instrument panel, (3) hand on the gear shift. The static-cue-based method extracts features in each frame in order to learn a hand presence model for each of the three regions. A second-stage classifier (linear support vector machine) produces the final activity classification. Experimental evaluation with different users and environmental variations under real-world driving shows the promise of applying the proposed systems for both postanalysis of captured driving data as well as for real-time driver assistance.

Face superresolution (SR), or face hallucination, refers to the technique of generating a high-resolution (HR) face image from a low-resolution (LR) one with the help of a set of training examples. It aims at transcending the limitations of electronic imaging systems. Applications of face SR include video surveillance, in which the individual of interest is often far from cameras. A two-step method is proposed to infer a high-quality and HR face image from a low-quality and LR observation. First, we establish the nonlinear relationship between LR face images and HR ones, according to radial basis function and partial least squares (RBF-PLS) regression, to transform the LR face into the global face space. Then, a locality-induced sparse representation (LiSR) approach is presented to enhance the local facial details once all the global faces for each LR training face are constructed. A comparison of some state-of-the-art SR methods shows the superiority of the proposed two-step approach, RBF-PLS global face regression followed by LiSR-based local patch reconstruction. Experiments also demonstrate the effectiveness under both simulation conditions and some real conditions.

There is a worldwide effort to apply 21st century intelligence to evolving our transportation networks. The goals of smart transportation networks are quite noble and manifold, including safety, efficiency, law enforcement, energy conservation, and emission reduction. Computer vision is playing a key role in this transportation evolution. Video imaging scientists are providing intelligent sensing and processing technologies for a wide variety of applications and services. There are many interesting technical challenges including imaging under a variety of environmental and illumination conditions, data overload, recognition and tracking of objects at high speed, distributed network sensing and processing, energy sources, as well as legal concerns. This paper presents a survey of computer vision techniques related to three key problems in the transportation domain: safety, efficiency, and security and law enforcement. A broad review of the literature is complemented by detailed treatment of a few selected algorithms and systems that the authors believe represent the state-of-the-art.

An efficient method is presented for extracting motion behaviors and contours of moving objects in a wide view and for creating panoramic background. In the field of making panorama, the main goal of existing methods is to create a pleasing wide view. For this purpose, such methods do not track moving objects. They attempt to find optimal seams so that the result does not contain cut objects or blurring. Hence, moving objects are removed, repeated, or placed in an arbitrary location in the final panoramic image. We expand panorama applications from artistic views to surveillance usages. To investigate moving object behavior, the proposed method attempts to find correspondences between positions of a moving object in different selected frames by using SIFT features. It also presents a new approach to combine various types of information in order to extract the exact boundary of moving objects in moving cameras. The required information is obtained from the moving object’s corresponding areas in other frames. Experiments were arranged to demonstrate the effectiveness and robustness of this method. The results show that this method, which uses fewer frames, is able to create better panoramic background compared with the existing methods.

A panorama parking assistant system (PPAS) for the automotive aftermarket together with a practical improved particle swarm optimization method (IPSO) are proposed in this paper. In the PPAS system, four fisheye cameras are installed in the vehicle with different views, and four channels of video frames captured by the cameras are processed as a 360-deg top-view image around the vehicle. Besides the embedded design of PPAS, the key problem for image distortion correction and mosaicking is the efficiency of parameter optimization in the process of camera calibration. In order to address this problem, an IPSO method is proposed. Compared with other parameter optimization methods, the proposed method allows a certain range of dynamic change for the intrinsic and extrinsic parameters, and can exploit only one reference image to complete all of the optimization; therefore, the efficiency of the whole camera calibration is increased. The PPAS is commercially available, and the IPSO method is a highly practical way to increase the efficiency of the installation and the calibration of PPAS in automobile 4S shops.

The exhaustive block partition search process in high efficiency video coding (HEVC) imposes a very high computational complexity on test module of HEVC encoder (HM). A fast coding unit (CU) depth algorithm using the spatio-temporal correlation of the depth information to fasten the search process is proposed. The depth of the coding tree unit (CTU) is predicted first by using the depth information of the spatio-temporal neighbor CTUs. Then, the depth information of the adjacent CU is incorporated to skip some specific depths when encoding the sub-CTU. As compared with the original HM encoder, experimental results show that the proposed algorithm can save more than 20% encoding time on average for intra-only, low-delay, low-delay P slices, and random access cases with almost the same rate-distortion performance.

The Katsevich algorithm is a breakthrough in the theoretically exact algorithms for helical cone beam computed tomography (CT). For future application in medical and industrial CT, determining how to implement it efficiently and accurately is the main task. We analyzed the slope law and intersection law of the k-lines, finding that the k-lines are not intersecting if the half-maximal fan angle (HMFA) is <21  deg (numerical solution, so it is approximate) and that the helical pitch and HMFA determine the depth of parallelism of k-lines. Using an appropriate pitch and an HMFA that is <21  deg , one can use a simplified Katsevich algorithm, whose filtration process can be done on the rows of the detector panel so that the preweighting, pre-rebinning, post-rebinning, and postweighting steps are all canceled. Simulation experiments show that the simplified algorithm can obtain highly precise images at a faster speed. Our results are intended to be valuable to those who are working on efficient implementations of the Katsevich-type algorithms.

Compressive sensing (CS) is an innovative technology, allowing us to capture signals with significantly fewer samples than those required by classical Nyquist theory. We propose a novel adaptive video compressive sensing algorithm to exploit the potential of CS in video acquisition. Each frame is divided into blocks to take advantage of its inhomogeneity. We first classify the blocks into one of three types based on their texture complexity and their temporal difference from neighboring frames based on which we determine the number of required measurements. In the reconstruction process, we use the measurements made for the later frames to assist the recovery of previous ones, thus ensuring improved reconstruction quality even when the number of measurements for each frame is limited. Our experimental results demonstrate that we not only obtain significant visual quality improvement but also achieve at least 2.5 dB gain in peak signal-to-noise ratio compared with the existing video compressive sensing algorithms.

A new method for segmenting intensity images into smooth surface segments is presented. The main idea is to divide the image into flat, planar, convex, concave, and saddle patches that coincide as well as possible with meaningful object features in the image. Therefore, we propose an adaptive region growing algorithm based on low-degree polynomial fitting. The algorithm uses a new adaptive thresholding technique with the L ∞ fitting cost as a segmentation criterion. The polynomial degree and the fitting error are automatically adapted during the region growing process. The main contribution is that the algorithm detects outliers and edges, distinguishes between strong and smooth intensity transitions and finds surface segments that are bent in a certain way. As a result, the surface segments corresponding to meaningful object features and the contours separating the surface segments coincide with real-image object edges. Moreover, the curvature-based surface shape information facilitates many tasks in image analysis, such as object recognition performed on the polynomial representation. The polynomial representation provides good image approximation while preserving all the necessary details of the objects in the reconstructed images. The method outperforms existing techniques when segmenting images of objects with diffuse reflecting surfaces.

Wheel steer angle information is crucial for the estimation of vehicle sideslip. Different from previous detection methods using angle sensors, this work presents a new wheel steer angle detection method based on a computer vision system providing image sequences recorded by a camera mounted on a car. The difference between wheel steering right and left is analyzed to determine steer direction while wheel steer angle in the image is derived from the information of the wheel contour extracted by the threshold segmentation and edge extraction. Experimental results show the efficiency of the proposed approach.

Computationally scalable image interpolation algorithm is always desirable for software and hardware implementations on center processing unit (CPU), digital signal processor, field-programmable gate array, and low-cost hardware. A low-complexity, computationally scalable, and data-adaptive image interpolation algorithm that has a simple and homogeneous structure to efficiently scale the computation is proposed. Specifically, the image interpolation as a denoising problem is formulated by proposing a new image model to relate the observed low-resolution pixels and missing high-resolution pixels. Applying the maximum-likelihood estimation using the new image model results in an adaptive linear filter, where the filter coefficients depend on the local noise covariance matrix, which is estimated by local noise samples. Due to low overhead of the proposed interpolator, the overall computation efficiently scales with the number of noise samples. Experimental results show that the proposed scalable algorithm outperforms the state-of-the-art fast algorithms and achieves more than 36 frames per second for upscaling a 540 p (960×540 ) video to a 1080 p (1920×1080 ) video using multithreaded C++ software implementation on a PC system with Intel i7 950 3 GHz CPU.

This work presents adaptive image steganography methods which locate suitable regions for embedding by contourlet transform, while embedded message bits are carried in discrete cosine transform coefficients. The first proposed method utilizes contourlet transform coefficients to select contour regions of the image. In the embedding procedure, some of the contourlet transform coefficients may change which may cause errors at the message extraction phase. We propose a novel iterative procedure to resolve such problems. In addition, we have proposed an improved version of the first method in which it uses an advanced embedding operation to boost its security. Experimental results show that the proposed base method is an imperceptible image steganography method with zero retrieval error rate. Comparisons with other steganography methods which utilize contourlet transform show that our proposed method is able to retrieve all messages perfectly, whereas the others fail. Moreover, the proposed method outperforms the ContSteg method in terms of PSNR and the higher-order statistics steganalysis method. Experimental evaluations of our methods with the well known DCT-based steganography algorithms have demonstrated that our improved method has superior performance in terms of PSNR and SSIM, and is more secure against the steganalysis attack.

In computer vision and graphics, reconstruction of a three-dimensional surface from a point cloud is a well-studied research area. As the surface contains information that can be measured, the application of surface reconstruction may be potentially important for applications in bioimaging. In the past decade, a number of algorithms for surface reconstruction have been developed. Generally speaking, these algorithms can be separated into two categories: explicit representation and implicit approximation. Most of these algorithms have a sound basis in mathematical theory. However, so far, no analytical evaluation between these algorithms has been presented. The straightforward method of evaluation has been by convincing through visual inspection. Therefore, we design an analytical approach by selecting surface distance, surface area, and surface curvature as three major surface descriptors. We evaluate these features in varied conditions. Our ground truth values are obtained from analytical shapes: the sphere, the ellipsoid, and the oval. Through evaluation we search for a method that can preserve the surface characteristics best and which is robust in the presence of noise. The results obtained from our experiments indicate that Poisson reconstruction method performs best. This outcome can now be used to produce reliable surface reconstruction of biological models.

Steganography is the art and science of hiding data in different media such as texts, audios, images, and videos. Data hiding techniques are generally divided into two groups: spatial and frequency domain techniques. Spatial domain methods generally have low security and, as a result, are less attractive to researchers. Discrete cosine transform (DCT) is the most common transform domain used in steganography and JPEG compression. Since a large number of the DCT coefficients of JPEG images are zero, the capacity of DCT domain-based steganography methods is not very high. We present a high-capacity method for hiding messages in the DCT domain. We describe the method in two classes where the receiver has and where the receiver does not have the cover image. In each class, we consider three cases for each coefficient. By considering n coefficients, there are 3 n different situations. The method embeds ⌊log 2  3 n ⌋ bits in these n coefficients. We show that the maximum reachable capacity by our method is 58% higher than the other general steganography methods. Experimental results show that the histogram-based steganalysis methods cannot detect the stego images produced by the proposed method while the capacity is increased significantly.

As any existing video codec, multiview video coding requires a highly efficient compression as well as a faster random access to any given view. Such efficiency for the coding approach can be achieved by exploiting both the temporal and the inter-view correlation. The temporal and view random access are improved by the increase of intracoded pictures (I pictures), which requires more bit rate. In this research study, we propose and evaluate an improved inter-view prediction structure mainly based on reducing the complexity of view random access at any instance T n and improving the inter-view correlation for P-view. These improvements can be guaranteed by the appropriate choice of the position of I-view, the type of the different views, and the number of each frame type to use. Compared to the prediction scheme, which is based on the hierarchical B pictures structure and used for joint multiview video model, experimental results have shown that the proposed prediction structure provides better bit-rate performance of up to 6.81% with a similar compression quality measured in peak signal-to-noise ratio and significantly improves view random access by up to 25%.

The maximum subarray problem is used to identify the subarray of a two-dimensional array, where the sum of elements is maximized. In terms of image processing, the solution has been used to find the brightest region within an image. Two parallel algorithms of the maximum subarray problem solve this problem in O(n) and O(log n) time. A field programmable gate array implementation has verified theoretical maximum performance; however, extensive customization is required, restricting general application. A more convenient platform for this work is a graphics processor unit since it offers a flexible trade-off between hardware customization and performance. Implementation of the maximum subarray algorithm on a graphics processor unit is discussed in this article for rectangular solutions and convex extensions are explored.

Manufacturers of commercial display devices continuously try to improve the perceived image quality of their products. By applying postprocessing techniques on the incoming signal, they aim to enhance the quality level perceived by the viewer. These postprocessing techniques are usually applied globally over the whole image but may cause side effects, the visibility and annoyance of which differ with local content characteristics. To better understand and utilize this, a three-phase experiment was conducted where observers were asked to score images that had different levels of quality in their regions of interest and in the background areas. The results show that the region of interest has a greater effect on the overall quality of the image than the background. This effect increases with the increasing quality difference between the two regions. Based on the subjective data we propose a model to predict the overall quality of images with different quality levels in different regions. This model, which is constructed on empirical bases, can help craft weighted objective metrics that can better approximate subjective quality scores.

Local binary patterns (LBPs) are effective facial texture feature descriptors in face recognition. However, the performance of original LBP-based face recognition methods rapidly deteriorates in the condition of nonmonotonic illumination variations. In order to overcome this drawback, we propose a LBP-based face recognition approach, namely relative gradient LBPs (RGLBPs), in which the relative gradient is first applied to the original face images to extract illumination invariant features. Then, an LBP describes textural and structural features for face recognition. Finally, the chi-square dissimilarity measure and the nearest neighbor classifier are used for classification. The experimental results validate that the proposed approach is efficient for the illumination problem in face recognition and also robust to expression and age variations.

We propose a two-phase approach to restore images corrupted by impulsive noise based on sparse representation. In the first phase, we identify the outlier candidates—the pixels that are likely to be corrupted by impulsive noise. In the second phase, the image is denoised via dictionary learning by using the outlier-free data. The dictionary learning task is formulated as a modified l 1 −l 1 minimization objective and solved under the alternating direction method. The experimental results demonstrate that our method can obtain better performances in terms of both quantitative evaluation and visual quality than the state-of-the-art impulse denoising methods.

A new technique for texture segmentation is presented. The method is based on the use of Laguerre Gauss (LG) functions, which allow an efficient representation of textures. In particular, the marginal densities of the LG expansion coefficients are approximated by the generalized Gaussian densities, which are completely described by two parameters. The classification and the segmentation steps are performed by using a modified k -means algorithm exploiting the Kullback–Leibler divergence as similarity metric. This clustering method is a more efficient system for texture comparison, thus resulting in a more accurate segmentation. The effectiveness of the proposed method is evaluated by using mosaic image sets created by using the Brodatz dataset, and real images.

Edge preserved enhancement is of great interest in medical images. Noise present in medical images affects the quality, contrast resolution, and most importantly, texture information and can make post-processing difficult also. An enhancement approach using an adaptive fusion algorithm is proposed which utilizes the features of shearlet transform (ST) and total variation (TV) approach. In the proposed method, three different denoised images processed with TV method, shearlet denoising, and edge information recovered from the remnant of the TV method and processed with the ST are fused adaptively. The result of enhanced images processed with the proposed method helps to improve the visibility and detectability of medical images. For the proposed method, different weights are evaluated from the different variance maps of individual denoised image and the edge extracted information from the remnant of the TV approach. The performance of the proposed method is evaluated by conducting various experiments on both the standard images and different medical images such as computed tomography, magnetic resonance, and ultrasound. Experiments show that the proposed method provides an improvement not only in noise reduction but also in the preservation of more edges and image details as compared to the others.

An improved method for compressing industrial computed tomography (CT) images is presented. To have higher resolution and precision, the amount of industrial CT data has become larger and larger. Considering that industrial CT images are approximately piece-wise constant, we develop a compression method based on contour coding. The traditional contour-based method for compressing gray images usually needs two steps. The first is contour extraction and then compression, which is negative for compression efficiency. So we merge the Freeman encoding idea into an improved method for two-dimensional contours extraction (2-D-IMCE) to improve the compression efficiency. By exploiting the continuity and logical linking, preliminary contour codes are directly obtained simultaneously with the contour extraction. By that, the two steps of the traditional contour-based compression method are simplified into only one. Finally, Huffman coding is employed to further losslessly compress preliminary contour codes. Experimental results show that this method can obtain a good compression ratio as well as keeping satisfactory quality of compressed images.

The usual tomography is achieved by acquiring measurements around an object with multiple angles. The possibility of obtaining a fluorescence tomographic image from measurements at only one angle is explored. Instead of rotating around the object, the camera (or the objective lens) moves toward (or away from) the object and takes photographs while the camera’s focal plane passes through the object. The volume of stacked two-dimensional pictures forms a blurred three-dimensional image. The true image can be obtained by deconvolving the system’s point spread function. Simplified computer simulations are carried out to verify the feasibility of the proposed method. The computer simulations indicate that it is feasible to obtain a tomographic image by using the in-and-out motion to acquire data.

Image fusion combines several images of the same scene into a fused image, which contains all important information. Multiscale transform and sparse representation can solve this problem effectively. However, due to the limited number of dictionary atoms, it is difficult to provide an accurate description for image details in the sparse representation–based image fusion method, and it needs a great deal of calculations. In addition, for the multiscale transform–based method, the low-pass subband coefficients are so hard to represent sparsely that they cannot extract significant features from images. In this paper, a nonsubsampled contourlet transform (NSCT) and sparse representation–based image fusion method (NSCTSR) is proposed. NSCT is used to perform a multiscale decomposition of source images to express the details of images, and we present a dictionary learning scheme in NSCT domain, based on which we can represent low-frequency information of the image sparsely in order to extract the salient features of images. Furthermore, it can reduce the calculation cost of the fusion algorithm with sparse representation by the way of nonoverlapping blocking. The experimental results show that the proposed method outperforms both the fusion method based on single sparse representation and multiscale decompositon.

We describe a new bad pixel correction algorithm designed originally to correct two-dimensional images containing high-fidelity spectroscopic data. Our method can be used to correct images with significant rates of adjacent bad pixels or bad pixel clusters such as taken by infrared focal plane arrays. It is applicable to any image data that can be locally approximated by a second-degree polynomial. A filter performing the correction can be easily implemented in programmable hardware utilizing very little resources. The accuracy of the method is analyzed and its performance tested on real spectroscopic data.

Image processing in photogrammetry is commonly used for scene reconstruction. Although two-dimensional applications can be solved using isolated images, reconstruction of three-dimensional scenes usually requires the use of multiple images simultaneously. Epipolar image rectification is a common technique for this purpose. It typically requires internal orientation parameters and, therefore, knowledge of camera calibration and relative orientation parameters between images. A reparameterization of the fundamental matrix through a completely geometric algorithm of seven parameters that enables the epipolar image rectification of a photogrammetric stereo pair without introducing any orientation parameters and without premarking ground control points is presented. The algorithm enables the generation of different stereoscopic models with a single photogrammetric pair from unknown cameras, scanned from a book, or frames from video sequences. Stereoscopic models with no parallaxes have been obtained with a standard deviation of <0.5  pixels .

Visual tracking under illumination changes is a challenging task for numerous computer vision applications. We propose a robust and efficient tracking algorithm based on maximum color difference histogram (MCDH) and a well-designed min-max-ratio (MMR) similarity metric. Appearance model is essential to the tracker’s robustness under illumination changes. We propose a new feature descriptor MCDH, calculated by exploiting the maximum color difference information within the eight-neighborhood of each pixel in the object region, to build the object appearance model. Unlike the traditional histogram-based algorithms, the MCDH is efficiently extracted by employing the local integral histogram which is propagated in a specially designed local image region. The similarity metric plays an important role on accurately locating the target. However, the existing metrics are not suitable for comparisons of MCDHs due to many zero-valued bins in MCDH. Therefore, we propose a new MMR metric, defined as the average ratio between the minimum and maximum of a MCDH bin pair. The combination of proposed components enables the tracker to be robust to illumination changes with high computational efficiency. Experiments demonstrate superior performance of the proposed tracking algorithm compared with 10 state-of-art tracking methods when illumination varies.

Image quality and depth perception rates in three-dimensional television (3-D TV) are undesirably decreased by coding due to the loss of high-frequency components caused by a block-based discrete cosine transform transform. Representative coding artifacts are blocking artifacts that seriously degrade the picture quality and depth perception rates. We propose a new blocking artifact reduction method in 3-D TV using an overcomplete 3-D dictionary. We first generate the overcomplete 3-D dictionary from natural and depth images using the k-singular value decomposition algorithm. Then, we perform deblocking using the 3-D dictionary after estimating an error threshold of the objective function by the third-order polynomial fitting. Experimental results demonstrate that the proposed method can effectively reduce annoying blocking artifacts in compressed 3-D images, i.e., video-plus-depth, and generate high-quality 3-D stereoscopic views.

Detection of red lesions [hemorrhages (HRs) and microaneurysms (MAs)] is crucial for the diagnosis of early diabetic retinopathy. A method based on background estimation and adapted to specific characteristics of HRs and MAs is proposed. Candidate red lesions are located by background estimation and Mahalanobis distance measure and then some adaptive postprocessing techniques, which include vessel detection, nonvessel exclusion based on shape analysis, and noise points exclusion by double-ring filter (only used for MAs detection), are conducted to remove nonlesion pixels. The method is evaluated on our collected image dataset, and experimental results show that it is better than or approximate to other previous approaches. It is effective to reduce the false-positive and false-negative results that arise from incomplete and inaccurate vessel structure.

We propose an efficient blind/no-reference image quality assessment algorithm using a log-derivative statistical model of natural scenes. Our method, called DErivative Statistics-based QUality Evaluator (DESIQUE), extracts image quality-related statistical features at two image scales in both the spatial and frequency domains. In the spatial domain, normalized pixel values of an image are modeled in two ways: pointwise-based statistics for single pixel values and pairwise-based log-derivative statistics for the relationship of pixel pairs. In the frequency domain, log-Gabor filters are used to extract the fine scales of the image, which are also modeled by the log-derivative statistics. All of these statistics can be fitted by a generalized Gaussian distribution model, and the estimated parameters are fed into combined frameworks to estimate image quality. We train our models on the LIVE database by using optimized support vector machine learning. Experiment results tested on other databases show that the proposed algorithm not only yields a substantial improvement in predictive performance as compared to other state-of-the-art no-reference image quality assessment methods, but also maintains a high computational efficiency.

A new method for unsupervised segmentation of moving objects in infrared videos is presented. This method consists of two steps: difference image quantization and spatial segmentation. In the first step, the changed pixels in the difference image are quantized to several classes by using Bayes decision. It can be used to cluster the changed pixels belonging to the same moving object together. The pixels of the difference image are replaced by their corresponding class labels, thus forming a class-map of the difference image. In the second step, each class in the class-map is considered as a subset of the possible seeds of moving objects. A self-adaptive region growing method is then used to image segmentation on the basis of these different subsets. One of the focuses of this work is on spatial segmentation, where a criterion is proposed for evaluation of moving object segmentation without ground truth in infrared videos. This criterion is used to evaluate the performance of the segmentation masks grown from different subsets of the possible seeds. The best segmented image is determined to be the final segmentation result. Experiments show the advantage and robustness of the proposed algorithm on real infrared videos.

Superresolution (SR) is known to be an ill-posed inverse problem, which may be solved using some regularization techniques. We have proposed an adaptive regularization method, based on a Charbonnier nonlinear diffusion model to solve an SR problem. The proposed model is flexible because of its automatic capability to reap the strengths of either linear isotropic diffusion, Charbonnier model, or semi-Charbonnier model, depending on the local features of the image. On the contrary, the models proposed from other research works are fixed and hence less feature dependent. This makes such models insensitive to local structures of the images, thereby producing poor reconstruction results. Empirical results obtained from experiments, and presented here, show that the proposed method produces superresolved images which are more natural and contain well-preserved and clearly distinguishable image structures, such as edges. In comparison with other methods, the proposed method demonstrates higher performance in terms of the quality of images it generates.

The dynamic programming-based dense stereo matching algorithms are faster than other global ones, but they usually suffer from the tangible horizontal “streaks” in their generated disparity maps. This problem takes place due to the independent processing of scanlines, ignoring vertical consistency between them. In this paper the path remembering method, which uses the information of the best paths of previous rows, is employed with new developments to establish more depth similarity between rows, especially along the slanted edges of objects in the scene. Also, a new way for estimating matching costs in the left border of the image is presented based on the information of the right-hand neighboring pixels. Mentioned improvements were applied to a well known and standard dynamic programming-based algorithm called “DP.” Evaluation of the new proposed algorithm on standard stereo images shows considerable improvements in the depth estimation along the slanted edges and in the left border of the final results. As a more precise report, the average error rate reduces from 14.2% to 10.4%.

Digital watermarking and data hiding are important tools for digital rights protection of media data. Spread spectrum (SS)-based watermarking and data-hiding approaches are popular due to their outstanding robustness, but their security might not be sufficient. To improve the security of SS, a SS-based image data-hiding approach is proposed by encrypting the discrete cosine transform coefficients of the host image with the piecewise linear chaotic map, before the operation of watermark embedding. To evaluate the performance of the proposed approach, simulations and analyses of its robustness and security are carried out. The average bit-error-rate values on 100 real images from the Berkeley segmentation dataset under the JPEG compression, additive Gaussian noise, salt and pepper noise, and cropping attacks are reported. Experimental results show that the proposed approach can maintain the high robustness of traditional SS schemes and, meanwhile, also improve the security. The proposed approach can extend the key space of traditional SS schemes from 10 19 to 10 47 and thus can resist brute-force attack and unauthorized detection watermark attack.

An efficient, robust, and accurate approach is developed for image registration, which is especially suitable for large-scale change and arbitrary rotation. It is named the adequately sampling polar transform and weighted angular projection function (ASPT–WAPF). The proposed ASPT model overcomes the oversampling problem of conventional log-polar transform. Additionally, the WAPF presented as the feature descriptor is robust to the alteration in the fovea area of an image, and reduces the computational cost of the following registration process. The experimental results show two major advantages of the proposed method. First, it can register images with high accuracy even when the scale factor is up to 10 and the rotation angle is arbitrary. However, the maximum scaling estimated by the state-of-the-art algorithms is 6. Second, our algorithm is more robust to the size of the sampling region while not decreasing the accuracy of the registration.

We introduce a stereovision system and the three-dimensional (3-D) image analysis algorithms for fabric pilling measurement. Based on the depth information available in the 3-D image, the pilling detection process starts from the seed searching at local depth maxima to the region growing around the selected seeds using both depth and distance criteria. After the pilling detection, the density, height, and area of individual pills in the image can be extracted to describe the pilling appearance. According to the multivariate regression analysis on the 3-D images of 30 cotton fabrics treated by the random-tumble and home-laundering machines, the pilling grade is highly correlated with the pilling density (R 2 =0.923 ) but does not consistently change with the pilling height and area. The pilling densities measured from the 3-D images also correlate well with those counted manually from the samples (R 2 =0.985 ).

The paper introduces a spherical coordinate system-based color model and studies the color change in the model. A circular cone with a spherical top tightly circumscribing the RGB color cube is equipped with a properly rotated spherical coordinate system. Similar to the commonly used color models with a hue component such as the HSV model, the spherical model specifies color by describing the color attributes recognized by human vision, using the components of the spherical coordinate system. The formulas of conversions between the spherical model and the RGB color model are provided, which are mathematically simpler and more intuitively understandable than those for commonly used models with a hue component. Most importantly, color changes are perceptually smoother in the spherical model. Comparisons between the spherical model and the HSV model on color changes are made in the paper.

The current JPEG steganalysis systems, which include various strategies for feature extraction, have attained outstanding achievements. However, a common shortcoming is that they are always conducted on the entire image and do not take advantage of the content diversity. In addition, compared with a low-dimensional feature set, an appropriate rich model with high-dimensional features can obtain substantial improvement in steganalysis performances. A new steganalysis algorithm based on image segmentation is proposed which enables us to utilize the content characteristics of JPEG images. The given images are segmented to several subimages according to the texture complexity and then high-dimensional steganalysis features of each sort of subimages with the same or close texture complexity are extracted separately to build a classifier. The steganalysis results of the whole image are figured out through a weighted fusing process of all categories of the subimages. Experimental results demonstrate that the proposed method exhibits excellent performances and improves the detection accuracy.