The encoding of vector quantization (VQ) needs expensive computation for searching the closet codeword to the input vectors. In order to reduce computation burden, Pan et al. have developed an efficient full-search-equivalent algorithm by using the characteristics of the sums and variances of a vector and its two fixed subvectors recently. However, some computational redundancies still exist in it. In this paper, we introduce a technique to efficiently partition a vector into two dynamic subvectors according to the patterns inside the block (or vector) and then further improve the computation load given by Pan's method. Experimental results show that the proposed method is superior to other algorithms in terms of processing time and the number of distortion calculations. Compared to Pan's algorithm, our algorithm can further reduce the processing time and the number of distortion calculations for various codebook sizes by 14.7% to 41.3% and 14.9% to 44.6%, respectively.

The chrominance subsampling has been widely used for the image and video compression. However, this can cause color distortion and image quality degradation. We developed a new sampling method in which the residual image after intra or inter prediction is subsampled instead of the original image subsampling. Since the residual image contains fewer information than the original image does generally, the information loss is reduced when the residual image is downsampled. Also the upsampling procedure is much less sensitive to the filtering methods selected as demonstrated in the experimental results. So we can apply a simple filtering to reduce the complexity. The experimental results show the effectiveness of the proposed sampling method. This sampling method can be further improved by applying adaptively according to the statistics of each block, and can be applied to the RGB image directly without color conversion.

Variable block size motion compensation has been adopted by the emerging video coding standard H.264. It can represent the motion characteristic in a macroblock more accurately, and therefore reduce the prediction error to achieve high compression gains. On the other hand, it causes high computational complexity in motion estimation at the encoder. The motion estimation exhaustively performed over all modes to find the best mode for inter coding is slow and computationally involved. In order to reduce the complexity, we proposed a fast inter mode decision algorithm for H.264 encoder. The proposed method efficiently determines a suitable block mode according to the motion field distribution and correlation within a macroblock. The experimental results show that the proposed method reduces a considerable amount of complexity at encoder, while the R-D performance of the proposed method is comparable to that of full mode search.

This paper proposes a new motion compensation method using "sliced blocks" in DCT-based hybrid video coding. In H.264 | MPEG-4 Advance Video Coding, a brand-new international video coding standard, motion compensation can be performed by splitting macroblocks into multiple square or rectangular regions. In the proposed method, on the other hand, macroblocks or sub-macroblocks are divided into two regions (sliced blocks) by an arbitrary line segment. The result is that the shapes of the segmented regions are not limited to squares or rectangles, allowing the shapes of the segmented regions to better match the boundaries between moving objects. Thus, the proposed method can improve the performance of the motion compensation. In addition, adaptive prediction of the shape according to the region shape of the surrounding macroblocks can reduce overheads to describe shape information in the bitstream. The proposed method also has the advantage that conventional coding techniques such as mode decision using rate-distortion optimization can be utilized, since coding processes such as frequency transform and quantization are performed on a macroblock basis, similar to the conventional coding methods. The proposed method is implemented in an H.264-based P-picture codec and an improvement in bit rate of 5% is confirmed in comparison with H.264.

Adaptive Block-size Transforms (ABT) has been widely used in image/video coding, since it exploits the maximum feasible signal length for transform coding. However, if the transforms in an ABT coding system are Integer Cosine Transforms (ICT), not only separate transform units but also different scaling matrices are required, which consume a vast amount of resources in practical implementations. In this paper, a new approach to compatible ABT is presented, by which 8x8, 8x4, 4x8 and 4x4 transforms can be processed in one transform unit. Furthermore, with Pre-scaled Integer Transform (PIT), the compatibility of scaling matrices especially for 8x4 and 4x8 ICT can be achieved and a single scaling matrix is required. Simulation results and analysis reveal that this approach greatly saves hardware resources and makes the implementation of ABT much easier without loss of performance.

We propose a fast motion search method for H.264/AVC with dual-path block size decision. H.264/AVC employs variable block sizes for motion compensation to reduce coding bits of inter-frame prediction error, which requires considerable amount of computation time when motion estimation is performed for every block size. Our algorithm contains two strategies to reduce computation time for motion search; block size mode reduction and search range reduction. According to these strategies, our algorithm consists of two stages. At the first stage, RDCost-based block size mode reduction is conducted. Rate-distortion function (RDCost) for skip mode is calculated at first, which determines the smallest block size for motion search. The second stage is a fast variable block size motion estimation which contains two paths, 16x16-first and 8x8-first. The 16x16-first path is invoked when the minimum block size determined at the first stage is larger than 8x8. In the 8x8-first path, search range for blocks larger than 8x8 is reduced according to distance between motion vectors for 8x8 blocks. From our experiment using JM 8.5, it is confirmed that our algorithm can reduce about 89.3% of computation time as compared to JM, with only negligible PSNR degradation.

Providing quality of service (QoS) guarantees in packet networks gives rise to several challenging issues. In general, multi-constrained path selection with or without optimization, is a NP-complete problem that cannot be exactly solved in polynomial time. Hence, accurate constraints-based routing algorithms with a fast running time are scarce, perhaps even non-existent. The expected impact of such a constrained-based routing algorithm has resulted in the proposal of numerous heuristics and a few exact QoS algorithms. This paper aims to give a thorough, concise and fair evaluation of the most important multiple constraint-based QoS Multicast routing algorithms known today, and it provides a descriptive overview and simulation results of these multi-constrained routing algorithms.

As the Internet does not widely support Internet protocol multicast, and content distribution networks are costly, application overlay has emerged as an alternative for deploying large scale streaming systems. Loss recovery in such architecture is a great challenge because of the correlation of packet losses results from the relaying nature, the variation of accumulated loss rates, and the dynamics of overlay structure, which is inevitable. Since each overlay node is capable of buffering a certain length of media data it has received, and there exists a temporal dependency between the buffers of nodes along a transmitting path, it is highly desirable to make full use of this buffering capability to carry out loss recovery. To this purpose, a retransmission-based approach is proposed in this paper. First, packet losses at a node are classified into two categories according to whether they can be repaired from the immediate upstream node, then, by making upstream nodes propagate loss information downstream, the proposed scheme enables each node efficiently recognize the nature of the loss it detects, and accordingly determine appropriate repair source, thus suppresses unnecessary retransmission requests. The proposed scheme is supported by theoretical analysis of the temporal dependency between overlay nodes, and its performance is verified by experimental results.

This paper investigates the transmission of H.264 /AVC video in the 3GPP Multimedia Broadcast/Multicast Streaming service (MBMS). Application-layer forward error correction (FEC) codes are used to combat transmission errors in the radio access network. In this FEC protection scheme, the media RTP stream is organized into source blocks spanning many RTP packets, over which FEC repair packets are generated. This paper proposes a novel method for unequal error protection that is applicable in MBMS. The method reduces the expected tune-in delay when a new user joins into a broadcast. It is based on four steps. First, temporally scalable H.264 /AVC streams are coded including reference and non-reference pictures or sub-sequences. Second, the constituent pictures of a group of pictures (GOP) are grouped according to their temporal scalability layer. Third, the interleaved packetization mode of RFC3984 is used to transmit the groups in ascending order of relevance for decoding. As an example, the non-reference pictures of a GOP are sent earlier than the reference pictures of the GOP. Fourth, each group is considered a source block for FEC coding and the strength of the FEC is selected according to its importance. Simulations show that the proposed method improves the quality of the received video stream and decreases the expected tune-in delay.

This paper presents an H.263+-based overlay video multicast system over the Internet. The proposed overlay video multicast system consists of two parts: overlay multicast tree suitable for the video traffic delivery and H.263+ rate control adaptive to overlay multicast tree. Overlay multicast tree is constructed to minimize the average delay for video traffic delivery, and H.263+ rate control is implemented to enhance the human visual perceptual quality over the multicast tree. Finally, we show by experimental results that the proposed overlay multicast system can provide the effective overlay multicast service over the Internet.

With the intensifying academic interest in video streaming over peer-to-peer(P2P) network, more and more streaming protocols have been proposed to address different problems, such as QoS, load balancing, transmission reliability, bandwidth effciency, etc. However, to the best of our knowledge, an important component of any practical P2P streaming system, the streaming service discovery, which is to discover potential peers from which a newcomer could receive the requested stream, is rarely granted particular consideration. In this paper, we are inspired from a special data structure, namely Segment Tree(ST), and propose a protocol to address this problem. Furthermore, we fully decouple the control plane and data plane in video streaming, and hence provide more flexibilities in designing protocols on both of them.

In video-on-demand (VOD) system, efficient video delivery scheme can largely increase system service ability by reducing the resource requirement. Most existing schemes are not suited for the instance that the characteristic of the workload changing greatly with time. A workload adaptive scheme is proposed in the paper. It absorbs the outstanding thoughts from two efficient schemes: fast broadcasting and patching scheme. The scheme allocates adaptively the number of channels according to request rate to minimize the required bandwidth. We show how to seamlessly perform the transition of changing the number of channels while guaranteeing the clients currently viewing a video will not experience any disruption. The scheme can provide zero-delay video-on-demand (VOD) service. Simulation results show that the scheme adapts nicely to the changing request rate and improves the performance of VOD service significantly in terms of total server bandwidth requirement.

In the context of visual signal analysis for media adaptation, this paper presents a stochastic method for sementation of motion vector fields or as known as displacement fields in compressed videos. The addressed field sequence is analysed and partitioned to temporally associated, motion-semantic regions by three analysis steps. The global motion based reliability analysis assesses the reliability extent of each encoded displacement vector. Upon the measure array, an initial partition is approximated by the region displacement prediction. The subsequent relaxation procedure optimises the partition shape, coverage, and location using the stochastic motion coherency model and the second-order random field based contour smoothness. The visually convincing results are demonstrated from the standard sequences.

This paper aims to apply the method of silhouette matching based on moment invariants to infer the human motion parameters from video sequences of single monocular uncalibrated camera. Currently, there are two ways of tracking human motion: Marker and Markerless. While a hybrid framework is introduced in this paper to recover the input video contents. A standard 3D motion database is built up by marker technique in advance. Given a video sequences, human silhouettes are extracted as well as the viewpoint information of the camera which would be utilized to project the standard 3D motion database onto the 2D one. Therefore, the video recovery problem is formulated as a matching issue of finding the most similar body pose in standard 2D library with the one in video image. The framework is applied to the special trampoline sport where we can obtain the complicated human motion parameters in the single camera video sequences, and a lot of experiments are demonstrated that this approach is feasible in the field of monocular video-based 3D motion reconstruction.

This study is aimed at obtaining smoothed trajectory of feature points in dynamic image, where the feature points are extracted at each image frame with missing and false detection. The image scene contains independently moving multiple objects with occlusion and appearance. We develop a state space model using Finite Random Set (FRS) to cope with this situation since FRS is a set of random variables with the number of the variables also being random (integer) variable so it is suitable for representing the variable number of feature points caused by appearance/occlusion and missing/false detection. By estimating the state of the model using Sequential Monte Carlo (SMC) implementation of Probability Hypothesis Density (PHD) filter, we obtain smoothed trajectory of the feature points. PHD is 1st order moment of FRS, and it has a property that its integration yeilds the expected number of feature points in the integrated region. The SMC implementation gives approximated solution by weighted particles, where the number of particles varies depending on the number of feature points in the scene. Experiment of dynamic image demonstrates that proposed method successfully smoothed the trajectory of feature points responding to appearance and occlusion of objects without being affeted by missing and false detection.

Moving objects segmentation is a challenge in computer vision. This paper focuses on the segmentation of moving vehicles in dynamic scene. We analyses the psychology of human vision and present a framework for segmenting moving vehicles in the highway. The proposed framework consists of two parts. Firstly, we propose an adaptive background update method in which the background is updated according to the change of illumination conditions and thus can adapt to the change of illumination sensitively. Secondly, we construct a Gaussian motion model to segment moving vehicles, in which the motion vectors of the moving pixels are modeled as a Gaussian model and an on-line EM algorithm is used to update the model. The Gaussian distribution of the adaptive model is elevated to determine which moving vectors result from moving vehicles and which from other moving objects such as waving trees. Finally, the pixels with motion vector result from the moving vehicles are segmented. Experimental results of several typical scenes show that the proposed model can detect the moving vehicles correctly and is immune from influence of the moving objects caused by the waving trees and the vibration of camera.

Human motion analysis research, especially the human tracking part, remains a challenging task by far. The difficulties lie in several aspects: self-occlusion, high dimensionality of parameter space and the gap between high-level image understanding and low-level image features etc. In our work we use particle filter to track human movement from monocular video sequences with an articulated human body model. We fuse region, color and boundary information to build a robust measurement function. Among them, the boundary information represented by Fourier Descriptors (FD) sets up a new and effective connection between the estimated model parameters and the image likelihoods. Compared with the previously used boundary or contour cue, FD has many noticeable advantages. Moreover, we introduce an adaptive property into the particle filter for more robust state propagation and measurement updating. Our method is shown to work effectively in experiments.

Traditionally, video segmentation usually extracts object using low-level features such as color, texture, edge, motion, and optical flow. This paper originally proposes that the connectivity of object motion is an advanced feature of video moving object because it can reflect semantic meanings of object to some extent. And it can be fully represented on cumulated difference image which is the combination of a certain number of interframe difference images. Based on this principle, a novel system is designed to extract initial moving object automatically. The system includes 3 key innovations: 1) System is applied on cumulated difference image which can make object more prominent than background noise. Object extraction is based on the connectivity of object motion and it can guarantee the integrity of the extracted object while eliminate big background regions which cannot be removed by conventional change detection methods, for example, intense-noise regions and shadow regions that are not connected tightly to object. 2) Video sequence analysis is performed ahead of video segmentation. Proper object extraction methods are adopted according to the characteristics of background noise and object motion. 3) The adaptive threshold is automatically determined on cumulated difference image after acute noises is removed. The threshold determined here is more reasonable. And with it, most noise can be eliminated while small-motion regions of object are preserved. Results show that this system can extract object in different kinds of sequences automatically, promptly and properly. Thus, this system is very suitable for real time video applications.

In this paper we present an adaptive version of the update step in the lifting implementation of MCTF (Motion Compensated Temporal Filtering). We explicitly take into account the local image content in order to avoid ghosting artifacts in the updated low-pass frame, tuning the update weighting factor where such artifacts are more likely to be perceived. The proposed solution is integrated into a 2D+t (in-band) wavelet based video codec and improves the subjective quality of sequences reconstructed at reduced frame rate.

In this work, hierarchical motion compensated three-dimensional (3-D) filter banks for spatio-temporal multi-resoution analysis are presented as new tools for scalable video format control. For recent developments in scalable video coding, most of them are based on 3-D wavelet transform with motion compensation. To achieve the function of frame-rate and spatial resolution scalabilities, motion compensated temporal filtering (MCTF) through lifting wavelet transform currently attracts many researchers as an effective temporal decomposition tool. As previous works, we proposed single stage deinterlacer banks as novel 3-D filter banks. Unlike other filter banks, our proposed system is constructed in a way unique to multi-dimensional systems by using invertible deinterlacers, which we have proposed before. The single stage deinterlacer banks decompose a progressive video into two subband sequences of a half frame-rate in the progressive scanning manner. Even though the system handles interlaced videos as intermediate data, introducing a multistage decomposition is simply achieved because the output sequences have also the progressive format as like the input. Our proposed hierarchical deinterlacer banks are accompanied by 2-D discrete wavelet transforms (DWT) as spatial transforms. The multistage technique provides interlaced sequences as well as several reduced-rate progressive sequences while maintaining the reconstruction of the original full-resolution full-rate video sequence. We show that fine granular control of spatio-temporal resolution can be acchieved through the hierarchical process. Some experimental results show novel functions and the significance of the proposed filter banks.

This paper makes a comparative study on the various spatial scalable coding frameworks. The frameworks with multiple image-domain motion aligned temporal filtering at various spatial resolutions, named as multi-T+2D, are mainly investigated. First we investigate a multi-T+2D scheme based on redundant frame representation. The cross spatial layer redundancy and prediction methods are discussed. The redundancy brings significant performance loss for schemes providing wide range SNR spatial scalability. To remove the redundancy produced in the multi-resolution temporal filtering while retaining the advantage of spatial-domain motion compensation, a novel non-redundant multi-T+2D scheme is proposed. Performance comparison is given among the discussed frameworks and it shows that the proposed non-redundant multi-T+2D framework has a good performance for fully scalable video coding. We also verify that the redundant multi-T+2D framework with cross spatial layer reconstruction feedback is practical in providing narrow range SNR scalability for each spatial layer.

This paper presents a new Base Layer coding scheme which introduces the rate-distortion optimization mode decision into hierarchical-fashion B-picture coding with open-loop structure. It can not only achieve high coding performance for Base Layer bit-rate point and meanwhile, but the scheme also keep the strong correlation between Base Layer and Enhancement Layer implemented by 3D subband scalable video codec, because they share the similar lifting based Motion-Compensated Temporal Filtering (MCTF) decomposition structure. In addition, how to exploit the correlation between Base Layer and Enhancement Layer is also investigated. Experimental results show this method improves the coding performance and visual quality at both low and high bit-rate points.

As of its open-loop structure and good decorrelation capability, motion-compensated temporal filtering (MCTF) provides a robust basis for highly-efficient scalable video coding. Combining MCTF with spatial wavelet decomposition and embedded quantization results in a 3D wavelet video compression system, providing temporal, spatial, and SNR scalability. Recent results indicate that the overall coding performance of these systems can be maximized if temporal filtering is performed in spatial domain (t+2D approach). However, as compared to non-scalable video coding, the performance of t+2D systems may not be satisfactory if spatial scalability needs to be provided. One important reason for this fact is the problem of spatial scalability of motion information. In this paper we present a conceptually new approach for t+2D-based video compression with spatially scalable motion information. We call our approach overcomplete MCTF since multiple spatial-domain temporal filtering operations are needed to generate the lower spatial scales of the temporal subbands. Specifically, the encoder performs MCTF-based generation of reference sequences for the coarser spatial scales. We find that the newly generated reference sequences are of satisfactory quality. Compared to the conventional t+2D system, our approach allows for optimization of the reconstruction quality at lower spatial scales while having reduced impact on the reconstruction quality at high spatial scales/bitrates.

The discrete wavelet transform (DWT) has been widely used in scalable video coding for its advantages in multi-resolution analysis and subband decomposition. In this paper, a spatially scalable video coding system based on H.264 coding method and in-band overcomplete discrete wavelet transform (ODWT) technique is proposed, which integrates the good compression performance of H.264 in low frequency domain with the efficient motion estimation of in- band ODWT in wavelet domain. Intra prediction, coefficients scan manner and inter prediction are improved to overcome the inefficiency of H.264 coding in high frequency subbands caused by different pixels distribution properties. Through series of subband analysis and statistical data retrieval for the three high frequency decompositions, intra prediction directions are optimized and three subsets of prediction mode are presented for the three high subbands respectively. They save over 30% bits for intra mode with similar performance. Moreover, novel zigzag scan tables are proposed to improve the coding efficiency by utilizing the oriented frequency characteristics of each high band. To inter prediction, an adaptive motion estimation method is proposed in which the motion information of low band is adaptively and effectively utilized to achieve much more accurate motion vector and more efficient motion compensation in high bands coding. Experimental results show that, all of the proposed methods endue the spatially scalable video coding system with over 0.4 dB gain in PSNR and 10.4% in rate reduction.

For video retrieval, a video is partitioned into a group of shots, which are then represented by either key frames or video shot representations. An optimal representation of a shot should include all the information about the frames concerned. In this paper, we propose an efficient representation scheme for a shot, which considers both the spatial frequency contents and the temporal statistics of the frames for video retrieval. In our scheme, each frame in a video shot is transformed into the frequency domain using the discrete cosine transform (DCT), and a number of values at each frequency are selected based on their probability of occurrence. This representation scheme allows retrieval to be carried out hierarchically, i.e. from low-frequency to high-frequency components. Experimental results show that our proposed scheme outperforms the alpha-trimmed average histogram method in terms of retrieval accuracy.

We propose a fast coarse-to-fine video retrieval scheme using shot-level spatio-temporal statistics. The proposed scheme consists of a two-step coarse search and a fine search. At the coarse-search stage, the shot-level motion and color distributions are computed as the spatio-temporal features for shot matching. The first-pass coarse search uses the shot-level global statistics to cut down the size of the search space drastically. By adding an adjacent shot of the first query shot, the second-pass coarse-search introduces the "causality" relation between two consecutive shots to improve the search accuracy. As a result, the final fine-search step based on local color features of key-frames of the query shot is performed to further refine the search result. Experimental results show that the proposed methods can achieve good retrieval performance with a much reduced complexity compared to single-pass methods.

In this paper, we propose a novel semi-automatic annotation scheme for home videos based on active learning. It is well-known that there is a large gap between semantics and low-level features. To narrow down this gap, relevance feedback has been introduced in a number of literatures. Furthermore, to accelerate the convergence to the optimal result, several active learning schemes, in which the most informative samples are chosen to be annotated, have been proposed in literature instead of randomly selecting samples. In this paper, a representative active learning method is proposed, which local consistency of video content is effectively taken into consideration. The main idea is to exploit the global and local statistical characteristics of videos, and the temporal relationship between shots. The global model is trained on a smaller pre-labeled video dataset, and the local information is obtained online in the process of active learning, and will be used to adjust the initial global model adaptively. The experiment results show that the proposed active learning scheme has significantly improved the annotation performance compared with random selecting and common active learning method.

As a very important category in sports video, racquet sports video, e.g. table tennis, tennis and badminton, has been paid little attention in the past years. Considering the characteristics of this kind of sports video, we propose a new scheme for structure indexing and highlight generating based on the combination of audio and visual information. Firstly, a supervised classification method is employed to detect important audio symbols including impact (ball hit), audience cheers, commentator speech, etc. Meanwhile an unsupervised algorithm is proposed to group video shots into various clusters. Then, by taking advantage of temporal relationship between audio and visual signals, we can specify the scene clusters with semantic labels including rally scenes and break scenes. Thirdly, a refinement procedure is developed to reduce false rally scenes by further audio analysis. Finally, an exciting model is proposed to rank the detected rally scenes from which many exciting video clips such as game (match) points can be correctly retrieved. Experiments on two types of representative racquet sports video, table tennis video and tennis video, demonstrate encouraging results.

In this paper, we present a learning-based approach to mining the capture intention of camcorder users, aiming at providing a novel viewpoint in terms of home video content analysis. In contrast to existing approaches to video analysis designed from the viewers' standpoint, this approach models the capture intention from a camcorder user's point of view, by investigating a set of effective intention oriented features. With this approach, not only the capture intention is effectively mined, but also a set of intention probability curves are produced for efficient browsing of home video content. The experimental evaluations indicate that the intention based approach is an effective complement to existing home video content analysis schemes.

Highlight detection is a form of video summarization techniques aiming at including the most expressive or attracting parts in the video. Most video highlights selection research work has been performed on sports video, detecting certain objects or events such as goals in soccer video, touch down in football and others. In this paper, we present a highlight detection method for film video. Highlight section in a film video is not like that in sports video that usually has certain objects or events. The methods to determine a highlight part in a film video can exhibit as three aspects: (a) locating obvious audio event, (b) detecting expressive visual content around the obvious audio location, (c) selecting the preferred portion of the extracted audio-visual highlight segments. We define a double filters model to detect the potential highlights in video. First obvious audio location is determined through filtering the obvious audio features, and then we perform the potential visual salience detection around the potential audio highlight location. Finally the production from the audio-visual double filters is compared with a preference threshold to determine the final highlights. The user study results indicate that the double filters detection approach is an effective method for highlight detection for video content analysis.

Fingerprints are useful for biometric purposes because of their well known properties of distinctiveness and persistence over time. However, owing to skin condition or incorrect finger pressure, some fingerprint images contain useless components. They are often mistaken for the terminations that are an essential minutia of a fingerprint. Mathematical Morphology (MM) is one powerful tool in image processing. The performance of MM is heavily dependent on structuring elements (SEs), but finding an optimal SE is a difficult and nontrivial task. In this work, we focus on automatically choosing an appropriate SE for eliminating useless components of fingerprint images using linear time Euclidean distance transform algorithm. We exploit the property that these useless components are thinner than fingerprint ridges. So, we estimate the width of useless components and fingerprint ridges using Euclidean distance transform values, and then provide a linear time algorithm to eliminate the useless components. We have developed a novel algorithm for the binarization of fingerprint images. The information of distance transform values can be obtained directly from the binarization phase. The results show that using this method on fingerprint images with useless components is faster than existing methods and achieves better quality than earlier methods.

Image processing technology is widely used in various consumer electronics products. Design of image filters plays a key role in image processing. And the design is further dependent on the target application. Moving from one application to other, the image filter has to be changed accordingly. Therefore, in this paper, high performance cost effective generalized filter architecture is proposed to handle various kind of noise in the image for applications like LCD display, TV noise etc, and image enhancer and edge detection. Proposed filter architecture is based on the concept of FIR filter and coring technique. Also we introduce a new scaling concept to take care of growing bits during arithmetic operation of filter. The filter architecture is fully pipelined, and implemented in the 90nm technology. The estimated gate count for the proposed filter is 50k, if used for all image components (Y/R, CB/G and CR/B) and the estimated frequency of operation is 500 MHz.

A large number of comb filtering techniques for a national television system committee (NTSC) or phase alternate each line (PAL) color decoding have been researched and developed for the last three decades. Comb filtering can separately obtain the luminance and the quadrature amplitude modulation (QAM) modulated chrominance information from a composite video burst signal (CVBS). However there is a difficulty in extracting the luminance and chrominance components from a composite video image because the cross-talk between them gives undesirable image artifacts. The three-dimensional (3-D) comb filter using spatio-temporal filtering kernel and adaptive two-dimensional (2-D) neural-based comb-filtering approach was developed to alleviate the dot crawl artifacts; however it shows limitation on color decoding. This paper presents an effective dot crawl artifact reduction algorithm in a composite video signal, in which undesirable dot crawl artifact is significantly reduced without losing fine image details. The proposed composite video artifact removal algorithm filters only detected candidate regions specified by dot crawl artifact decision map. The possible comb-filtering error region is generated on video image using luminance and chrominance edge information. Simulation and analysis show that the proposed algorithm with nonlinear bilateral filtering removes efficiently the dot crawl artifacts on composite video image and supports improving further video enhancement techniques.

In general, wavelet coefficients are composed of a few large coefficients and a lot of small ones. Therefore, each wavelet coefficient is efficiently modeled as a random variable of a Bernoulli-Gaussian mixture distribution with unknown parameters. The Bernoulli-Gaussian mixture is composed of the multiplication of the Bernoulli random variable and the Gaussian mixture random variable. In this paper, we propose a denoising algorithm using the Bernoulli-Gaussian mixture model based on sparse characteristics of the wavelet coefficient. The denoising is performed with Bayesian estimation. We present an effective denoising method through simplified parameter estimation for the Bernoulli random variable using a local expected square error. Simulation results showed that our method outperformed the states of the art denoising methods.

When images compressed with block-based compression techniques are transmitted over a noisy channel, unexpected block losses occur. Conventional methods that do not consider edge directions can cause blocked blurring artifacts. In this paper, we present a post-processing-based block recovery scheme using Haar wavelet features. The adaptive selection of neighboring blocks is performed based on the energy of wavelet subbands (EWS) and difference between DC values (DDC). The lost blocks are recovered by linear interpolation in the spatial domain using selected blocks. The method using only EWS performs well for horizontal and vertical edges, but not as well for diagonal edges. Conversely, only using DDC performs well for diagonal edges with the exception of line- or roof-type edge profiles. Therefore, we combine EWS and DDC for better results. The proposed directional recovery method is effective for the strong edge because exploit the varying neighboring blocks adaptively according to the edges and the directional information in the image. The proposed method outperforms the previous methods that used only fixed blocks.

The search for efficient image denoising methods still is an open question. In this paper, we develop a universal algorithm based on grey relational analysis for similar binary image denoising. After taking the differences in spatial distribution between noise and edge into consideration, we select two referential sequences to represent inner off/on pixels, and a group of comparative sequences to stand for the pixels to be processed. Then, by analyzing the grey relational coefficients of the two kinds of sequences, we distinguish edge pixels from non-edge pixels, and reset the non-edge pixels to binary, thereby noise reduced and edges kept. The interest of the method lies in the fact that, without any precedent knowledge, it not only can reduce speckle, salt & pepper and gaussian noise at one time but also provide a trade-off between edge reservation and noise reduction via grey relational threshold. Experimental results show that the method obviously outperforms the three conventional spatial filters: median filter, wiener filter and mean filter. Possible applications include the processing before the recognition of printed or handwritten character, vehicle license plate, and the optimization of scanned binary trademark, engineering drawing and extracted binary watermark.

The extension of H.264/AVC hybrid video coding towards scalable video coding (SVC) using motion-compensated temporal filtering (MCTF) is presented. Utilizing the lifting approach to implement MCTF, the motion compensation features of H.264/AVC can be re-used for the MCTF prediction step and extended in a straightforward way for the MCTF update step. The MCTF extension of H.264/AVC is also incorporated into a video codec that provides SNR, spatial, and (similar to hybrid video coding) temporal scalability. The paper provides a description of these techniques and presents experimental results that validate their efficiency. In addition applications of SVC to video transmission and video surveillance are described.

In most of the existing in-band wavelet video coding schemes, over-complete form of reference bands is used to solve the wavelet shift-variance problem on motion compensated temporal filtering (MCTF) in wavelet domain. It can more or less improve the coding efficiency of the in-band MCTF scheme. However, there is a dilemma always existing when the input video is decomposed into a low-pass band and several high-pass bands. If the temporal filtering in the lowpass band does not use any information from the high-pass bands, the coding efficiency will decrease significantly for the higher resolution reconstructed video. If the temporal filtering in the low-pass band does use data from the high-pass bands, it will bring serious drifting error when the high-pass bands are not available at the decoder. In this paper, for the latter case, we analyze when drifting errors occur and how they will propagate along the lifting structure for decoding at the lower resolution. Based on the analysis, we introduce two new inter coding modes at macroblock level for spatial low-pass band MCTF to make a better trade-off between the lower resolution drifting error and the high resolution coding efficiency. Furthermore, we present the criterion to adaptively select the proper coding mode for each macroblock of spatial low-pass band. The experiment results show that our proposed scheme can dramatically reduce the drifting error about 0.4~2.6dB for different bit rates at low resolution, while for the high resolution, the performance loss is marginal.

In this paper, we propose a low complexity prioritized bit-plane coding scheme to improve the rate-distortion performance of cyclical block coding in MPEG-21 scalable video coding. Specifically, we use a block priority assignment algorithm to firstly transmit the symbols and the blocks with potentially better rate-distortion performance. Different blocks are allowed to be coded unequally in a coding cycle. To avoid transmitting priority overhead, the encoder and the decoder refer to the same context to assign priority. Furthermore, to reduce the complexity, the priority assignment is done by a look-up-table and the coding of each block is controlled by a simple threshold comparison mechanism. Experimental results show that our prioritized bit-plane coding scheme can offer up to 0.5dB PSNR improvement over the cyclical block coding described in the joint scalable verification model (JSVM).

Let the whole region be the whole bit rate range that customers are interested in, and a sub-region be a specific bit rate range. The weighting factor of each sub-region is determined according to customers' interest. A new type of region of interest (ROI) is defined for the SNR scalability as the gap between the coding efficiency of SNR scalability scheme and that of the state-of-the-art single layer coding for a sub-region is a monotonically non-increasing function of its weighting factor. This type of ROI is used as a performance index to design a customer oriented SNR scalability scheme. Our scheme can be used to achieve an optimal customer oriented scalable tradeoff (COST). The profit can thus be maximized.

The shift-variant property of discrete wavelet transform leads to the intrinsic coupling across various spatial subbands during motion aligned temporal filtering. It brings a big challenge how to achieve a better trade-off between subband independency and motion alignment efficiency in the context of providing spatial scalability in 3D wavelet video coding. This paper first investigates the issue of subband coupling in-depth. From the investigations on analysis and synthesis filters, we verify the existence and causes of subband coupling phenomenon and illustrate the subband leakage due to motion shift. Furthermore, we propose a method to measure the strength of this kind of coupling. Based on these investigations, we focus on schemes which preserving most subband coupling relationship and recommend that spatial highpass subbands should not be dropped completely. The issue of rate allocation for spatial highpass subbands is considered in this paper. An error propagation model is proposed to describe the effect of subband coupling in video reconstruction. The synthesis gain of each subband is estimated according to this model and it is finally adopted to guide the rate allocation algorithm. Experimental results have fully demonstrated the promising of the proposed techniques in improving both objective and subjective qualities of low resolution video, especially at middle and high bit rates, for 3D video coding schemes with spatial scalability.

Some of the most powerful schemes for scalable video coding are based on the so-called t+2D paradigm. In these schemes, temporal redundancy is first exploited through a motion-compensated multiresolution decomposition and the resulting temporal subband frames are then generally spatially decomposed with a wavelet transform. Thus, temporal and spatial scalability are achieved, in particular when also the motion information is properly managed between resolution levels. However, spatial wavelet transform may not be the most appropriate for exploiting the spatial redundancy of the detail subbands, since very often the power spectral density of these frames is not as concentrated at low frequencies as in the case of natural images. Recently, we have shown that orthonormal 4-band transforms can provide similar or even better results, especially for textured video sequences. In this paper, we elaborate on this idea and compare several M-band transforms for spatial decomposition of the temporal detail frames. Several M-band filter bank designs are given and lapped transforms are also considered. We show by simulation results that lapped transforms achieve a rate versus distortion performance that is comparable and sometimes better to that of the dyadic biorthogonal 9/7 wavelet transform, for a lower complexity.

A hierarchical structure for sports event classification based on audio and video content analysis is proposed in this paper. Compared to the event classifications in other games, those of cricket are very challenging and yet unexplored. We have successfully solved cricket video classification problem using a six level hierarchical structure. The first level performs event detection based on audio energy and Zero Crossing Rate (ZCR) of short-time audio signal. In the subsequent levels, we classify the events based on video features using a Hidden Markov Model implemented through Dynamic Programming (HMM-DP) using color or motion as a likelihood function. For some of the game-specific decisions, a rule-based classification is also performed. Our proposed hierarchical structure can easily be applied to any other sports. Our results are very promising and we have moved a step forward towards addressing semantic classification problems in general.

Shot boundary detection servers as a preliminary step to structure the content of videos. Up to now, a large number of methods have been proposed. We give a brief overview of previous works with a novel view, focusing on the solutions of the two main disturbances, i.e., abrupt illuminance change and great camera or object motion. Then this paper presents a novel shot boundary detection framework, consisting of three components: fade out/in (abbreviated as FOI) detector, cut detector and gradual transition (abbreviated as GT) detector. The key technique of FOI detector is the recognition of monochrome frames. For cut detection, a second-order difference method is firstly applied to obtain candidate cuts, and then a post-processing procedure is taken to eliminate the false positives. In GT detector, the twin-comparison approach is employed to detect short gradual transition which lasts less than six frames, while for long gradual transition, an improvement of twin-comparison algorithm is designed. Firstly, to effectively reduce the false alarms of quick motion, the lower threshold is self-adaptive to motion feature. Secondly, an FSA (finite state automata) model is adopted to replace the twin-comparison strategy. This framework makes good use of various features and successfully integrates all the modules together. Finally, the system is evaluated on the TRECVID benchmarking platform and the experimental results reveal the effectiveness of our system.

In this paper, we propose a novel statistical method for video analysis and object-oriented structured video representation. Although the existing shot-based approaches facilitate our access to video than the raw video data do, they are still not effective for the semantic level video browsing and retrieval. To represent video structure on semantic levels, the proposed method employs an extended hidden Markov model trained by EM algorithm in order to explore the meaningful objects and the important video structures. According to the principles of the proposed method, important objects and video structure with the consideration of short term statistic as well as long term recurrence can be captured and several interesting video analysis tasks can also be performed. Finally, experiments based on several different videos validate the effectiveness of the proposed approach.

Video shot clustering based on visual feature extraction is an important and challenging task in video browsing and retrieval systems. We propose in this paper a novel method for video shot grouping directly processed in MPEG domain. This method places emphasis on extracting shot features in a way that reflects viewer's actual visual perception. In particular, a spatio-temporal human attention model is constructed. Spatially, each frame is split into two areas, attention region and background, indicating two independent parts of viewer's attention: interest of a dominant motive region and a global impression of the surroundings. Temporally, a weighted color histogram is used to emulate human attention affected by camera motion. With this camera attention weighted strategy, a set of shot similarity measures are constructed. Shot grouping based on these similarities achieve promising results in our experiments on MPEG-7 test videos.

There are growing needs for quick preview of video contents for the purpose of improving accessibility of video archives as well as reducing network traffics. In this paper, a storyboard that contains a user-specified number of keyframes is produced from a given video sequence. It is based on hierarchical cluster analysis of feature vectors that are derived from wavelet coefficients of video frames. Consistent use of extracted feature vectors is the key to avoid a repetition of computationally-intensive parsing of the same video sequence. Experimental results suggest that a significant reduction in computational time is gained by this strategy.

In this paper, we present a new and efficient clustering approach for scene analysis in sports video. This method is generic and does not require any prior domain knowledge. It performs in an unsupervised manner and relies on the scene likeness analysis of the shots in the video. The two most similar shots are merged into the same scene in each iteration. And this procedure is repeated until the merging stop criterion is satisfied. The stop criterion is defined based on a J value which is defined according to the Fisher Discriminant Function. We call this method J-based Scene Clustering. By using this method, the low-level video content representation-shots could be clustered into the midlevel video content representation-scenes, which are useful for high-level sports video content analysis such as playbreak parsing, story units detection, highlights extraction and summarization, etc. Experimental results obtained from various types of broadcast sports videos demonstrate the efficacy of the proposed approach. Moreover, in this paper, we also present a simple application of our scene clustering method to story units detection in periodic sports videos like archery video, diving video and so on. The experimental results are encouraging.

The key technique of 3-D GIS is to realize quick and high-quality 3-D visualization, in which 3-D roaming system based on landform plays an important role. However how to increase efficiency of 3-D roaming engine and process a large amount of landform data is a key problem in 3-D landform roaming system and improper process of the problem would result in tremendous consumption of system resources. Therefore it has become the key of 3-D roaming system design that how to realize high-speed process of distributed data for landform DEM (Digital Elevation Model) and high-speed distributed modulation of various 3-D landform data resources. In the paper we improved the basic ant algorithm and designed the modulation strategy of 3-D GIS landform resources based on the improved ant algorithm. By initially hypothetic road weights σ_i , the change of the information factors in the original algorithm would transform from ▵τ_j to ▵τ_j+σ_i and the weights was decided by 3-D computative capacity of various nodes in network environment. So during the course of initial phase of task assignment, increasing the resource information factors of high task-accomplishing rate and decreasing ones of low accomplishing rate would make load accomplishing rate approach the same value as quickly as possible, then in the later process of task assignment, the load balanced ability of the system was further improved. Experimental results show by improving ant algorithm, our system not only decreases many disadvantage of the traditional ant algorithm, but also like ants looking for food effectively distributes the complicated landform algorithm to many computers to process cooperatively and gains a satisfying search result.

We propose a new algorithm that efficiently fills the unknown regions of an image caused by replacement or removal of part of the foreground. Previous approaches to inpainting and texture synthesis are either fast, but not applicable for small-scale regions, or slow, but fills large regions with good quality, but is slow. The algorithm in this paper is based on the patch-based best-fit searching strategy, in which a partly-known patch is filled by searching the known part of the image for a patch of pixels closely matching the known neighbors. This keeps the linear structure and texture of the image. We found that the exhaustive searching strategies used in previous work are the main source of inefficiency; most matches are located in the neighborhood of the target patch. Inspired by this spatial continuity, we introduce a more efficient searching strategy.

This paper presents a new method which allows the artificial creation of cast shadows in the context of video post-production applications for which video objects are inserted in an original video sequence. The manipulated video objects are assumed to have been previously segmented together with their cast shadows in their original images, and to have been inserted in the edited images. This paper deals with the problem of the artificial creation of the corresponding cast shadows. A reference image which represents the scene illuminated background is assumed to be available. The proposed method is based on a precise shadow model and on the use of the 3-D structure of the scene background. This 3-D information is roughly obtained manually in a pre-processing phase. Then, the illumination conditions, i.e. a shadow model and the light source position are estimated. The shape and texture of the created cast shadows are defined by combining geometrical relations linking the light source, the video objects and the cast shadows, and by applying the inverse shadow model to artificially shade the shadow areas. Experimental results show that it is possible to obtain satisfactory visual results.

This work presents a technique for achieving multiresolution representations of deformable objects used in virtual reality applications. The proposed model may define different levels of detail inside the objects in order to obtain a plausible sensorial feedback answer. The technique here described uses a subdivision scheme based on wavelet transforms and a dynamic model based in a mass-spring representation. The dynamic model allows the simulation of the whole volume behaviour incorporating a skeleton that reduces the number of nodes managed by other existing representations and improves its stability too. The paper presents an analysis of the performance achieved with the proposed model together with some examples used in the tests. The results show that the dynamic behaviour of the model is stable and coherent at different resolution levels.

We present a complete system for the automatic creation of talking head video sequences from text messages. Our system converts the text into MPEG-4 Facial Animation Parameters and synthetic voice. A user selected 3D character will perform lip movements synchronized to the speech data. The 3D models created from a single image vary from realistic people to cartoon characters. A voice selection for different languages and gender as well as a pitch shift component enables a personalization of the animation. The animation can be shown on different displays and devices ranging from 3GPP players on mobile phones to real-time 3D render engines. Therefore, our system can be used in mobile communication for the conversion of regular SMS messages to MMS animations.

Augmented reality (AR) has recently gained significant attention. The previous AR techniques usually need a fiducial marker with known geometry or objects of which the structure can be easily estimated such as cube. Placing a marker in the workspace of the user can be intrusive. To overcome this limitation, we present an AR system using invisible markers which are created/drawn with an infrared (IR) fluorescent pen. Two cameras are used: an IR camera and a visible camera, which are positioned in each side of a cold mirror so that their optical centers coincide with each other. We track the invisible markers using IR camera and visualize AR in the view of visible camera. Additional algorithms are employed for the system to have a reliable performance in the cluttered background. Experimental results are given to demonstrate the viability of the proposed system. As an application of the proposed system, the invisible marker can act as a Vision-Based Identity and Geometry (VBIG) tag, which can significantly extend the functionality of RFID. The invisible tag is the same as RFID in that it is not perceivable while more powerful in that the tag information can be presented to the user by direct projection using a mobile projector or by visualizing AR on the screen of mobile PDA.

Due to the large expansion of heterogeneous networks, the scalability features of next generation video coding schemes have become a very important topic for future developments. These algorithms enable to generate a universal (temporally, spatially and SNR) scalable bit-stream so that the video may be decodable by any kind of device from a mobile phone with narrow bandwidth and small display to a PC with DSL connection and large TFT display. In this paper we present a new approach to improve the SNR scalability for scalable video schemes by combining motion information scalability and texture updates in order to get an optimal trade-off between motion and texture for a given bit-rate. Consequently we can better exploit the redundancy between successive frames than the current strategies based on requantization and/or bit-plane coding.

Scalable video coding (SVC) has a great advantage due to its very easy adaptation to unpredictable bandwidth variations and network conditions. However, in some scalable video coding schemes, either a single prediction is used, which leads to either drift or coding inefficiency, or a different prediction is obtained for each reconstructed version, which leads to add complexity. So, the new techniques have to be developed to improve the performance of the SVC scheme. Wyner-Ziv coding gives us the surprising insight that efficient data compression can also be achieved by exploiting source statistics-partially or wholly-at the decoder only. Based on the Wyner-Ziv framework, Xu and Xiong proposed a Layer Wyner-Ziv scheme (LWZ) by treating a standard coded video as a base layer, and building the bit-plane enhancement layer using Wyner-Ziv coding. The LWZ scheme has error robustness and channel adaptation. However, the coding efficiency gap between the nonscalable video coding and LWZ is very large. The reason is that only the current base layer and lower enhancement layers are used as side information, and the correlation between the temporal adjacent frames isn't utilized. In order to improve the coding efficiency of LWZ, a scalable video coding based on Wyner-Ziv (SVC-WZ) is proposed in this paper. In our SVC-WZ scheme, we try to use the correlation between temporal adjacent frames rather than always use the correlation between the enhancement layers and the current base layer as SI during the enhancement layer encoding. Using the correlation between temporal adjacent frames would make SI more accurate, thus it could improve the coding efficiency. Our experimental results show that the SVC-WZ scheme can achieve consistently better coding efficiency than the LWZ scheme while keeping all the properties of LWZ.

Scalable video coding (SVC) has become more and more important with the enrichment of multimedia data and the diversification of network and terminal devices. In current MPEG SVC activities, a scalable extension of H.264/AVC, called scalable video model (SVM), is proposed by HHI and has shown further coding efficiency improvement and scalability functionality. However, the SVM model doesn't provide an efficient rate control scheme now, and rate control is achieved through a full search for selecting a suitable quantization parameter (QP). That is very inefficient and much time-consuming. In this paper, an efficient rate control scheme is proposed for the SVM, which is derived from the state-of-the-art hybrid rate control schemes of JVT with some considerations for scalable video coding. In the proposed rate control scheme, the rate distortion optimization (RDO) involved in the step of encoding temporal subband pictures is only implemented on the low-pass subband pictures, and rate control is independently applied to each spatial layer. For each spatial layer, the rate control is implemented at GOP, picture and basic unit levels. Furthermore, for the temporal subband pictures obtained from the motion compensation temporal filtering (MCTF), the target bit allocation and quantization parameter selection inside a GOP could make full use of the hierarchical relations inherent from the MCTF. The proposed rate control scheme has been implemented into SVM3.0 and experiment results show that the proposed algorithm can achieve the target bit rate with little bit rate fluctuation and keep fine image quality at the same time, but the computation complexity is reduced heavily.

In this paper a new fully scalable - wavelet based - video coding architecture is proposed, where motion compensated temporal filtered subbands of spatially scaled versions of a video sequence can be used as base layer for inter-scale predictions. These predictions take place between data at the same resolution level without the need of interpolation. The prediction residuals are further transformed by spatial wavelet decompositions. The resulting multi-scale spatiotemporal wavelet subbands are coded thanks to an embedded morphological dilation technique and context based arithmetic coding. Dyadic spatio-temporal scalability and progressive SNR scalability are achieved. Multiple adaptation decoding can be easily implemented without the need of knowing a predefined set of operating points. The proposed coding system allows to compensate some of the typical drawbacks of current wavelet based scalable video coding architectures and shows interesting visual results even when compared with the single operating point video coding standard AVC/H.264.

High Definition Television is currently a hot topic that generates research and commercial interests in video industry. This paper relates to video coding devices and methods for easy and fast conversion of video formats for digital TV. The proposed algorithm provides a generic framework that allows any video format conversions in a scalable way. We focused particularly on solutions to extract Standard Definition video signals from High Definition ones. Practically, we address spatial scalability solutions between video formats that are not linked up by dyadic decomposition and/or istropic decomposition. The challenge is to generate a single scalable bitstream in a compliant way that permits any spatial resolution decodings while saving bandwidth compared to classical simulcast approaches. Practically, our scalable coder is based on a multi-layer approach and on frame scales and borders introduction. Moreover, to encode/decode the macroblocks of the high resolution pictures, usual video coding approaches exploit the inheritance knowing the macroblocks of the decoded low resolution pictures. Approaches for inter-layer prediction of motion data have already been proposed for dyadic decompositions but do not work in case of non dyadic inter-layer resolutions. Consequently we propose a different managing of the inter layer prediction. All these techniques have been successfully implemented in the MPEG-SVC reference software. Obtained results achieve good coding efficiency and are comparable to the simulcast state of the art while providing features induced by scalability.

It is well-known that the problem of addressing heterogeneous networks in multicast can be solved by simultaneous transmission of multiple bitstreams of different bitrates and by layered encoding. This paper analyzes the use of H.264/AVC video coding in simulcast and for layered encoding. The sub-sequence feature of H.264/AVC enables hierarchical temporal scalability, which allows disposal of reference pictures from a coded bitstream without affecting the decoding of the remaining stream. In this paper we extend the scope of the H.264/AVC sub-sequence coding technique to quality scalability. The resulting quality scalable coding technique is similar to conventional coarse-granularity quality scalability but fully compatible with the H.264/AVC standard. It is found that the proposed method drops bitrate consumption in the core network compared to simulcast up to 20%. However, the bitrate required for enhanced-quality reception for scalably coded bitstreams is considerably higher than that of non-scalable bitstreams.

In content based image retrieval similarity measurement is one of the most important aspects in a large image database for efficient search and retrieval to find the best answer for a user query. Color and texture are among the more expressive of the visual features. Considerable work has been done in designing efficient descriptors for these features for applications such as similarity retrieval. The MPEG-7 specifies a standard set of descriptors for color, texture and shape. In the Human Vision System (HVS), visual information is not perceived equally; some information may be more important than other information. The purpose of this paper is to show how the MPEG-7 descriptor based on human vision system can be efficiently utilized for image matching. We considered scalable color descriptor (SCD) and edge histogram descriptor (EHD) descriptors of MPEG7. To increase the matching performance we used Linde-Buzo-Gray (LBG) vector quantization algorithm and image splitting by magnitude weighting technique for efficient use of SCD and EHD, respectively. The proposed matching method is considered to be a more efficient image content-based retrieval than EHD and SCD. Experimental results support this claim. Experiments on 3000 images from Corel Photo collections show that the proposed method yields better retrieval performance especially for semantic similarity.

We present an application of image analysis techniques to automatically annotate biological images depicting gene expression patterns in developing embryos of fruit fly (Drosophila melanogaster), a model organism to study gene interaction. The aim is to determine the view (lateral versus dorsal/ventral [non-lateral]), orientation (anterior-left or anterior-right), and the developmental stage of the embryo. We employed contour curvature analysis, symmetry of the gene expression patterns, and shape differences at the anterior and posterior ends of the embryo, among others, for these purposes. An analysis of a pilot database of 3500 images indicates that view was correctly identified in 62%, orientation in 85%, and developmental stage in 73% of the images. We observed that correct inferences had better separation in feature space than incorrect inferences. This means that, although these methods do not exhibit very high classification accuracy, they could be employed to identify images which need manual intervention, thereby reducing the target set for biologists. The novelty in this work is in the integration of well-established image analysis with the biological knowledge for annotating the embryos. Our examinations show that features that provide discrimination ability among different views, different orientations, and different developmental stages are often restricted to certain regions of the embryo, which agrees with the longstanding knowledge in the developmental biological community.

This paper presents a fingerprinting method based on equivalence classes. An equivalence class is composed of a reference image and all its variations (or replicas). For each reference image, a decision function is built. The latter determines if a given image belongs to its corresponding equivalence class. This function is built in three steps: synthesis, projection, and analysis. In the first step, the reference image is replicated using different image operators (like JPEG compression, average filtering, etc). During the projection step, the replicas are projected onto a distance space. In the final step, the distance space is analyzed, using machine learning algorithms, and the decision function is built. In this study, three machine learning approaches are compared: orthotope, support vectors machine (SVM), and support vectors data description (SVDD). The orthotope is a computationally efficient ad-hoc method. It consists in building a generalized rectangle in the distance space. The SVM and SVDD are two more general learning algorithms.

In this paper a novel image classification system is proposed. Atmosphere serves an important role in generating the scene's topic or in conveying the message behind the scene's story, which belongs to abstract attribute level in semantic levels. At first, five atmosphere semantic categories are defined according to rules of photo and film grammar, followed by global luminance and hue features. Then the hierarchical SVM classifiers are applied. In each classification stage, corresponding features are extracted and the trained linear SVM is implemented, resulting in two classes. After three stages of classification, five atmosphere categories are obtained. At last, the text annotation of the atmosphere semantics and the corresponding features by Extensible Markup Language (XML) in MPEG-7 is defined, which can be integrated into more multimedia applications (such as searching, indexing and accessing of multimedia content). The experiment is performed on Corel images and film frames. The classification results prove the effectiveness of the definition of atmosphere semantic classes and the corresponding features.

In this paper, separated feature indexing was used for region-based image retrieval. In the beginning, three methods, such as K-means, CA-Clustering and Blobworld, were used for region generation in this paper. Comparing their results, the retrieval results using Blobworld perform better than the other two. Therefore, Blobworld is chosen to generate regions in the following experiments. After generating a few regions in an image, in this paper, a few heterogeneous features, such as color signature, moments, contrast, anisotropy, edge orientation histogram, edge density, perimeter, area, position, etc. were used for region matching. According to the variations of two or more query regions, the weights to each feature were automatically generated. Furthermore, the individual indexing was built and maintained for each heterogeneous feature, i.e., separated feature indexing. Once the retrieval results from separated feature indexing were obtained, two processes about normalization and combining distances from these heterogeneous features should be completed in order to achieve the final retrieval result. In our experiments, two functions were tested for normalizing heterogeneous distances and four functions for combining the normalized distances into a global measure. The experimental results show that the best result can be obtained by using the one based on norm1 and mean2 operations. We used the precision/recall and the average normalized modified retrieval rank (ANMRR) to evaluate the experiments in this paper.

A new fast template matching algorithm is proposed in this work. It extracts the local self-similarity as image feature. Feature matching is based on equal comparison and clustering. Its time complexity reduced to 0(size of search image), the theoretic lower limit for classical image template matching. The new algorithm is generalized to partial matching of two images. By partitioning images into blocks by nona-tree decomposition, it detects the matching regions between blocks and merges the matching results using first order Markov chain in horizontal direction. In the vertical direction merging is implemented directly according neighbor parts connected and keeping the consistent displacements of correspond parts in two image. The algorithm is robust for the linear transformation of image grey value and image noise.

Standard video compression techniques apply motion-compensated prediction combined with transform coding of the prediction error. In the context of prediction with fractional-pel motion vector resolution it was shown, that aliasing components contained in an image signal are limiting the prediction accuracy obtained by motion compensation. In order to consider aliasing, quantisation and motion estimation errors, camera noise, etc., we analytically developed a two-dimensional (2D) non-separable interpolation filter, which is calculated for each frame independently by minimising the prediction error energy. For every fractional-pel position to be interpolated, an individual set of 2D filter coe±cients is determined. Since transmitting filter coefficients as side information results in an additional bit rate, which is almost independent for different total bit rates and image resolutions, the overall gain decreases when total bit rates decrease. In this paper we present an algorithm, which regards the non-separable two-dimensional filter as a polyphase filter. For each frame, predicting the interpolation filter impulse response through evaluation of the polyphase filter, we only have to encode the filter coefficients prediction error. This enables bit rate savings, needed for transmitting filter coe±cients of up to 75% compared to PCM coding. A coding gain of up to 1,2 dB Y-PSNR at same bit rate or up to 30% reduction of bit rate is obtained for HDTV-sequences compared to the standard H.264/AVC. Up to 0,5 dB (up to 10% bit rate reduction) are achieved for CIF-sequences.

An improved intra-prediction method for AVS-M, which is developed by AVS (Audio Video Coding Standard Working Group of China) to meet the requirements of multimedia applications in next-generation mobile communications, is proposed in this paper. Utilizing the spatial and temporal correlation of the video sequences, the proposed method only processes a portion of normative 9 intra-prediction modes to decide the best mode, therefore, speeds up intra prediction with some degradation in the coding gain. Moreover, a complexity-scalable technique is presented to control the complexity of intra prediction, by which the video coder is matched to the limited computational resource of communication devices. The proposed methods are verified. The results show that the improved mode-decision method can significantly save encoding time at expense of little coding loss and the complexity-scalable technique works robustly and accurately. In addition, the proposed method also can be applied to other standards, such as H.264.

MPEG-4 supports object-level video coding. It is a challenge to design an optimal bit allocation strategy which considers not only how to distribute bits among multiple video objects (MVO's) but also how to achieve optimization between the texture and shape information. In this paper, we present a uniform framework for optimal multiple video object bit allocation in MPEG-4. We combine the rate-distortion (R-D) models for the texture and shape information of arbitrarily shaped video objects to develop the joint texture-shape rate-distortion models. The dynamic programming (DP) technique is applied to optimize the bit allocation for the multiple video objects. The simulation results demonstrate that the proposed joint texture-shape optimization algorithm outperforms the MPEG-4 verification model on the decoded picture quality.

Recursive wavelet filters and an alternative algorithm for implementing wavelet transform are presented in this paper. The recursive filters use previously calculated (past) wavelet coefficients as inputs to calculate the current wavelet coefficient, and provide the same transform results as convolutional FIR and lifting wavelet filters. The coefficients of the recursive filters are derived from those of conventional FIR wavelet filters. The wavelet transform with recursive filters requires a smaller amount of memory and is easy to implement in hardware. Another important advantage of the recursive filters is that perfect reconstruction can be easily achieved using recursive wavelet filters if a sequence of pixels to be transformed is extended by boundary pixel repetition. Boundary pixel repetition can be more efficient than the widely used method of symmetric extension for image and video coding.

Residual operations are introduced in many video compression algorithms to exploit temporal redundancy in video sequences. One of the most effective and computationally simple algorithms is 3D-DWT-SPIHT algorithm. Nevertheless, it only utilizes intra-GOF temporal redundancy. In order to eliminate inter-GOF redundancy, the Simple GOF Residual Operation (SGRO) algorithm was introduced, where residual operations are performed every two GOFs. However, when background mutations occur, the PSNR of reconstructed target GOF significantly drops; on the other hand, when video sequences are temporally stationary, SGRO algorithm fails to fully utilize inter-GOF temporal redundancy due to its imperative insertion of no-residual-operation GOFs. In this paper, we propose a new Adaptive GOF Residual Operation (AGRO) algorithm, based upon a criterion on residual operations, which is derived from an empirical formula of image complexity established by us. AGRO algorithm always selects the best residual operation manner according to the contents of video sequences: by detecting contents of video sequences, it cancels residual operations where background mutations happen, while encouraging residual operations where video sequences are temporally stationary. Therefore, AGRO algorithm prevents the significant drop in compression effects resulted from background mutations, and in the meantime, fully utilizes inter-GOF temporal redundancy. In addition, AGRO algorithm demonstrates an innate error-propagation-resistant property. Numerical results show that AGRO algorithm renders a significant PSNR increase over SGRO algorithm whenever background mutations occur or temporally stationary sequences dominate.

Compared to contemporary video coding standards, the H.264 standard achieves higher compression efficiency at the expense of high computational complexity at the encoder to carry out rate distortion optimization (RDO). In this paper, we investigated the encoding speed improvement technology of H.264 and focus on RDO to select the encode mode for fast coding of H.264. Based on the analysis of motion activity information of the macroblock to be coded in inter frame, we proposed a methodology to simplify the computation of encoder while maintaining the coding efficiency. At first, we established the relationships between INTRA mode and motion activity in inter frame. Then the INTRA mode skipping algorithm (IMSA) is proposed and applied to RDO procedure. If the motion activity of a macroblock is small enough, then IMSA adaptively skips the whole INTRA mode decision process because the encoder benefits little from such type of macroblocks. Simulation results demonstrate its valuable efficiency of considerably reducing 30%~60% encoding complexity while incurring little even no loss in quality and increase in bit rate. Additionally, it is easier than other published fast mode decision algorithms and more efficient when working together with them.

Audio Video coding Standard (AVS) is established by the Working Group of China in the same name. AVS-video is an application driven coding standard. AVS Part 2 targets to high-definition digital video broadcasting and high-density storage media and AVS Part 7 targets to low complexity, low picture resolution mobility applications. Integer transform, intra and inter-picture prediction, in-loop deblocking filter and context-based two dimensional variable length coding are the major compression tools in AVS-video, which are well-tuned for target applications. It achieves similar performance to H.264/AVC with lower cost.

Conventional video coding techniques make use of the most recently decoded reference frame(s) for motioncompensated inter prediction. However, it has been shown that to allow using reference frames in a flexible way such that not only the latest reference frames are used is beneficial. A typical use of flexible reference frame is feedback based reference picture selection, wherein error-free reference frames available in both the encoder and decoder sides are selected and used for inter prediction reference. This paper first overviews support of reference picture selection in different video coding standards, and then presents three specific feedback based reference picture selection methods using flexible reference frames. In addition, a novel simple reference frame management method that enables using of flexible reference frame is proposed. The reference frame management method enables much simpler video codec implementations compared to the complex reference frame management methods in H.263 Annex U and H.264/AVC. The proposed coding methods and some conventional methods are compared with each other. Simulation results show significantly improved error resiliency performance of the proposed reference picture selection methods compared to conventional methods. The effect on the performance imposed by feedback delay variation is also shown. Thanks to the merits, support of flexible reference frame and the reference frame management has been adopted to the AVS-M video coding standard.

AVS is a new digital audio-video coding standard established by China. AVS will be used in digital TV broadcasting and next general optical disk. AVS adopted many digital audio-video coding techniques developed by Chinese company and universities in recent years, it has very low complexity compared to H.264, and AVS will charge very low royalty fee through one-step license including all AVS tools. So AVS is a good and competitive candidate for Chinese DTV and next generation optical disk. In addition, Chinese government has published a plan for satellite TV signal directly to home(DTH) and a telecommunication satellite named as SINO 2 will be launched in 2006. AVS will be also one of the best hopeful candidates of audio-video coding standard on satellite signal transmission.

The new video coding standard known as H.264/MPEG-4 Advanced Video Coding (AVC), now in its fourth version, has demonstrated significant achievements in terms of coding efficiency, robustness to a variety of network channels and conditions, and breadth of application. The recent fidelity range extensions have further improved compression quality and further broadened the range of applications, and the recent corrigenda have excised the inevitable errata of the initially-approved versions of the specification. Patent licensing programs have begun, the standard has been adopted into a variety of application specifications, and products suitable for widespread deployment have begun to appear. New work toward the near-term development of scalable video coding (SVC) extensions is also under way. This paper does not attempt to review the details of the H.264/MPEG-4 AVC technical design, as that subject has been covered already in a number of publications. Instead, it covers only the high-level design characteristics and focuses more on the recent developments in the standardization community and the deployment status of the specification.

VC-1 is a state-of-the-art video compression format that provides highly competitive video quality, from very low through very high bit rates, at reasonable computational complexity. The technology includes several innovations to improve the rate-distortion performance envelope of the format and to provide additional tools for perceptual quality improvement. This paper describes some of the key innovations of VC-1 that contributes to its quality and complexity advantages. The focus of this paper is on those tools which are unique to VC-1, and differentiate it from other video compression standards.

With the increasing use of digital imaging in general consumer applications, there is a great deal of interest in developing new products that increase the value and enjoyment level of viewing digital images in consumers' living rooms. One way to enrich image viewing and sharing is to combine images with voice annotation and music. A picture VCD system (PVCD) was developed for multimedia authoring, centered around and driven by still photos, with an emphasis on composing still images with sound, including music and spoken annotations. We describe the overall system, as well as major enabling technology components, including multimedia authoring, semantic image classification, and cross-media indexing. The finished multimedia bit stream is primarily recorded on DVD or VCD to facilitate enriched enjoyment through a TV set, but can also be shared on a desktop/laptop, via email, or online.

Video motion analysis and its applications have been a classic research topic for decades. In this paper, we explore the problem of real time video semantics understanding based on motion information. The work can be divided into two segments: global / camera motion estimation and object motion analysis. The former involves optical flow analysis and semantic meaning parsing, and the latter involves object detection and tracking. Although each of these topics has been studied extensively in the literature, a thorough system combining all of them without human intervention, especially under a real time application scenario, is still worthy of further investigation. In this paper we develop our approach toward such a destination and propose an integral architecture. The usability and efficiency of the proposed system have been demonstrated through experiments. Results of this project have numerous applications in digital entertainment, such as video and image summarization, annotation, retrieval and editing.

We present a framework to detect the regions of singing voice in musical audio signals. This work is oriented towards the development of a robust transcriber of lyrics for karaoke applications. The technique leverages on a combination of low-level audio features and higher level musical knowledge of rhythm and tonality. Musical knowledge of the key is used to create a song-specific filterbank to attenuate the presence of the pitched musical instruments. This is followed by subband processing of the audio to detect the musical octaves in which the vocals are present. Text processing is employed to approximate the duration of the sung passages using freely available lyrics. This is used to obtain a dynamic threshold for vocal/ non-vocal segmentation. This pairing of audio and text processing helps create a more accurate system. Experimental evaluation on a small database of popular songs shows the validity of the proposed approach. Holistic and per-component evaluation of the system is conducted and various improvements are discussed.

With the proliferation of camera equipped cell phones and the deployment of the higher data rate 2.5G and 3G infra structure systems, providing consumers with video-equipped cellular communication infrastructure is highly desirable, and can drive the development of a large number of valuable applications. However, for an uplink wireless channel, both the bandwidth and battery energy in a mobile phone are limited for video communications. In this paper, we pursue an energy efficient video communication solution through joint video summarization and transmission adaptation over a slow fading wireless channel. Coding and modulation schemes and packet transmission strategy are optimized and adapted to the unique packet arrival and delay characteristics of the video summaries. In additional to the optimal solution, we also propose a heuristic solution that is greedy but has close to optimal performance. Operational energy efficiency-summary distortion performance is characterized under an optimal summarization setting. Simulation results show the advantage of the proposed scheme with respect to energy efficiency and video transmission quality.

Home video collections constitute an important source of content to be experienced within the digital entertainment context. To make such content easy to access and reuse, various video analysis technologies have been researched and developed to extract video assets for management tasks, including video shot/scene detection, keyframe extraction, and video skimming/summarization. However, one less addressed issue is to investigate how useful those assets are in helping consumers managing their video collections and the usage pattern of the assets. In this paper, we present Personal Video Manager, both as a home video management system and an explorative research platform to enable a systematic analysis and understanding of consumers' demand on video assets and video processing technologies. For understanding consumer's interest, PVM adopts database management technologies to model and archive how consumers identify video assets and utilize them for management tasks. The PVM mining engine performs data mining on such archived data to mine useful knowledge of consumer's preference on video assets and behavior on utilizing the assets. As revealed in the experiment, consumer's interaction embeds rich information to be leveraged in developing more effective video analysis technologies.

In this paper, an efficient method for human facial expression recognition is presented. Our method includes two major techniques: spatially maximum occurrence model (SMOM), which is used to describe the different facial expressions; and elastic shape-texture matching (ESTM), which is used to compute the similarity between two images. The combination of these two techniques, namely the SMOM-ESTM method, is used to classify the facial expressions. With our approach, the recognition rate based on the AR database is 94.5%.

This paper presents a complete functional system capable of detecting people and tracking their motion in either live camera feed or pre-recorded video sequences. The system consists of two main modules, namely the detection and tracking modules. Automatic detection aims at locating human faces and is based on fusion of color and feature-based information. Thus, it is capable of handling faces in different orientations and poses (frontal, profile, intermediate). To avoid false detections, a number of decision criteria are employed. Tracking is performed using a variant of the well-known Kanade- Lucas-Tomasi tracker, while occlusion is handled through a re-detection stage. Manual intervention is allowed to assist both modules if required. In manual mode, the system can track any object of interest, so long as there are enough features to track.

Non-frontal illumination of objects may cause specular reflections and strong self-shadowing. Those phenomena change the appearance of objects to such an extent that they may not be recognized properly. We propose a method to automatically discard the areas of the image which are degraded beyond recovery by adverse illumination conditions. The method is based on a comparison between local variances of image gradient and is computationally efficient. We show that proposed method, implemented with a face verification system based on local DCTmod2 features and a GMM classifier, reduces total recognition errors in the presence of changing directional illumination conditions. Consequently, we show that proposed segmentation method can be used as an automatic estimator of mismatch between the illumination conditions present during the acquisition of training and testing images. We propose an adaptive thresholding scheme that uses the mismatch estimate to further reduce the recognition error.

In this paper, a real-time method is proposed as a solution to the problem of facial expression classiffication in video sequences. The user manually places some of the Candide grid nodes to the face depicted at the first frame. The grid adaptation system, based on deformable models, tracks the entire Candide grid as the facial expression evolves through time, thus producing a grid that corresponds to the greatest intensity of the facial expression, as shown at the last frame. Certain points that are involved into creating the Facial Action Units movements are selected. Their geometrical displacement information, de ned as the coordinates' difference between the last and the first frame, is extracted to be the input to a six class Support Vector Machine system. The output of the system is the facial expression recognized. The proposed real-time system, recognizes the 6 basic facial expressions with an approximately 98% accuracy.

A novel technique that can implement face detection directly in the wavelet compressed domain is presented in this paper. The algorithm takes the entropy decoding and inverse quantized wavelet transform coefficients of JPEG2000 picture as input, and outputs the locations of the detected faces. The main contribution of this work is in proposing a multi-level gradient energy representation of face pattern based on wavelet compressed data, which permits pertinent high contrast facial parts, such as eyes, nose and mouth, to be highlighted in a compact mode no matter the face is big or small. A neural-network based classifier is designed to decide a gradient energy pattern as face or non-face. In contrast to the traditional spatial-domain techniques, the proposed compressed domain technique eliminates the unnecessary decompression step and thus has lower computational complexity. Moreover, compared with the previous methods based on DCT compressed domain, the proposed multi-level gradient energy presentation removes the complex spatial scaling operation in compressed domain and overcomes block quantization problem. Based on test results of a variety of pictures, the presented algorithm was found to be more efficient and effective than the previous related methods.

The illumination changes on face images make face recognition a very difficult task. In this paper, a human face representation scheme that is insensitive to illumination variation is proposed in order to deal with the problem. The variations in lighting over human faces are modeled by means of Principal Component Analysis (PCA) on a number of blurred faces under different lighting conditions. Then the 'difference image', which is the difference between the original image and the reconstructed image, is used for face recognition. We also propose an uncorrelated Linear Discriminant Analysis technique for face recognition based on the eigen-illumination representation scheme. This method can obtain the uncorrelated optimal discriminant vectors (UODVs) so that the extracted features are uncorrelated. Experimental results show that the proposed method is effective to deal with varying illimunation problem for face recognition.

Based on recent works on Wyner-Ziv coding (or lossy source coding with decoder side information), we consider the case with noisy channel and addresses distributed joint source-channel coding, while targeting at the important application of scalable video transmission over wireless networks. In Wyner-Ziv coding, after quantization, Slepian-Wolf coding (SWC) is used to reduce the rate. SWC is traditionally realized by sending syndromes of a linear channel code. Since syndromes of the channel code can only compress but cannot protect, for transmission over noisy channels, additional error protection is needed. However, instead of using one channel code for SWC and one for error protection, our idea is to use a single channel code to achieve both compression and protection. We replace the traditional syndrome-based SWC scheme by the parity-based one, where only parity bits of the Slepian-Wolf channel code are sent. If the amount of transmitted parity bits increases above the Slepian-Wolf limit, the added redundancy is exploited to cope against the noise in the transmission channel. Using IRA codes for practical parity-based SWC, we design a novel layered Wyner-Ziv video coder which is robust to channel failures and thus very suitable for wireless communications. Our simulation results show great advantages of the proposed solution based on joint source-channel coding compared to the traditional approach where source and channel coding are performed separately.

The JPEG2000 image coding standard defines two kinds of region of interest (ROI) compression methods--the general scaling based method and the maximum shift (maxshift) method. There are two major drawbacks of the two methods. First, they would significantly reduce the compression efficiency by increasing the dynamic range (or number of bitplanes) of wavelet coefficients. Second, they do not have the especial protection for the ROI against the bit errors in image communication application, i.e., remote medical treatment and Satellite Communication. When the bit errors occurred, the reconstruction quality of ROI and BG would reduce at the same time. In this paper, a new and flexible ROI image compression method is proposed according to the Wyner-Ziv theorem on source coding with side information. The reconstructed low quality ROI is regard as a noisy version of original ROI in the proposed method. So, along with the parity bits from the original ROI, the reconstructed low quality ROI as side information can be utilized to Turbo decoder to decode the high quality ROI. Our experimental results show that the proposed method applying the Wyner-Ziv coding to ROI compression can highly improve the compression efficiency as well as efficiently protect the ROI against the bit errors.

In distributed video coding, the complexity of the video encoder is reduced at the cost of a more complex video decoder. Using the principles of Slepian andWolf, video compression is then carried out using channel coding principles, under the assumption that the video decoder can temporally predict side-information that is correlated with the source video frames. In recent work on distributed video coding the application of turbo codes has been studied. Turbo codes perform well in typical (tele-)communications settings. However, in distributed video coding the dependency channel between source and side-information is inherently non-stationary, for instance due to occluded regions in the video frames. In this paper, we study the modeling of the virtual dependency channel, as well as the consequences of incorrect model assumptions on the turbo decoding process. We observe a strong dependency of the performance of the distributed video decoder on the model of the dependency channel.

There are several problems in video coding that can not be adequately addressed through the use of conventional source coding techniques. For instance, any video coding system that is based on a predictive framework will suffer from performance degradations if there is uncertainty in the predictions. In this paper, we propose a method that incorporates binning codes and channel decoding to address video coding problems that have multiple possible predictions at the decoder. We will propose solutions based on binning codes to three separate problems in video coding and show that the results are superior to pre-existing methods.

Motion Compensated Temporal Filtering (MCTF) has proved to be an efficient coding tool in the design of open-loop scalable video codecs. In this paper we propose a MCTF video coding scheme based on lifting where the prediction step is implemented using PRISM (Power efficient, Robust, hIgh compression Syndrome-based Multimedia coding), a video coding framework built on distributed source coding principles. We study the effect of integrating the update step at the encoder or at the decoder side. We show that the latter approach allows to improve the quality of the side information exploited during decoding. We present the analytical results obtained by modeling the video signal along the motion trajectories as a first order auto-regressive process. We show that the update step at the decoder allows to half the contribution of the quantization noise. We also include experimental results with real video data that demonstrate the potential of this approach when the video sequences are coded at low bitrates.

Both digital radioscopy and radiographic testing rely on human experts to perform manual interpretation of images, and the recognition of welding defects demands the inspector's vast experience. With computer detect defects in a weld image; many countries have been made seeking the development of automatic system of inspection of welding defects. Defects were detected and segmented is key step for automatic recognition from welding image. This paper presents an algorithm of defects detection and segmentation for weld radiographic inspection which is based on wavelet analysis. The detection algorithm based on waveform analysis classifies weld defects as trough-anomaly, peak-anomaly and slanttrough- anomaly three types firstly. Aiming at to trough-anomaly and slant-trough-anomaly, weld defects inside them can be classified as crack and non-crack according to the width of two peaks in following. Different segmentation algorithms are adopted to these two types defects sequently. The experimental results show that the algorithm is very effective.

The text in a video frame can help us to understand the semantics of video content directly. Although there are many approaches that can automatically detect and localize text a video, most of them use the original pixels of an image to find the text regions. In this paper, we present an approach to automatically localize captions in MPEG compressed videos. Caption regions are segmented from background by using their distinguishing texture characteristics. Unlike previously published ones which fully decompress the video sequence before extracting the caption regions or only extract text regions in Intra-(I-) frames, our approach detect and localize caption regions directly in the DCT compressed domain. Therefore, only very small amounts of decoding processes are required. Experiments show that a good caption detection rate can be obtained, and the average recalls of Intra- and Inter-frame detections are 97.77% and 97.84%, respectively.

In this paper, a novel method is proposed to locate one-dimensional barcodes in JPEG 2000 images. The barcode locating system consists of two parts: candidate barcode location detection, barcode location verification and refinement. Both parts are designed to work in the compressed domain of JPEG 2000 images. The locations of candidate barcodes are extracted from the header data and verified by examining part of the decoded coefficients of the JPEG 2000 file. Since only a small amount of the compressed data is used, this algorithm has a low complexity relative to algorithms which use all of the pixel data.

In this paper, we introduce a real time vision-based human motion capturing system using two CCD cameras. For realtime processing consideration, we propose some simple but effective methods to estimate the motion parameters (BAPs) of the human object. Our method analyzes the vertical projection profile and the horizontal projection profile in each view to identify different arm and leg postures. With the identified postures, we can apply the Kalman filter to track the motion parameters (joint angels). Our motion analysis method is divided into macro motion analysis and micro motion analysis. The former identifies certain well-defined postures and the latter traces the variation of joint angle or BAPs.

Label location and recognition has become a crucial task for today's Unmanned Aerial Vehicles. We proposed a morphology-based algorithm to locate and recognize labels. This algorithm is insensitive to scaling and rotation, and able to work at low resolution. The label positioning and recognition strategy we designed is divided into two steps. First, at the altitude of 10m or so, we apply dilation processing and edge detection on the images sent back by UAV. Then combining the current heading information of the vehicle, we are able to give the topology map of all labels. After that the vehicle is lowered to about 5m and we apply erosion processing on the returned image and then recognize each label using image measurement and image analysis methods. The validity of this algorithm is well verified at ARCC 2004.

The automatic target detection under natural backgrounds is an important topic in the field of automatic target recognition. Fractal dimension is generally related to the roughness of the surface. The fractal dimension of the man-made object is usually lower than the background's because mostly it is smoother than natural background. This feature can be used to detect the target automatically. In the computing of the fractal dimension, the irregular values often appear at the boundary of the different textures in the image. This phenomenon can be called 'edge effect'. It may result in the difficulty in the followed image processing such as thresholding and cluster segmentation. The main reason of the edge effect is the same contribution of the every pixel in the neighborhood of the pixel where the fractal dimension being calculated. In this paper, in order to weaken the 'edge effect' in the fractal dimension computation, a 2-D Parzon window is designed. The accuracy of fractal dimension calculated after multiplied by the Parzon window is discussed, and a new algorithm is proposed to apply in the automatic target detection. The proposed automatic target detection algorithm is adopted in the experiments in images under complex land or sea backgrounds. The correctly detection rate is above 95%. The robust of this algorithm is represented in the cases of the variety of the light, rotation, size changing and occlusion of the target.

We investigate and compare combined compression-encryption schemes. We assess the respective security, and we show how encryption affects the image coding efficiency. The techniques employ the wavelet-based compression algorithms JPEG2000 and SPIHT, and we randomly permute and rotate blocks of wavelet coefficients in different wavelet-subbands to encrypt image data within the compression pipeline. We identify weak points of the propsed encryption techniques, and possible attacks are highlighted. The investigated methods allow to trade off security for compression performance. The results also have interesting implications with respect to the significance of the zerotree-hypothesis as stated in the context of compression schemes.

VLC codeword concatenations are not secured in most video encryption algorithms because direct encryption of them will usually not result in valid VLC codeword concatenations, and then the encrypted video stream does not conform to standard format. So how to secure VLC codewords has become a difficult issue in video encryption. This paper presents a light-weight approach to solve this problem. In the proposed scheme, the VLC codewords of TCOEF (Transform coefficient) and MV (motion vector) are randomly substituted with other legal codeword. The substitute codewords are from the standard VLC table, thus the encrypted video stream is format-compliant as an ordinary compressed stream that can be decoded by a conventional decoder. In addition, our scheme not only encrypts the sigh bits of TCOEF and MV, but also encrypts INTRADC codewords (DC coefficients for INTRA blocks) using secret key cryptography. This can also disguise the video content more and still conform to the standard format. Therefore, the security of this scheme is significantly enhanced while the complexity overhead is slightly increased. The experimental results show that video stream can be encrypted in real-time and the video content is fully disguised.

This paper presents a pay-video scheme that manages video stream, key stream and payment stream efficiently. In our scheme, the owner segments a video into fragments and encrypts them with independent keys. The keys are generated with a novel concept called as hash interval, where each hash interval discloses a range of numbers without disclosing any information on numbers outside of the range. The video fragments are then broadcast on one or more channels. A buyer can purchase the keys to decrypt any fragments and, within each fragment, any desired quality level. The accompanying payment protocol is integrated with the key management protocol seamlessly and hence the computation cost is very low.

Fingerprinting schemes use digital watermarks to determine originators of unauthorized/pirated copies. Multiple users may collude and collectively escape identification by creating an average or median of their individually watermarked copies. We present a codebook generation scheme to enhance the Anti-Collusion fingerprinting (AND-ACC) scheme to make it scalable for supporting large number of users. We evaluated our proposed codebook generation scheme using HVS based watermarking scheme and our own fingerprints. The simulation results with standard test images show good collusion detection performance over average and median collusion attacks.

Digital image forgery detection is becoming increasing important. In recently 2 years, a new upsurge has been started to study direct detection methods, which utilize the hardware features of digital cameras. Such features may be weakened or lost once tampered, or they may not be consistent if synthesizing several images into a single one. This manuscript first clarifies the concept of trueness of digital images and summarizes these methods with their crack by a general model. The recently proposed EM algorithm plus Fourier transform that checks the Color Filter Array (CFA) interpolation statistical feature (ISF) is taken as a case study. We propose 3 methods to recover the CFA-ISF of a fake image: (1) artificial CFA interpolation (2) a linear CFA-ISF recovery model with optimal uniform measure (3) a quadratic CFA-ISF recovery model with least square measure. A software prototype CFA-ISF Indicator & Adjustor integrating the detection and anti-detection algorithms is developed and shown. Experiments under our product validate the effectiveness of our methods.

When transmitted over unreliable channels, the compressed video can suffer severe degradation. Some strategies were employed to make an acceptable quality of the decoded image sequence. Error concealment (EC) technique is one of effective approaches to diminish the quality degradation. A number of EC algorithms have been developed to combat the transmission errors for MPEG-coded video. These methods always work well to reconstruct the smooth or regular damaged macroblocks. However, for damaged macroblocks were irregular or high-detail, the reconstruction may follow noticeable blurring consequence or not match well with the surrounding macroblocks. This paper proposes an edgebased temporal EC model to conceal the errors. In the proposed method, both the spatial and the temporal contextual features in compressed video are measured by using an edge detector, i.e. Sobel operator. The edge information surrounding a damaged macroblock is utilized to estimate the lost motion vectors based on the boundary matching technique. Next, the estimated motion vectors are used to reconstruct the damaged macroblock by exploiting the information in reference frames. In comparison with traditional EC algorithms, the proposed method provides a significant improvement on both objective peak signal-to-noise ratio (PSNR) measurement and subjective visual quality of MPEG-coded video.

CABAC is one of the main entropy coding methods in the H.264 video compression standard. As a binary arithmetic coding, CABAC can achieve extremely high compression efficiency, but it is sensitive to channel errors. After analyzing H.264 CABAC framework, an algorithm is proposed to integrate error detection into CABAC. A forbidden symbol is introduced into the coding alphabet and is allocated a small probability. Whenever this redundant interval is observed in the decoder output, an error is detected. Mathematical analysis shows that the distribution of error detection rate approximates geometric probability with the redundancy as its parameter. A small amount of extra redundancy can be very effective in detecting errors very quickly, and the compression efficiency of CABAC will not be noticeably undermined. The value of redundancy can easily be adjusted through a single parameter to suit the error characteristics of the channels in real implementations. Some useful information about the position of the error is also obtained through this error detection scheme, which can substantially improve the efficiency of succeeding error resilience processing. Experimental results confirm these conclusions.

In recent years, various spatial error concealment techniques have been proposed only by partially considering the four following factors: 1) image continuity, 2) edge preservation, 3) texture recovery and 4) concealment complexity. Therefore, either the results of edge and texture recovery are unsatisfactory or the computation loan is too heavy to be acceptable. Aiming to overcome the above problems, a strap-based (strap means a set of consecutive macroblocks in horizon) framework, instead of block-based framework, is introduced in this paper. Within this framework, the content of each corrupted strap is classified into four classes: smooth area, edge area, low detail area and high detail area. Then a suitable method is selected for each class. Briefly speaking, bilinear interpolation, directional interpolation, best neighborhood match and Markov Random Field (MRF) model-based maximum a posterior (MAP) estimation are employed to conceal the above classes. During content classification, the gradient information of those pixels near the corrupted strap is calculated and presented as gradient points on a unit circle. The centroid, scatter degree and average gradient magnitude of those gradient points are calculated and used to classify the corrupted content. The results of our experiments demonstrate the efficiency of the proposed method and the impressive improvements in both objective and subjective measures have been achieved.

A novel Multiple Description Coding method, named Packeted Multiple Description Coding (PMDC) is presented in this paper. First, we separate the wavelet transformed image into several sub-images and code each sub-image by a method called Unequal Loss Protected Multiple Description Subband Coding (ULPMDC). With this algorithm, we can first form two sub-descriptions, which contain packets from the image, and then we transmit them by two different channels. If no more than one sub-description per sub-image is lost, the proposed algorithm will get very good reconstruction quality. If more sub-descriptions per sub-image are lost, the reconstruction quality is severely degraded. To provide better quality of the reconstructed image, we then modify the packetizing process, and propose a novel packetized MDC framework, named Cross Packetized Multiple Description Coding (CPMDC). In this method, instead of applying ULPMDC on sub-images, we first code the whole image into two descriptions. Then we split each description horizontally and vertically to form row packets and column packets, respectively. The simulation results show that if only some row packets or column packets are lost, the reconstruction quality is still as good as the former scheme. If we lose both columns and rows, CPMDC will still be better than PTSQ.

In this paper, we present a novel data partitioning method, based on motion estimation, aiming at preventing error propagation when transmitting video streaming over wireless networks. Macroblock is treated as basic unit in this method, and all macroblocks of a coded frame are differentiated into different levels of importance according to their own impact factor which is defined based on combination of two types of impact: one is the impact of a macroblock on the quality of next frame and the other is the impact on the quality of current frame. The first impact is evaluated by counting the total of referred times of all pixel within the macroblock and the second one is evaluated by calculating difference between original and predicted macroblock. Then, combined with header information, all macroblocks of a coded frame is partitioned into three types of partitions. Finally, different rates of FEC channel code are applied to different partitions such that the important data can be protected strongly. While improving the quality of video like other data partitioning methods, an advantage of the proposed method is to balance the quality of the current and the next frame. Compared with other data partitioning methods such as in H.263++ and H.264, the proposed method can efficiently limit error and prevent error propagation. Experimental results show the proposed method has obtained stable quality for entire video sequence and achieve better PSNR under the two-state Markov channel model.

In this paper the flexible macroblock ordering (FMO or arbitrary slice ordering) feature of the H.264/AVC codec has been used to improve the robustness of the transmitted video through unreliable channels. FMO improves the decoded video quality remarkably in the presence of an appropriate error concealment algorithm. There are a number of mapping structures to distribute the macroblocks in individual slice groups. In this paper, we propose a new mapping scheme that provides an opportunity for transmission layers to protect the macroblocks that are more important. This will introduce a new unequal error protection (UEP) scheme based on FMO. Our simulation results show that this UEP scheme is efficient and successful, especially at high bit error rates.

This paper studies the rate-distortion performance of symmetric scalar quantizers to extend previous work published by the first author. We first provide a theoretical analysis of dead-zone plus uniform threshold quantization (DZ+UTQ) for nearly-uniform-reconstruction quantization (NURQ). The quantization performance is particularly investigated for Generalized Gaussian (such as Laplacian and Gaussian) sources using the squared-error distortion measure. According to the analysis, we note that the rate-distortion optimized quantizer is DZ+UTQ with NURQ for Laplacian sources, and is very similar to a DZ+UTQ for a uniform reconstruction quantizer (URQ). We further provide theoretical analysis of rate-distortion constrained DZ+UTQ with NURQ and URQ to cover other Generalized Gaussian sources and give the rate-distortion performance comparison theoretically between NURQ and URQ for DZ+UTQ. We conclude that a URQ is a near-optimal reconstruction rule for many sources, and that a DZ+UTQ classification rule is an effective classifier for it. URQ can be considered as a sub-optimal case of NURQ; the advantage of URQ being its simpler reconstruction rule. Based on the theoretical findings, a new DZ+UTQ quantization rounding technique for URQ is developed and integrated into recent H.264/AVC reference software to improve its encoding performance. Up to 1.0 dB performance improvement is observed, particularly in the very high bit rate range.

Rate control is a central part of standard video codecs. Many rate control schemes including some for JVT video coding (MPEG-4 AVC /H.264) employ a quadratic formulation of the rate-distortion (R-D) function. The rate control for JVT video coding is rather complex. In this paper, we analyse the problems existing in the quadratic R-D model and in the recent rate control scheme for JVT video coding. According to the analysis and experimental results, we present an improved quadratic R-D model and a frame-layer rate control scheme for JVT video coding based on the model. This quadratic R-D model is utilized for QP determination in rate control scheme, where the model parameters are estimated using statistical linear regression analysis. The model is also used for computing the starting quantization step (QP) for the first frame in the sequence. The Experimental results show that due to the accuracy of the improved quadratic R-D model, the rate control is quite effective, where the generated bit rates are very close to the target bit rates while achieving good R-D performance. And the starting QP is very close to the average QP, by which the performance is also improved.

Rate control for video encoders can be partitioned into procedures of rate allocation and bit achievement at several levels. For non-real-time applications, we propose to adopt a two-pass video encoding mechanism, where the framelevel rate allocation in the second pass is characterized of using content-aware models constructed by using information collected in the first pass. In our proposed scheme, a video sequence is divided into units of non-overlapping window, where a two-pass procedure is sequentially applied. The goal of the first pass is to find the encoding modes and motion vectors (MVs) for each MB and generate the proposed R-λ (rate vs. Lagrangian multiplier) and D-λ (distortion vs. Lagrangian multiplier) models to represent content characteristics of each frame. In the second pass, the encoding modes and MVs for MBs found in the first pass are retained, but the QPs are re-assigned MB-by-MB according to bit-rate and distortion predictions based on the above-constructed models. The performance of the proposed two-pass video encoder was compared with the H.264 reference model JM8.0. From the results, the proposed method outperforms JM 8.0 in PSNR by up to 1.38 dB for the test images used. Besides, the proposed algorithm is capable of keeping a controlled bit rate of much more precision (< 0.017% of error, two orders better than JM8.0), as well as a slightly smoother video quality, than that obtained by JM 8.0.

The H.264/AVC standard improves coding efficiency significantly compared with existing standards. This is at the expense of great computational complexity at the encoder. H.264/AVC adopts rate-distortion optimization (RDO) technique to obtain the best result while maximizing visual quality and minimizing bitrates. To carry out RDO, the encoder undergoes all possible combinations of coding modes exhaustively. H.264/AVC exploits directional spatial correlation with adjacent blocks for intra prediction. There are 9 directional modes for 4x4 luma blocks, 4 directional modes for 16x16 macroblocks and 8x8 chroma blocks, respectively. To determine the appropriate direction, based on the RDO computation, the number of mode combinations in each MB is 592. For a fast intra mode decision, we propose an algorithm to reduce the number of RDO computations using simple directional masks and adjacent mode information. The proposed method carries out 4 RDO calculations at the most for 4x4 luma blocks, and 2 RDO computations for 16x16 luma blocks and 8x8 chroma blocks, respectively. We verified the proposed algorithm by implementing on JM reference software and comparing with the case of exhaustive search. Simulation results show the proposed method reduces the ending time up to 70% with negligible loss of PSNR and bitrate increase.

To get high compress efficiency, the latest H.264/AVC international video coding standard introduced many more advanced tools than previously used standards, such as using sophisticated prediction and rate-distortion (RD) mode selection. Although these new coding tools can significantly improve video coding efficiency, the improvement in performance comes at substantially higher computational complexity as a result of more complicated mode decision. To reduce the complexity of H.264 encoding, various fast mode decision algorithms have been proposed. Many of these algorithms take advantage of motion and video content classification so as to reduce the number of modes tried during mode decision. When used for video sequences with high motion or detailed visual information, video coding e±ciency can be significantly compromised as a result of imperfect classification. In this paper, an efficient Probability Based Macroblock Mode Selection (PBMMS) algorithm is proposed. Each MB is predicted to different probability MB based on spatial and special correlation, which then leads to 50-60% computation savings and some compression performance improvements over existing H.264 fast mode decision algorithms.

The outstanding coding performance of H.264 comes with the cost of significantly higher complexity, making it too complex to be applied widely. This work aims at accelerating the H.264 encoder using joint algorithm/code-level optimization techniques so as to make it feasible to perform real-time encoding on a commercial personal computer. We propose a fast inter-mode decision scheme based on spatio-temporal information of neighboring macroblocks for the algorithm-level optimization. We use a commercial profiling tool to identify most time consuming modules and then apply several code-level optimization techniques, including frame-memory rearrangement, single-instruction-multipledata (SIMD) implementations based on the Intel MMX/SSE2 instruction sets. Search mode reordering and early termination for variable block-size motion estimation, are then applied to speed up these time-critical modules. The simulation results show that our proposed joint optimization H.264 encoder achieves a speed-up factor of up to 18 compared to the reference encoder without introducing serious quality degradation.

We propose an architecture for digital rights management based on a renewable, random key pre-distribution (KPD) scheme, HARPS (hashed random preloaded subsets). The proposed architecture caters for broadcast encryption by a trusted authority (TA) and by "parent" devices (devices used by vendors who manufacture compliant devices) for periodic revocation of devices. The KPD also facilitates broadcast encryption by peer devices, which permits peers to distribute content, and efficiently control access to the content encryption secret using subscription secrets. The underlying KPD also caters for broadcast authentication and mutual authentication of any two devices, irrespective of the vendors manufacturing the device, and thus provides a comprehensive solution for securing interactions between devices taking part in a DRM system.

This paper presents a robust watermarking scheme based on multi-band wavelet and principle component analysis (PCA) technique. Incorporating the PCA technique, the developed blind watermarking in multi-band wavelet domain can successfully resist common signal processing such as JPEG compression with quality factor as low as 15, and geometric distortion such as cropping (cropped by 50%). Different from many other watermarking schemes, in which the watermark detection threshold is chosen empirically, the false positive of the proposed watermarking scheme could be calculated, so watermark detection threshold could be chosen based only on the target false positive. Comparing with similar watermarks in conventional two-band wavelet domain, greater perceptual transparency and more robustness could be achieved for the proposed watermarking scheme. The parameterized multi-band wavelet also leads to a more secure embedding domain, which makes attacks more difficult.

To ensure secure digital content delivery, MPEG has dedicated significant effort to DRM (Digital Rights Management) issues. The IPMP (Intellectual Property Management and Protection) components, the fourth part of the new MPEG standard project-MPEG-21 (Multimedia Framework) is one of the most important parts of this standardization activity. It defines an interoperable framework for IPMP. Fairly soon after MPEG-4, with its IPMP hooks became an International Standard, MPEG decided to start a new project named "MPEG-4 IPMP Extensions" on more interoperable IPMP systems and tools. The project includes standardized ways of retrieving IPMP tools from remote locations, standardized messages exchanged among IPMP tools and the terminal. It also addresses authentication of IPMP tools, and has provisions for integrating Rights Expressions according to the Rights Data Dictionary and the Rights Expression Language. After MPEG-2/4 IPMP standardization work is finished, efforts are currently ongoing to define the similar mechanism for the management and protection of intellectual property in the various parts of the MPEG-21 standard currently under development. The present paper describes the technologies that are now included in MPEG-21 IPMP Components Committee Draft to effectively manage and protect the digital content with the flexible IPMP scheme, which specifies how protection is applied to Digital Items and facilitates the exchange of governed content declared by a standard constructor MPEG-21 DID (Digital Item Declaration) between MPEG-21 Peers.

Digital rights management (DRM) provides a unified approach to specifying, interpreting, enforcing and managing digital rights throughout the entire life cycle of digital assets. Using a declarative rights expression language (REL) for specifying rights and conditions in the form of licenses, as opposite to some other approaches (such as data structures and imperative languages), has been considered and adopted as a superior technology for implementing effective, interoperable and scalable DRM systems. This paper discusses some principles and issues for designing RELs, based on the experiences of developing a family of REL's (DPRL, XrML 1.x, XrML 2.0 and MPEG REL). It starts with an overview of a family tree of the past and current REL's, and their development history, followed by an analysis of their data models and a comparison with access-control oriented models. It then presents a number of primary design principles such as syntactic and semantic un-ambiguity, system interoperability, expressiveness in supporting business models and future extensibility, and discusses a number of key design issues such as maintaining stateful information, multi-tier issuance of rights, meta rights, identification of individual and aggregate objects, late-binding of to-be-identified entities, as well as some advanced ones on revocation and delegation of rights. The paper concludes with some remarks on REL profiling and extension for specific application domains.

In this paper, we propose a semi-blind video watermarking scheme, where we use pseudo-random robust semi-global features of video in the spatially wavelet transform domain. We design the watermark sequence via solving an optimization problem, such that the features of the mark-embedded video are the quantized versions of the features of the original video. We use a novel approach in generating the watermark such that the temporal correlation within the video frames is reflected on the generated watermark. This is achieved by i) using a special weight distribution along the time axis ii) modifying the regularization step to reflect the temporal characteristic of the host. We experimentally show the robustness of our algorithm against temporal filtering attacks and at the same time show a common weakness in 3D transform based video mark embedding methods.

Most existing license management schemes for DRM systems do not support the protection of user privacy; moreover, some other schemes such as PrecePt can only bind the license to a specified device though privacy protection is concerned. In this paper, we propose a flexible license management protocol named FLMP (Flexible License Management Protocol) which provides a more powerful and flexible license management scheme to satisfy the various requirements of users for privacy protection and use convenience.

In this paper we present a new framework, based on subdivision surface fitting, for high rate compression and coding of 3D models. Our algorithm fits the input 3D model, represented by a polygonal mesh, with a piecewise smooth subdivision surface represented by a coarse control polyhedron. Our fitting scheme, particularly suited for meshes issued from mechanical or CAD parts, aims at getting close to the optimality in terms of control points number, while remaining independent of the connectivity of the input mesh. The found subdivision control polyhedron is much more compact than the original mesh and visually represents the same shape after several subdivision steps, without artifacts or cracks, like traditional lossy compression schemes. This control polyhedron is then encoded specifically to give the final compressed stream. Experiments conducted on several 3D models have proven the coherency and the efficiency of our framework, compared with existing compression methods.

A novel model based and waveform based hybrid coding system aimed at very low bit rate coding of face images in video sequences is presented. The well known Principal Component Analysis (PCA) concept which is widely used for face recognition systems is adopted for model based coding, with modifications to cope with video compression. Model failures and unknown objects are handled by waveform based coding employing conventional prediction/transform block-based coding scheme. The two coding modes are combined under rate-distortion framework, where Lagrangian cost function is used to determine the most efficient prediction for each block. Simulations show that the system can achieve high compression ratios while maintaining the robustness and generality, which indicates its potential use for videophone application.

Recently, in mobile environments, a variety of character images have been provided to mobile phones and PDAs. Most of character images are an indexed image with 256 index values, requiring a lossless compression due to the property of the indexed data. Encoded image data is delivered to user clients from a server of content providers. Mobile devices are usually equipped with low processing power compared with desktop devices so that the simplicity of decoding process as well as the satisfactory compression ratio of encoding process is required. Especially, the complexity of a decoder needs to be reduced for the fast view of images on the LCD monitor. Many lossless compression schemes have been proposed and adopted in practical fields. We propose an efficient compression scheme satisfying these requirements. The main difference distinguishing the conventional approaches and our proposed method is that ours is mainly interested in character image sequence, where each sequence is composed of twelve images. In experiments performed on many test sequences, we show that our proposed method produces a compression ratio of 3.32:1.

An efficient way to transmit multi-view images is to send the texture image together with a corresponding depth image. The depth image specifies the distance between each pixel and the camera. With this information, arbitrary views can be generated at the decoder. In this paper, we propose a new algorithm for the coding of depth images that provides an efficient representation of smooth regions as well as geometric features such as object contours. Our algorithm uses a segmentation procedure based on a quadtree decomposition and models the depth image content with piecewise linear functions. We achieved a bit-rate as low as 0.33 bit/pixel, without any entropy coding. The attractivity of the coding algorithm is that, by exploiting specific properties of depth images, no degradations are shown along discontinuities, which is important for perceived depth.

The paper presents a simple and effective non-iterative method to reduce ringing artifacts of a restored image, which focuses on the artifacts both near the boundaries and close to the edges of the image. We introduce the power spectrum theory into wavelet domain and suppress the ringing artifacts in both the coarse and the fine scales separately. In low frequency subbands, we analyze the power spectrum of low frequency coefficients and calculate the peak regions that cause the ringing artifacts. After determining the peaks, we suppress them by a Center Weighted Median (CWM) filter from outside to inside layer-by-layer. In high frequency subbands, we use an adaptive filter to filter out residual ringing and preserve sharp edges and details. Experimental results demonstrate that our proposed post-filtering algorithm achieves better subjective and objective results compared with other methods. Also, it has very low complexity and high restoration speed.

At low bit rates (0.2bpp or below), images coded by JPEG suffer severe visual distortions. These distortions include blurred edges due to truncation of high frequency coefficients and blocking artifacts due to the block-based nature of JPEG. In this paper, we propose techniques to reduce these two artifacts so as to obtain an image of better visual quality as well as higher PSNR. For blocking artifacts, we adjust the low frequency DCT coefficients by minimizing abrupt changes in the image under certain constraint. The abrupt changes are measured using the mean square values of the difference between adjacent pixels. For blurred edges, we first detect them using the Canny edge detector and then represent them using an edge model with their parameters estimated. The distortion around these edges can then be removed by adjusting the parameters of the edge model under the quantization constraint. The method requires about four seconds for a 512x512 image using a high-end PC. Experimental results show that the proposed approach can improve the visual quality as well as the fidelity of JPEG coded images.

A pre/post-filtering framework for DCT image coding has been proposed recently. Together withWiener filtering technique, the new framework has been investigated for error resilient image coding. In this paper, after a review of existing methods, we analyze the limitation associated with one of the current design criteria, the reconstruction gain, and propose an alternative that offers better control of the error distribution, thereby improving the visual quality of the reconstructed image in the presence of transmission error. The performance of the proposed scheme is demonstrated through various multiple description coding (MDC) simulations.

A new low-cost video frame-rate up conversion (FRC) technique that exploits compressed-domain information is proposed in this paper. Recent worldwide growth of digital video applications such as home AV network or mobile multimedia have been demanding higher video compression ratio in order to meet system requirements related to limited bandwidth or storage capacity. Enhancement of temporal resolution using FRC at decoder side is attractive in a sense that the R-D performance can be maintained even at low bitrate condition by dropping frames during encoding and recovering them at decoder side by FRC. The proposed FRC method aims to provide accurate estimates of block-based motion vectors for recovered frames based on compressed-domain information. The goal of our study is to realize a decoder design with robust FRC over a variety of implementation platforms from mobile to consumer set-top.

This paper presents a perceptual post filtering coder for digital colour images in YCrCb colour space. The approach builds on our earlier perceptual coder (PC) and exploits intra-band and inter-orientation masking properties of Human Visual System (HVS) to identify, estimate and recover the amount of perceived visual information loss due to compression. The proposed technique applies to our earlier perceptual coder (PC) which retains most of the embedded Block Coding with Optimized Truncation (EBCOT) features and is bit-stream compliant to the JPEG2000 standard. We use PC coder to compress images with some information loss and hence loss of quality. The images are then reconstructed from the compressed bit-stream with our proposed post filtering coding technique that attempts to recover the perceived loss of visual information with a HVS model. The simulation results have shown that our proposed perceptual post filtering coder achieves comparable or superior visual performance over that of our PC, and that of JPEG2000 verification model 8.0 coder with both MSE and visual masking.

In this paper, we present a virtual viewpoint video synthesis algorithm to satisfy the following three aims: low computing consuming; real time interpolation and acceptable video quality. In contrast with previous technologies, this method obtain incompletely 3D structure using neighbor video sources instead of getting total 3D information with all video sources, so that the computation is reduced greatly. So we demonstrate our interactive multi-view video synthesis algorithm in a personal computer. Furthermore, adopting the method of choosing feature points to build the correspondence between the frames captured by neighbor cameras, we need not require camera calibration. Finally, our method can be used when the angle between neighbor cameras is 25-30 degrees that it is much larger than common computer vision experiments. In this way, our method can be applied into many applications such as sports live, video conference, etc.

In this paper, a simplified implementation of the Concentric Mosaics image-based rendering technique is proposed. The greatest difficulty for an ordinary user in obtaining the pre-captured images for use in the Concentric Mosaics technique is the precise control of the rotation of a long beam. In the proposed Simplified Concentric Mosaics technique, the camera positions where the pre-captured images will be taken are not precisely controlled, but are estimated from the pre-captured images. We use a stereo technique for estimation of camera positions in this special scenario instead of using the traditional camera pose estimation methods of computer vision. The estimation errors have been analyzed and a closed-loop constraint is used to achieve better rotation angle estimation. In addition, a ratio fitting technique is proposed to select good matching features in the special tri-view matching detection scenario, which subsequently improves rotation angle estimation. Another contribution of the paper is a pre-processing step to eliminate or reduce the possible vertical offsets and other distortions in the pre-captured images, which are caused by any possible motions of the camera that deviate from the ideal one. In a column-based view synthesis technique like the proposed method and the conventional Concentric Mosaics technique, these vertical offsets and distortions in the pre-captured images will lower the quality of the synthesized images. Thus our pre-processing can be applied on both the proposed method and the ordinary Concentric Mosaics technique. The pre-captured image data structures of both Concentric Mosaics and the proposed method have been illustrated and the comparison has been made. The proposed technique has a similar data structure and thus a similar rendering algorithm as the conventional Concentric Mosaics technique. As a result, it meets our objective that an ordinary user can obtain the Concentric Mosaics type image data and plug it into a common Concentric Mosaics rendering framework. Simulation results show that the proposed method can achieve good rendering results.

Image-based rendering (IBR) is an emerging technology for photo-realistic rendering of scenes from a collection of densely sampled images and videos. Recently, an object-based approach for rendering and the compression of a class of dynamic image-based representations called plenoptic videos was proposed. The plenoptic video is a simplified dynamic light field, which is obtained by capturing videos at regularly locations along a series of line segments. In the object-based approach, objects at large depth differences are segmented into layers for rendering and compression. The rendering quality in large environment can be significantly improved, as demonstrated by the pop-up lightfields. In addition, by coding the plenoptic video at the object level, desirable functionalities such as scalability of contents, error resilience, and interactivity with individual IBR objects, can be achieved. An important step in the object-based approach is to segment the objects in the video streams into layers or image-based objects, which is largely done by semi-automatic technique. To reduce the segmentation time for segmenting plenoptic videos, efficient tracking techniques are highly desirable. This paper proposes a new automatic object tracking method based on the level-set method. Our method, which utilizes both local and global features of the image sequences instead of global features exploited in previous approach, can achieve better tracking results for objects, especially with non-uniform energy distribution. Due to possible segmentation errors around object boundaries, natural matting with Bayesian approach is also incorporated into our system. Using the alpha map and texture so estimated, it is very convenient to composite the image-based objects onto the background of the original or other plenoptic videos. Furthermore, a MPEG-4 like object-based algorithm is developed for compressing the plenoptic videos, which consist of the alpha maps, depth maps and textures of the segmented image-based objects from different video plenoptic streams. Experimental results show that satisfactory renderings can be obtained by the proposed approaches.

Ray-Space representation has superiority in rendering arbitrary viewpoint images of complicated scene in real-time. Ray-Space interpolation is one of the key techniques to make Ray-Space based Free Viewpoint Television (FTV) feasible. This paper presents a directionality based interpolation method for Ray-Space based FTV system, in which characteristic pixels are first extracted from sparse Ray-Space slice, and their directionalities are determined by block matching, while directionalities of other pixels to be interpolated are obtained by interpolating with the directionalities of these characteristic pixels. Experimental results show that the proposed method improves visual quality as well as PSNRs of rendered intermediate viewpoint image greatly.

This paper proposes a new object-based compression system for a class of dynamic image-based representations called plenoptic videos (PVs). PVs are simplified dynamic light fields, where the videos are taken at regularly spaced locations along line segments instead of a 2-D plane. The proposed system employs an object-based approach, where objects at different depth values are segmented to improve the rendering quality as in the pop-up light fields. Furthermore, by coding the plenoptic video at the object level, desirable functionalities such as scalability of contents, error resilience, and interactivity with individual IBR objects can be achieved. Besides supporting the coding of the texture and binary shape maps for IBR objects with arbitrary shapes, the proposed system also supports the coding of gray-scale alpha maps as well as geometry information in the form of depth maps to respectively facilitate the matting and rendering of the IBR objects. To improve the coding performance, the proposed compression system exploits both the temporal redundancy and spatial redundancy among the video object streams in the PV by employing disparity-compensated prediction or spatial prediction in its texture, shape and depth coding processes. To demonstrate the principle and effectiveness of the proposed system, a multiple video camera system was built and experimental results show that considerable improvements in coding performance are obtained for both synthetic scene and real scene, while supporting the stated object-based functionalities.

Quadrature filters are a well known means for local spectral analysis of images and to extract relevant structure. Recently, there has been the discovering of an isotropic quadrature filter for images that does not need steering with respect to orientation and provides the user with geometric (local orientation) and structural (local phase) information. Here, we present a further extension of this, an isotropic quadrature filter for 3D data. With only four convolutions we can calculate the local orientation, energy and the phase of locally intrinsically 1D structuresy in 3D image data. There is a wide variety of useful applications for a filter of this kind. In this paper, we will restrict ourselves to the application of phase-based motion or flow field estimation.

Video sequences captured by handheld digital camera need to be stabilized to eliminate the tiresome effects caused by camera's undesirable shake or jiggle. The key issue of video stabilization is to estimate the global motion parameters between two successive frames. In this paper, a novel circular block matching algorithm is proposed to estimate the global motion parameters. This algorithm can deal with not only translational motion but even large rotational motion. For an appointed circular block in current frame, a four-dimensional rotation invariant feature vector is firstly extracted from it and used to judge if it is an effective block. Then the rotation invariant features based circular block matching process is performed to find the best matching blocks in reference frame for those effective blocks. With the matching results of any two effective blocks, a two-dimensional motion model is constructed to produce one group of frame motion parameters. A statistical method is proposed to calculate the estimated global motion parameters with all groups of global motion parameters. Finally, using the estimated motion parameters as the initial values, an iteration algorithm is introduced to obtain the refined global motion parameters. The experimental results show that the proposed algorithm is excellent in stabilizing frames with even burst global translational and rotational motions.

As opposed to the global shutter, which starts and stops the light integration of each pixel at the same time by incorporating a sample-and-hold switch with analog storage in each pixel, the electronic rolling shutter found in most low-end CMOS image sensors today collects the image data row by row, analogous to an open slit that scans over the image sequentially. Each row integrates the light when the slit passes over it. Therefore, the scanlines of the image are not exposed at the same time. This sensor architecture creates an objectionable geometric distortion, known as the rolling shutter effect, for moving objects. In this paper, we address this problem by using digital image processing techniques. A mathematical model of the rolling shutter is developed. The relative image motion between the moving objects and the camera is determined by block-based motion estimation. A Bezier curve fitting is applied to smooth the resulting motion data , which are then used for the alignment of scanlines. The basic ideas behind the algorithm presented here can be generalized to deal with other complicated cases.

The pel-recursive computation of 2-D optical flow has been extensively studied in computer vision to estimate motion from image sequences, but it still raises a wealth of issues, such as the treatment of outliers, motion discontinuities and occlusion. It relies on spatio-temporal brightness variations due to motion. Our proposed adaptive regularized approach deals with these issues within a common framework. It relies on the use of a data-driven technique called Mixed Norm (MN) to estimate the best motion vector for a given pixel. In our model, various types of noise can be handled, representing different sources of error. The motion vector estimation takes into consideration local image properties and it results from the minimization of a mixed norm functional with a regularization parameter depending on the kurtosis. This parameter determines the relative importance of the fourth norm and makes the functional convex. The main advantage of the developed procedure is that no knowledge of the noise distribution is necessary. Experiments indicate that this approach provides robust estimates of the optical flow.

Global-motion estimators are an important part of current video-coding systems like MPEG-4, content analysis and description systems like MPEG-7, and many video-object segmentation algorithms. Feature-based motion estimators use the motion vectors obtained for a set of selected points to calculate the parameters of the globalmotion model. This involves the detection of feature points, the computation of correspondences between two sets of features, and the motion parameter estimation. In this paper, we will present a feature-based global-motion estimation system and discuss each of its parts in detail. The idea is to provide an overview of a general purpose feature-based motion estimator and to point out the important design aspects. We evaluate the performance of di erent feature detection algorithms, propose an efficient feature-correspondence algorithm, and we compare the di erence between a non-linear parameter estimation and a linear approximation. Finally, the RANSAC based robust parameter estimation is examined, we show why it does not reach its theoretical performance, and propose a modification to increase its accuracy. Our global-motion estimator has an average accuracy of ≈0.15 pixels with real-time execution.

Recently established international video coding standard H.264/AVC and the upcoming standard on scalable video coding (SVC) bring part of the solution to high compression ratio requirement and heterogeneity requirement. However, these algorithms have unbearable complexities for real-time encoding. Therefore, there is an important challenge to reduce encoding complexity, preferably in a scalable manner. Motion estimation and motion compensation techniques provide significant coding gain but are the most time-intensive parts in an encoder system. They present tremendous research challenges to design a flexible, rate-distortion optimized, yet computationally efficient encoder, especially for various applications. In this paper, we present a scalable motion estimation framework for complexitydistortion consideration. We propose a new progressive initial search (PIS) method to generate an accurate initial search point, followed by a fast search method, which can greatly benefit from the tighter bounds of the PIS. Such approach offers not only significant speedup but also an optimal distortion performance for a given complexity constrain. We analyze the relationship between computational complexity and distortion (C-D) through probabilistic distance measure extending from the complexity and distortion theory. A configurable complexity quantization parameter (Q) is introduced. Simulation results demonstrate that the proposed scalable complexity-distortion framework enables video encoder to conveniently adjust its complexity while providing best possible services.

This paper presents a method of mapping visual data into haptic cues for the blind to support them in navigation. Employing auditory cues alone to support blind navigation may help some extent but too much information to the ear may lead to confusion so important information such as dangers from the obstacles, stairs, potholes, sharp edges etc given through haptic signals not only reduce the over load on the auditory channel but also improves the speed of navigation by enabling quicker reaction. The proposed system for representing visual information intuitively and easily using haptic signals may assist the visually impaired in their navigation. This technique translates the images captured from digital cameras, extracts the required information and converts it into tactile signals. The use of the haptic channel is suggested in order to prevent overload of the auditory system and to use the special qualities of the haptic sense to convey important messages of danger and hazards. This paper presents a tactile display design and discusses the application framework for using these displays for tactile information design experimentation.