The encoding of vector quantization (VQ) needs expensive computation for searching the closet codeword to the input vectors. In order to reduce computation burden, Pan et al. have developed an efficient full-search-equivalent algorithm by using the characteristics of the sums and variances of a vector and its two fixed subvectors recently. However, some computational redundancies still exist in it. In this paper, we introduce a technique to efficiently partition a vector into two dynamic subvectors according to the patterns inside the block (or vector) and then further improve the computation load given by Pan's method. Experimental results show that the proposed method is superior to other algorithms in terms of processing time and the number of distortion calculations. Compared to Pan's algorithm, our algorithm can further reduce the processing time and the number of distortion calculations for various codebook sizes by 14.7% to 41.3% and 14.9% to 44.6%, respectively.

The chrominance subsampling has been widely used for the image and video compression. However, this can cause color distortion and image quality degradation. We developed a new sampling method in which the residual image after intra or inter prediction is subsampled instead of the original image subsampling. Since the residual image contains fewer information than the original image does generally, the information loss is reduced when the residual image is downsampled. Also the upsampling procedure is much less sensitive to the filtering methods selected as demonstrated in the experimental results. So we can apply a simple filtering to reduce the complexity. The experimental results show the effectiveness of the proposed sampling method. This sampling method can be further improved by applying adaptively according to the statistics of each block, and can be applied to the RGB image directly without color conversion.

Variable block size motion compensation has been adopted by the emerging video coding standard H.264. It can represent the motion characteristic in a macroblock more accurately, and therefore reduce the prediction error to achieve high compression gains. On the other hand, it causes high computational complexity in motion estimation at the encoder. The motion estimation exhaustively performed over all modes to find the best mode for inter coding is slow and computationally involved. In order to reduce the complexity, we proposed a fast inter mode decision algorithm for H.264 encoder. The proposed method efficiently determines a suitable block mode according to the motion field distribution and correlation within a macroblock. The experimental results show that the proposed method reduces a considerable amount of complexity at encoder, while the R-D performance of the proposed method is comparable to that of full mode search.

This paper proposes a new motion compensation method using "sliced blocks" in DCT-based hybrid video coding. In H.264 | MPEG-4 Advance Video Coding, a brand-new international video coding standard, motion compensation can be performed by splitting macroblocks into multiple square or rectangular regions. In the proposed method, on the other hand, macroblocks or sub-macroblocks are divided into two regions (sliced blocks) by an arbitrary line segment. The result is that the shapes of the segmented regions are not limited to squares or rectangles, allowing the shapes of the segmented regions to better match the boundaries between moving objects. Thus, the proposed method can improve the performance of the motion compensation. In addition, adaptive prediction of the shape according to the region shape of the surrounding macroblocks can reduce overheads to describe shape information in the bitstream. The proposed method also has the advantage that conventional coding techniques such as mode decision using rate-distortion optimization can be utilized, since coding processes such as frequency transform and quantization are performed on a macroblock basis, similar to the conventional coding methods. The proposed method is implemented in an H.264-based P-picture codec and an improvement in bit rate of 5% is confirmed in comparison with H.264.

Adaptive Block-size Transforms (ABT) has been widely used in image/video coding, since it exploits the maximum feasible signal length for transform coding. However, if the transforms in an ABT coding system are Integer Cosine Transforms (ICT), not only separate transform units but also different scaling matrices are required, which consume a vast amount of resources in practical implementations. In this paper, a new approach to compatible ABT is presented, by which 8x8, 8x4, 4x8 and 4x4 transforms can be processed in one transform unit. Furthermore, with Pre-scaled Integer Transform (PIT), the compatibility of scaling matrices especially for 8x4 and 4x8 ICT can be achieved and a single scaling matrix is required. Simulation results and analysis reveal that this approach greatly saves hardware resources and makes the implementation of ABT much easier without loss of performance.

We propose a fast motion search method for H.264/AVC with dual-path block size decision. H.264/AVC employs variable block sizes for motion compensation to reduce coding bits of inter-frame prediction error, which requires considerable amount of computation time when motion estimation is performed for every block size. Our algorithm contains two strategies to reduce computation time for motion search; block size mode reduction and search range reduction. According to these strategies, our algorithm consists of two stages. At the first stage, RDCost-based block size mode reduction is conducted. Rate-distortion function (RDCost) for skip mode is calculated at first, which determines the smallest block size for motion search. The second stage is a fast variable block size motion estimation which contains two paths, 16x16-first and 8x8-first. The 16x16-first path is invoked when the minimum block size determined at the first stage is larger than 8x8. In the 8x8-first path, search range for blocks larger than 8x8 is reduced according to distance between motion vectors for 8x8 blocks. From our experiment using JM 8.5, it is confirmed that our algorithm can reduce about 89.3% of computation time as compared to JM, with only negligible PSNR degradation.

Providing quality of service (QoS) guarantees in packet networks gives rise to several challenging issues. In general, multi-constrained path selection with or without optimization, is a NP-complete problem that cannot be exactly solved in polynomial time. Hence, accurate constraints-based routing algorithms with a fast running time are scarce, perhaps even non-existent. The expected impact of such a constrained-based routing algorithm has resulted in the proposal of numerous heuristics and a few exact QoS algorithms. This paper aims to give a thorough, concise and fair evaluation of the most important multiple constraint-based QoS Multicast routing algorithms known today, and it provides a descriptive overview and simulation results of these multi-constrained routing algorithms.

As the Internet does not widely support Internet protocol multicast, and content distribution networks are costly, application overlay has emerged as an alternative for deploying large scale streaming systems. Loss recovery in such architecture is a great challenge because of the correlation of packet losses results from the relaying nature, the variation of accumulated loss rates, and the dynamics of overlay structure, which is inevitable. Since each overlay node is capable of buffering a certain length of media data it has received, and there exists a temporal dependency between the buffers of nodes along a transmitting path, it is highly desirable to make full use of this buffering capability to carry out loss recovery. To this purpose, a retransmission-based approach is proposed in this paper. First, packet losses at a node are classified into two categories according to whether they can be repaired from the immediate upstream node, then, by making upstream nodes propagate loss information downstream, the proposed scheme enables each node efficiently recognize the nature of the loss it detects, and accordingly determine appropriate repair source, thus suppresses unnecessary retransmission requests. The proposed scheme is supported by theoretical analysis of the temporal dependency between overlay nodes, and its performance is verified by experimental results.

This paper investigates the transmission of H.264 /AVC video in the 3GPP Multimedia Broadcast/Multicast Streaming service (MBMS). Application-layer forward error correction (FEC) codes are used to combat transmission errors in the radio access network. In this FEC protection scheme, the media RTP stream is organized into source blocks spanning many RTP packets, over which FEC repair packets are generated. This paper proposes a novel method for unequal error protection that is applicable in MBMS. The method reduces the expected tune-in delay when a new user joins into a broadcast. It is based on four steps. First, temporally scalable H.264 /AVC streams are coded including reference and non-reference pictures or sub-sequences. Second, the constituent pictures of a group of pictures (GOP) are grouped according to their temporal scalability layer. Third, the interleaved packetization mode of RFC3984 is used to transmit the groups in ascending order of relevance for decoding. As an example, the non-reference pictures of a GOP are sent earlier than the reference pictures of the GOP. Fourth, each group is considered a source block for FEC coding and the strength of the FEC is selected according to its importance. Simulations show that the proposed method improves the quality of the received video stream and decreases the expected tune-in delay.

This paper presents an H.263+-based overlay video multicast system over the Internet. The proposed overlay video multicast system consists of two parts: overlay multicast tree suitable for the video traffic delivery and H.263+ rate control adaptive to overlay multicast tree. Overlay multicast tree is constructed to minimize the average delay for video traffic delivery, and H.263+ rate control is implemented to enhance the human visual perceptual quality over the multicast tree. Finally, we show by experimental results that the proposed overlay multicast system can provide the effective overlay multicast service over the Internet.

With the intensifying academic interest in video streaming over peer-to-peer(P2P) network, more and more streaming protocols have been proposed to address different problems, such as QoS, load balancing, transmission reliability, bandwidth effciency, etc. However, to the best of our knowledge, an important component of any practical P2P streaming system, the streaming service discovery, which is to discover potential peers from which a newcomer could receive the requested stream, is rarely granted particular consideration. In this paper, we are inspired from a special data structure, namely Segment Tree(ST), and propose a protocol to address this problem. Furthermore, we fully decouple the control plane and data plane in video streaming, and hence provide more flexibilities in designing protocols on both of them.

In video-on-demand (VOD) system, efficient video delivery scheme can largely increase system service ability by reducing the resource requirement. Most existing schemes are not suited for the instance that the characteristic of the workload changing greatly with time. A workload adaptive scheme is proposed in the paper. It absorbs the outstanding thoughts from two efficient schemes: fast broadcasting and patching scheme. The scheme allocates adaptively the number of channels according to request rate to minimize the required bandwidth. We show how to seamlessly perform the transition of changing the number of channels while guaranteeing the clients currently viewing a video will not experience any disruption. The scheme can provide zero-delay video-on-demand (VOD) service. Simulation results show that the scheme adapts nicely to the changing request rate and improves the performance of VOD service significantly in terms of total server bandwidth requirement.

In the context of visual signal analysis for media adaptation, this paper presents a stochastic method for sementation of motion vector fields or as known as displacement fields in compressed videos. The addressed field sequence is analysed and partitioned to temporally associated, motion-semantic regions by three analysis steps. The global motion based reliability analysis assesses the reliability extent of each encoded displacement vector. Upon the measure array, an initial partition is approximated by the region displacement prediction. The subsequent relaxation procedure optimises the partition shape, coverage, and location using the stochastic motion coherency model and the second-order random field based contour smoothness. The visually convincing results are demonstrated from the standard sequences.

This paper aims to apply the method of silhouette matching based on moment invariants to infer the human motion parameters from video sequences of single monocular uncalibrated camera. Currently, there are two ways of tracking human motion: Marker and Markerless. While a hybrid framework is introduced in this paper to recover the input video contents. A standard 3D motion database is built up by marker technique in advance. Given a video sequences, human silhouettes are extracted as well as the viewpoint information of the camera which would be utilized to project the standard 3D motion database onto the 2D one. Therefore, the video recovery problem is formulated as a matching issue of finding the most similar body pose in standard 2D library with the one in video image. The framework is applied to the special trampoline sport where we can obtain the complicated human motion parameters in the single camera video sequences, and a lot of experiments are demonstrated that this approach is feasible in the field of monocular video-based 3D motion reconstruction.

This study is aimed at obtaining smoothed trajectory of feature points in dynamic image, where the feature points are extracted at each image frame with missing and false detection. The image scene contains independently moving multiple objects with occlusion and appearance. We develop a state space model using Finite Random Set (FRS) to cope with this situation since FRS is a set of random variables with the number of the variables also being random (integer) variable so it is suitable for representing the variable number of feature points caused by appearance/occlusion and missing/false detection. By estimating the state of the model using Sequential Monte Carlo (SMC) implementation of Probability Hypothesis Density (PHD) filter, we obtain smoothed trajectory of the feature points. PHD is 1st order moment of FRS, and it has a property that its integration yeilds the expected number of feature points in the integrated region. The SMC implementation gives approximated solution by weighted particles, where the number of particles varies depending on the number of feature points in the scene. Experiment of dynamic image demonstrates that proposed method successfully smoothed the trajectory of feature points responding to appearance and occlusion of objects without being affeted by missing and false detection.

Moving objects segmentation is a challenge in computer vision. This paper focuses on the segmentation of moving vehicles in dynamic scene. We analyses the psychology of human vision and present a framework for segmenting moving vehicles in the highway. The proposed framework consists of two parts. Firstly, we propose an adaptive background update method in which the background is updated according to the change of illumination conditions and thus can adapt to the change of illumination sensitively. Secondly, we construct a Gaussian motion model to segment moving vehicles, in which the motion vectors of the moving pixels are modeled as a Gaussian model and an on-line EM algorithm is used to update the model. The Gaussian distribution of the adaptive model is elevated to determine which moving vectors result from moving vehicles and which from other moving objects such as waving trees. Finally, the pixels with motion vector result from the moving vehicles are segmented. Experimental results of several typical scenes show that the proposed model can detect the moving vehicles correctly and is immune from influence of the moving objects caused by the waving trees and the vibration of camera.

Human motion analysis research, especially the human tracking part, remains a challenging task by far. The difficulties lie in several aspects: self-occlusion, high dimensionality of parameter space and the gap between high-level image understanding and low-level image features etc. In our work we use particle filter to track human movement from monocular video sequences with an articulated human body model. We fuse region, color and boundary information to build a robust measurement function. Among them, the boundary information represented by Fourier Descriptors (FD) sets up a new and effective connection between the estimated model parameters and the image likelihoods. Compared with the previously used boundary or contour cue, FD has many noticeable advantages. Moreover, we introduce an adaptive property into the particle filter for more robust state propagation and measurement updating. Our method is shown to work effectively in experiments.

Traditionally, video segmentation usually extracts object using low-level features such as color, texture, edge, motion, and optical flow. This paper originally proposes that the connectivity of object motion is an advanced feature of video moving object because it can reflect semantic meanings of object to some extent. And it can be fully represented on cumulated difference image which is the combination of a certain number of interframe difference images. Based on this principle, a novel system is designed to extract initial moving object automatically. The system includes 3 key innovations: 1) System is applied on cumulated difference image which can make object more prominent than background noise. Object extraction is based on the connectivity of object motion and it can guarantee the integrity of the extracted object while eliminate big background regions which cannot be removed by conventional change detection methods, for example, intense-noise regions and shadow regions that are not connected tightly to object. 2) Video sequence analysis is performed ahead of video segmentation. Proper object extraction methods are adopted according to the characteristics of background noise and object motion. 3) The adaptive threshold is automatically determined on cumulated difference image after acute noises is removed. The threshold determined here is more reasonable. And with it, most noise can be eliminated while small-motion regions of object are preserved. Results show that this system can extract object in different kinds of sequences automatically, promptly and properly. Thus, this system is very suitable for real time video applications.

In this paper we present an adaptive version of the update step in the lifting implementation of MCTF (Motion Compensated Temporal Filtering). We explicitly take into account the local image content in order to avoid ghosting artifacts in the updated low-pass frame, tuning the update weighting factor where such artifacts are more likely to be perceived. The proposed solution is integrated into a 2D+t (in-band) wavelet based video codec and improves the subjective quality of sequences reconstructed at reduced frame rate.

In this work, hierarchical motion compensated three-dimensional (3-D) filter banks for spatio-temporal multi-resoution analysis are presented as new tools for scalable video format control. For recent developments in scalable video coding, most of them are based on 3-D wavelet transform with motion compensation. To achieve the function of frame-rate and spatial resolution scalabilities, motion compensated temporal filtering (MCTF) through lifting wavelet transform currently attracts many researchers as an effective temporal decomposition tool. As previous works, we proposed single stage deinterlacer banks as novel 3-D filter banks. Unlike other filter banks, our proposed system is constructed in a way unique to multi-dimensional systems by using invertible deinterlacers, which we have proposed before. The single stage deinterlacer banks decompose a progressive video into two subband sequences of a half frame-rate in the progressive scanning manner. Even though the system handles interlaced videos as intermediate data, introducing a multistage decomposition is simply achieved because the output sequences have also the progressive format as like the input. Our proposed hierarchical deinterlacer banks are accompanied by 2-D discrete wavelet transforms (DWT) as spatial transforms. The multistage technique provides interlaced sequences as well as several reduced-rate progressive sequences while maintaining the reconstruction of the original full-resolution full-rate video sequence. We show that fine granular control of spatio-temporal resolution can be acchieved through the hierarchical process. Some experimental results show novel functions and the significance of the proposed filter banks.

This paper makes a comparative study on the various spatial scalable coding frameworks. The frameworks with multiple image-domain motion aligned temporal filtering at various spatial resolutions, named as multi-T+2D, are mainly investigated. First we investigate a multi-T+2D scheme based on redundant frame representation. The cross spatial layer redundancy and prediction methods are discussed. The redundancy brings significant performance loss for schemes providing wide range SNR spatial scalability. To remove the redundancy produced in the multi-resolution temporal filtering while retaining the advantage of spatial-domain motion compensation, a novel non-redundant multi-T+2D scheme is proposed. Performance comparison is given among the discussed frameworks and it shows that the proposed non-redundant multi-T+2D framework has a good performance for fully scalable video coding. We also verify that the redundant multi-T+2D framework with cross spatial layer reconstruction feedback is practical in providing narrow range SNR scalability for each spatial layer.

This paper presents a new Base Layer coding scheme which introduces the rate-distortion optimization mode decision into hierarchical-fashion B-picture coding with open-loop structure. It can not only achieve high coding performance for Base Layer bit-rate point and meanwhile, but the scheme also keep the strong correlation between Base Layer and Enhancement Layer implemented by 3D subband scalable video codec, because they share the similar lifting based Motion-Compensated Temporal Filtering (MCTF) decomposition structure. In addition, how to exploit the correlation between Base Layer and Enhancement Layer is also investigated. Experimental results show this method improves the coding performance and visual quality at both low and high bit-rate points.

As of its open-loop structure and good decorrelation capability, motion-compensated temporal filtering (MCTF) provides a robust basis for highly-efficient scalable video coding. Combining MCTF with spatial wavelet decomposition and embedded quantization results in a 3D wavelet video compression system, providing temporal, spatial, and SNR scalability. Recent results indicate that the overall coding performance of these systems can be maximized if temporal filtering is performed in spatial domain (t+2D approach). However, as compared to non-scalable video coding, the performance of t+2D systems may not be satisfactory if spatial scalability needs to be provided. One important reason for this fact is the problem of spatial scalability of motion information. In this paper we present a conceptually new approach for t+2D-based video compression with spatially scalable motion information. We call our approach overcomplete MCTF since multiple spatial-domain temporal filtering operations are needed to generate the lower spatial scales of the temporal subbands. Specifically, the encoder performs MCTF-based generation of reference sequences for the coarser spatial scales. We find that the newly generated reference sequences are of satisfactory quality. Compared to the conventional t+2D system, our approach allows for optimization of the reconstruction quality at lower spatial scales while having reduced impact on the reconstruction quality at high spatial scales/bitrates.

The discrete wavelet transform (DWT) has been widely used in scalable video coding for its advantages in multi-resolution analysis and subband decomposition. In this paper, a spatially scalable video coding system based on H.264 coding method and in-band overcomplete discrete wavelet transform (ODWT) technique is proposed, which integrates the good compression performance of H.264 in low frequency domain with the efficient motion estimation of in- band ODWT in wavelet domain. Intra prediction, coefficients scan manner and inter prediction are improved to overcome the inefficiency of H.264 coding in high frequency subbands caused by different pixels distribution properties. Through series of subband analysis and statistical data retrieval for the three high frequency decompositions, intra prediction directions are optimized and three subsets of prediction mode are presented for the three high subbands respectively. They save over 30% bits for intra mode with similar performance. Moreover, novel zigzag scan tables are proposed to improve the coding efficiency by utilizing the oriented frequency characteristics of each high band. To inter prediction, an adaptive motion estimation method is proposed in which the motion information of low band is adaptively and effectively utilized to achieve much more accurate motion vector and more efficient motion compensation in high bands coding. Experimental results show that, all of the proposed methods endue the spatially scalable video coding system with over 0.4 dB gain in PSNR and 10.4% in rate reduction.

For video retrieval, a video is partitioned into a group of shots, which are then represented by either key frames or video shot representations. An optimal representation of a shot should include all the information about the frames concerned. In this paper, we propose an efficient representation scheme for a shot, which considers both the spatial frequency contents and the temporal statistics of the frames for video retrieval. In our scheme, each frame in a video shot is transformed into the frequency domain using the discrete cosine transform (DCT), and a number of values at each frequency are selected based on their probability of occurrence. This representation scheme allows retrieval to be carried out hierarchically, i.e. from low-frequency to high-frequency components. Experimental results show that our proposed scheme outperforms the alpha-trimmed average histogram method in terms of retrieval accuracy.

We propose a fast coarse-to-fine video retrieval scheme using shot-level spatio-temporal statistics. The proposed scheme consists of a two-step coarse search and a fine search. At the coarse-search stage, the shot-level motion and color distributions are computed as the spatio-temporal features for shot matching. The first-pass coarse search uses the shot-level global statistics to cut down the size of the search space drastically. By adding an adjacent shot of the first query shot, the second-pass coarse-search introduces the "causality" relation between two consecutive shots to improve the search accuracy. As a result, the final fine-search step based on local color features of key-frames of the query shot is performed to further refine the search result. Experimental results show that the proposed methods can achieve good retrieval performance with a much reduced complexity compared to single-pass methods.

In this paper, we propose a novel semi-automatic annotation scheme for home videos based on active learning. It is well-known that there is a large gap between semantics and low-level features. To narrow down this gap, relevance feedback has been introduced in a number of literatures. Furthermore, to accelerate the convergence to the optimal result, several active learning schemes, in which the most informative samples are chosen to be annotated, have been proposed in literature instead of randomly selecting samples. In this paper, a representative active learning method is proposed, which local consistency of video content is effectively taken into consideration. The main idea is to exploit the global and local statistical characteristics of videos, and the temporal relationship between shots. The global model is trained on a smaller pre-labeled video dataset, and the local information is obtained online in the process of active learning, and will be used to adjust the initial global model adaptively. The experiment results show that the proposed active learning scheme has significantly improved the annotation performance compared with random selecting and common active learning method.

As a very important category in sports video, racquet sports video, e.g. table tennis, tennis and badminton, has been paid little attention in the past years. Considering the characteristics of this kind of sports video, we propose a new scheme for structure indexing and highlight generating based on the combination of audio and visual information. Firstly, a supervised classification method is employed to detect important audio symbols including impact (ball hit), audience cheers, commentator speech, etc. Meanwhile an unsupervised algorithm is proposed to group video shots into various clusters. Then, by taking advantage of temporal relationship between audio and visual signals, we can specify the scene clusters with semantic labels including rally scenes and break scenes. Thirdly, a refinement procedure is developed to reduce false rally scenes by further audio analysis. Finally, an exciting model is proposed to rank the detected rally scenes from which many exciting video clips such as game (match) points can be correctly retrieved. Experiments on two types of representative racquet sports video, table tennis video and tennis video, demonstrate encouraging results.

In this paper, we present a learning-based approach to mining the capture intention of camcorder users, aiming at providing a novel viewpoint in terms of home video content analysis. In contrast to existing approaches to video analysis designed from the viewers' standpoint, this approach models the capture intention from a camcorder user's point of view, by investigating a set of effective intention oriented features. With this approach, not only the capture intention is effectively mined, but also a set of intention probability curves are produced for efficient browsing of home video content. The experimental evaluations indicate that the intention based approach is an effective complement to existing home video content analysis schemes.

Highlight detection is a form of video summarization techniques aiming at including the most expressive or attracting parts in the video. Most video highlights selection research work has been performed on sports video, detecting certain objects or events such as goals in soccer video, touch down in football and others. In this paper, we present a highlight detection method for film video. Highlight section in a film video is not like that in sports video that usually has certain objects or events. The methods to determine a highlight part in a film video can exhibit as three aspects: (a) locating obvious audio event, (b) detecting expressive visual content around the obvious audio location, (c) selecting the preferred portion of the extracted audio-visual highlight segments. We define a double filters model to detect the potential highlights in video. First obvious audio location is determined through filtering the obvious audio features, and then we perform the potential visual salience detection around the potential audio highlight location. Finally the production from the audio-visual double filters is compared with a preference threshold to determine the final highlights. The user study results indicate that the double filters detection approach is an effective method for highlight detection for video content analysis.

Fingerprints are useful for biometric purposes because of their well known properties of distinctiveness and persistence over time. However, owing to skin condition or incorrect finger pressure, some fingerprint images contain useless components. They are often mistaken for the terminations that are an essential minutia of a fingerprint. Mathematical Morphology (MM) is one powerful tool in image processing. The performance of MM is heavily dependent on structuring elements (SEs), but finding an optimal SE is a difficult and nontrivial task. In this work, we focus on automatically choosing an appropriate SE for eliminating useless components of fingerprint images using linear time Euclidean distance transform algorithm. We exploit the property that these useless components are thinner than fingerprint ridges. So, we estimate the width of useless components and fingerprint ridges using Euclidean distance transform values, and then provide a linear time algorithm to eliminate the useless components. We have developed a novel algorithm for the binarization of fingerprint images. The information of distance transform values can be obtained directly from the binarization phase. The results show that using this method on fingerprint images with useless components is faster than existing methods and achieves better quality than earlier methods.

Image processing technology is widely used in various consumer electronics products. Design of image filters plays a key role in image processing. And the design is further dependent on the target application. Moving from one application to other, the image filter has to be changed accordingly. Therefore, in this paper, high performance cost effective generalized filter architecture is proposed to handle various kind of noise in the image for applications like LCD display, TV noise etc, and image enhancer and edge detection. Proposed filter architecture is based on the concept of FIR filter and coring technique. Also we introduce a new scaling concept to take care of growing bits during arithmetic operation of filter. The filter architecture is fully pipelined, and implemented in the 90nm technology. The estimated gate count for the proposed filter is 50k, if used for all image components (Y/R, CB/G and CR/B) and the estimated frequency of operation is 500 MHz.

A large number of comb filtering techniques for a national television system committee (NTSC) or phase alternate each line (PAL) color decoding have been researched and developed for the last three decades. Comb filtering can separately obtain the luminance and the quadrature amplitude modulation (QAM) modulated chrominance information from a composite video burst signal (CVBS). However there is a difficulty in extracting the luminance and chrominance components from a composite video image because the cross-talk between them gives undesirable image artifacts. The three-dimensional (3-D) comb filter using spatio-temporal filtering kernel and adaptive two-dimensional (2-D) neural-based comb-filtering approach was developed to alleviate the dot crawl artifacts; however it shows limitation on color decoding. This paper presents an effective dot crawl artifact reduction algorithm in a composite video signal, in which undesirable dot crawl artifact is significantly reduced without losing fine image details. The proposed composite video artifact removal algorithm filters only detected candidate regions specified by dot crawl artifact decision map. The possible comb-filtering error region is generated on video image using luminance and chrominance edge information. Simulation and analysis show that the proposed algorithm with nonlinear bilateral filtering removes efficiently the dot crawl artifacts on composite video image and supports improving further video enhancement techniques.

In general, wavelet coefficients are composed of a few large coefficients and a lot of small ones. Therefore, each wavelet coefficient is efficiently modeled as a random variable of a Bernoulli-Gaussian mixture distribution with unknown parameters. The Bernoulli-Gaussian mixture is composed of the multiplication of the Bernoulli random variable and the Gaussian mixture random variable. In this paper, we propose a denoising algorithm using the Bernoulli-Gaussian mixture model based on sparse characteristics of the wavelet coefficient. The denoising is performed with Bayesian estimation. We present an effective denoising method through simplified parameter estimation for the Bernoulli random variable using a local expected square error. Simulation results showed that our method outperformed the states of the art denoising methods.

When images compressed with block-based compression techniques are transmitted over a noisy channel, unexpected block losses occur. Conventional methods that do not consider edge directions can cause blocked blurring artifacts. In this paper, we present a post-processing-based block recovery scheme using Haar wavelet features. The adaptive selection of neighboring blocks is performed based on the energy of wavelet subbands (EWS) and difference between DC values (DDC). The lost blocks are recovered by linear interpolation in the spatial domain using selected blocks. The method using only EWS performs well for horizontal and vertical edges, but not as well for diagonal edges. Conversely, only using DDC performs well for diagonal edges with the exception of line- or roof-type edge profiles. Therefore, we combine EWS and DDC for better results. The proposed directional recovery method is effective for the strong edge because exploit the varying neighboring blocks adaptively according to the edges and the directional information in the image. The proposed method outperforms the previous methods that used only fixed blocks.

The search for efficient image denoising methods still is an open question. In this paper, we develop a universal algorithm based on grey relational analysis for similar binary image denoising. After taking the differences in spatial distribution between noise and edge into consideration, we select two referential sequences to represent inner off/on pixels, and a group of comparative sequences to stand for the pixels to be processed. Then, by analyzing the grey relational coefficients of the two kinds of sequences, we distinguish edge pixels from non-edge pixels, and reset the non-edge pixels to binary, thereby noise reduced and edges kept. The interest of the method lies in the fact that, without any precedent knowledge, it not only can reduce speckle, salt & pepper and gaussian noise at one time but also provide a trade-off between edge reservation and noise reduction via grey relational threshold. Experimental results show that the method obviously outperforms the three conventional spatial filters: median filter, wiener filter and mean filter. Possible applications include the processing before the recognition of printed or handwritten character, vehicle license plate, and the optimization of scanned binary trademark, engineering drawing and extracted binary watermark.

The extension of H.264/AVC hybrid video coding towards scalable video coding (SVC) using motion-compensated temporal filtering (MCTF) is presented. Utilizing the lifting approach to implement MCTF, the motion compensation features of H.264/AVC can be re-used for the MCTF prediction step and extended in a straightforward way for the MCTF update step. The MCTF extension of H.264/AVC is also incorporated into a video codec that provides SNR, spatial, and (similar to hybrid video coding) temporal scalability. The paper provides a description of these techniques and presents experimental results that validate their efficiency. In addition applications of SVC to video transmission and video surveillance are described.

In most of the existing in-band wavelet video coding schemes, over-complete form of reference bands is used to solve the wavelet shift-variance problem on motion compensated temporal filtering (MCTF) in wavelet domain. It can more or less improve the coding efficiency of the in-band MCTF scheme. However, there is a dilemma always existing when the input video is decomposed into a low-pass band and several high-pass bands. If the temporal filtering in the lowpass band does not use any information from the high-pass bands, the coding efficiency will decrease significantly for the higher resolution reconstructed video. If the temporal filtering in the low-pass band does use data from the high-pass bands, it will bring serious drifting error when the high-pass bands are not available at the decoder. In this paper, for the latter case, we analyze when drifting errors occur and how they will propagate along the lifting structure for decoding at the lower resolution. Based on the analysis, we introduce two new inter coding modes at macroblock level for spatial low-pass band MCTF to make a better trade-off between the lower resolution drifting error and the high resolution coding efficiency. Furthermore, we present the criterion to adaptively select the proper coding mode for each macroblock of spatial low-pass band. The experiment results show that our proposed scheme can dramatically reduce the drifting error about 0.4~2.6dB for different bit rates at low resolution, while for the high resolution, the performance loss is marginal.

In this paper, we propose a low complexity prioritized bit-plane coding scheme to improve the rate-distortion performance of cyclical block coding in MPEG-21 scalable video coding. Specifically, we use a block priority assignment algorithm to firstly transmit the symbols and the blocks with potentially better rate-distortion performance. Different blocks are allowed to be coded unequally in a coding cycle. To avoid transmitting priority overhead, the encoder and the decoder refer to the same context to assign priority. Furthermore, to reduce the complexity, the priority assignment is done by a look-up-table and the coding of each block is controlled by a simple threshold comparison mechanism. Experimental results show that our prioritized bit-plane coding scheme can offer up to 0.5dB PSNR improvement over the cyclical block coding described in the joint scalable verification model (JSVM).

Let the whole region be the whole bit rate range that customers are interested in, and a sub-region be a specific bit rate range. The weighting factor of each sub-region is determined according to customers' interest. A new type of region of interest (ROI) is defined for the SNR scalability as the gap between the coding efficiency of SNR scalability scheme and that of the state-of-the-art single layer coding for a sub-region is a monotonically non-increasing function of its weighting factor. This type of ROI is used as a performance index to design a customer oriented SNR scalability scheme. Our scheme can be used to achieve an optimal customer oriented scalable tradeoff (COST). The profit can thus be maximized.

The shift-variant property of discrete wavelet transform leads to the intrinsic coupling across various spatial subbands during motion aligned temporal filtering. It brings a big challenge how to achieve a better trade-off between subband independency and motion alignment efficiency in the context of providing spatial scalability in 3D wavelet video coding. This paper first investigates the issue of subband coupling in-depth. From the investigations on analysis and synthesis filters, we verify the existence and causes of subband coupling phenomenon and illustrate the subband leakage due to motion shift. Furthermore, we propose a method to measure the strength of this kind of coupling. Based on these investigations, we focus on schemes which preserving most subband coupling relationship and recommend that spatial highpass subbands should not be dropped completely. The issue of rate allocation for spatial highpass subbands is considered in this paper. An error propagation model is proposed to describe the effect of subband coupling in video reconstruction. The synthesis gain of each subband is estimated according to this model and it is finally adopted to guide the rate allocation algorithm. Experimental results have fully demonstrated the promising of the proposed techniques in improving both objective and subjective qualities of low resolution video, especially at middle and high bit rates, for 3D video coding schemes with spatial scalability.

Some of the most powerful schemes for scalable video coding are based on the so-called t+2D paradigm. In these schemes, temporal redundancy is first exploited through a motion-compensated multiresolution decomposition and the resulting temporal subband frames are then generally spatially decomposed with a wavelet transform. Thus, temporal and spatial scalability are achieved, in particular when also the motion information is properly managed between resolution levels. However, spatial wavelet transform may not be the most appropriate for exploiting the spatial redundancy of the detail subbands, since very often the power spectral density of these frames is not as concentrated at low frequencies as in the case of natural images. Recently, we have shown that orthonormal 4-band transforms can provide similar or even better results, especially for textured video sequences. In this paper, we elaborate on this idea and compare several M-band transforms for spatial decomposition of the temporal detail frames. Several M-band filter bank designs are given and lapped transforms are also considered. We show by simulation results that lapped transforms achieve a rate versus distortion performance that is comparable and sometimes better to that of the dyadic biorthogonal 9/7 wavelet transform, for a lower complexity.

A hierarchical structure for sports event classification based on audio and video content analysis is proposed in this paper. Compared to the event classifications in other games, those of cricket are very challenging and yet unexplored. We have successfully solved cricket video classification problem using a six level hierarchical structure. The first level performs event detection based on audio energy and Zero Crossing Rate (ZCR) of short-time audio signal. In the subsequent levels, we classify the events based on video features using a Hidden Markov Model implemented through Dynamic Programming (HMM-DP) using color or motion as a likelihood function. For some of the game-specific decisions, a rule-based classification is also performed. Our proposed hierarchical structure can easily be applied to any other sports. Our results are very promising and we have moved a step forward towards addressing semantic classification problems in general.

Shot boundary detection servers as a preliminary step to structure the content of videos. Up to now, a large number of methods have been proposed. We give a brief overview of previous works with a novel view, focusing on the solutions of the two main disturbances, i.e., abrupt illuminance change and great camera or object motion. Then this paper presents a novel shot boundary detection framework, consisting of three components: fade out/in (abbreviated as FOI) detector, cut detector and gradual transition (abbreviated as GT) detector. The key technique of FOI detector is the recognition of monochrome frames. For cut detection, a second-order difference method is firstly applied to obtain candidate cuts, and then a post-processing procedure is taken to eliminate the false positives. In GT detector, the twin-comparison approach is employed to detect short gradual transition which lasts less than six frames, while for long gradual transition, an improvement of twin-comparison algorithm is designed. Firstly, to effectively reduce the false alarms of quick motion, the lower threshold is self-adaptive to motion feature. Secondly, an FSA (finite state automata) model is adopted to replace the twin-comparison strategy. This framework makes good use of various features and successfully integrates all the modules together. Finally, the system is evaluated on the TRECVID benchmarking platform and the experimental results reveal the effectiveness of our system.

In this paper, we propose a novel statistical method for video analysis and object-oriented structured video representation. Although the existing shot-based approaches facilitate our access to video than the raw video data do, they are still not effective for the semantic level video browsing and retrieval. To represent video structure on semantic levels, the proposed method employs an extended hidden Markov model trained by EM algorithm in order to explore the meaningful objects and the important video structures. According to the principles of the proposed method, important objects and video structure with the consideration of short term statistic as well as long term recurrence can be captured and several interesting video analysis tasks can also be performed. Finally, experiments based on several different videos validate the effectiveness of the proposed approach.

Video shot clustering based on visual feature extraction is an important and challenging task in video browsing and retrieval systems. We propose in this paper a novel method for video shot grouping directly processed in MPEG domain. This method places emphasis on extracting shot features in a way that reflects viewer's actual visual perception. In particular, a spatio-temporal human attention model is constructed. Spatially, each frame is split into two areas, attention region and background, indicating two independent parts of viewer's attention: interest of a dominant motive region and a global impression of the surroundings. Temporally, a weighted color histogram is used to emulate human attention affected by camera motion. With this camera attention weighted strategy, a set of shot similarity measures are constructed. Shot grouping based on these similarities achieve promising results in our experiments on MPEG-7 test videos.

There are growing needs for quick preview of video contents for the purpose of improving accessibility of video archives as well as reducing network traffics. In this paper, a storyboard that contains a user-specified number of keyframes is produced from a given video sequence. It is based on hierarchical cluster analysis of feature vectors that are derived from wavelet coefficients of video frames. Consistent use of extracted feature vectors is the key to avoid a repetition of computationally-intensive parsing of the same video sequence. Experimental results suggest that a significant reduction in computational time is gained by this strategy.

In this paper, we present a new and efficient clustering approach for scene analysis in sports video. This method is generic and does not require any prior domain knowledge. It performs in an unsupervised manner and relies on the scene likeness analysis of the shots in the video. The two most similar shots are merged into the same scene in each iteration. And this procedure is repeated until the merging stop criterion is satisfied. The stop criterion is defined based on a J value which is defined according to the Fisher Discriminant Function. We call this method J-based Scene Clustering. By using this method, the low-level video content representation-shots could be clustered into the midlevel video content representation-scenes, which are useful for high-level sports video content analysis such as playbreak parsing, story units detection, highlights extraction and summarization, etc. Experimental results obtained from various types of broadcast sports videos demonstrate the efficacy of the proposed approach. Moreover, in this paper, we also present a simple application of our scene clustering method to story units detection in periodic sports videos like archery video, diving video and so on. The experimental results are encouraging.

The key technique of 3-D GIS is to realize quick and high-quality 3-D visualization, in which 3-D roaming system based on landform plays an important role. However how to increase efficiency of 3-D roaming engine and process a large amount of landform data is a key problem in 3-D landform roaming system and improper process of the problem would result in tremendous consumption of system resources. Therefore it has become the key of 3-D roaming system design that how to realize high-speed process of distributed data for landform DEM (Digital Elevation Model) and high-speed distributed modulation of various 3-D landform data resources. In the paper we improved the basic ant algorithm and designed the modulation strategy of 3-D GIS landform resources based on the improved ant algorithm. By initially hypothetic road weights σ_i , the change of the information factors in the original algorithm would transform from ▵τ_j to ▵τ_j+σ_i and the weights was decided by 3-D computative capacity of various nodes in network environment. So during the course of initial phase of task assignment, increasing the resource information factors of high task-accomplishing rate and decreasing ones of low accomplishing rate would make load accomplishing rate approach the same value as quickly as possible, then in the later process of task assignment, the load balanced ability of the system was further improved. Experimental results show by improving ant algorithm, our system not only decreases many disadvantage of the traditional ant algorithm, but also like ants looking for food effectively distributes the complicated landform algorithm to many computers to process cooperatively and gains a satisfying search result.

We propose a new algorithm that efficiently fills the unknown regions of an image caused by replacement or removal of part of the foreground. Previous approaches to inpainting and texture synthesis are either fast, but not applicable for small-scale regions, or slow, but fills large regions with good quality, but is slow. The algorithm in this paper is based on the patch-based best-fit searching strategy, in which a partly-known patch is filled by searching the known part of the image for a patch of pixels closely matching the known neighbors. This keeps the linear structure and texture of the image. We found that the exhaustive searching strategies used in previous work are the main source of inefficiency; most matches are located in the neighborhood of the target patch. Inspired by this spatial continuity, we introduce a more efficient searching strategy.

This paper presents a new method which allows the artificial creation of cast shadows in the context of video post-production applications for which video objects are inserted in an original video sequence. The manipulated video objects are assumed to have been previously segmented together with their cast shadows in their original images, and to have been inserted in the edited images. This paper deals with the problem of the artificial creation of the corresponding cast shadows. A reference image which represents the scene illuminated background is assumed to be available. The proposed method is based on a precise shadow model and on the use of the 3-D structure of the scene background. This 3-D information is roughly obtained manually in a pre-processing phase. Then, the illumination conditions, i.e. a shadow model and the light source position are estimated. The shape and texture of the created cast shadows are defined by combining geometrical relations linking the light source, the video objects and the cast shadows, and by applying the inverse shadow model to artificially shade the shadow areas. Experimental results show that it is possible to obtain satisfactory visual results.

This work presents a technique for achieving multiresolution representations of deformable objects used in virtual reality applications. The proposed model may define different levels of detail inside the objects in order to obtain a plausible sensorial feedback answer. The technique here described uses a subdivision scheme based on wavelet transforms and a dynamic model based in a mass-spring representation. The dynamic model allows the simulation of the whole volume behaviour incorporating a skeleton that reduces the number of nodes managed by other existing representations and improves its stability too. The paper presents an analysis of the performance achieved with the proposed model together with some examples used in the tests. The results show that the dynamic behaviour of the model is stable and coherent at different resolution levels.

We present a complete system for the automatic creation of talking head video sequences from text messages. Our system converts the text into MPEG-4 Facial Animation Parameters and synthetic voice. A user selected 3D character will perform lip movements synchronized to the speech data. The 3D models created from a single image vary from realistic people to cartoon characters. A voice selection for different languages and gender as well as a pitch shift component enables a personalization of the animation. The animation can be shown on different displays and devices ranging from 3GPP players on mobile phones to real-time 3D render engines. Therefore, our system can be used in mobile communication for the conversion of regular SMS messages to MMS animations.

Augmented reality (AR) has recently gained significant attention. The previous AR techniques usually need a fiducial marker with known geometry or objects of which the structure can be easily estimated such as cube. Placing a marker in the workspace of the user can be intrusive. To overcome this limitation, we present an AR system using invisible markers which are created/drawn with an infrared (IR) fluorescent pen. Two cameras are used: an IR camera and a visible camera, which are positioned in each side of a cold mirror so that their optical centers coincide with each other. We track the invisible markers using IR camera and visualize AR in the view of visible camera. Additional algorithms are employed for the system to have a reliable performance in the cluttered background. Experimental results are given to demonstrate the viability of the proposed system. As an application of the proposed system, the invisible marker can act as a Vision-Based Identity and Geometry (VBIG) tag, which can significantly extend the functionality of RFID. The invisible tag is the same as RFID in that it is not perceivable while more powerful in that the tag information can be presented to the user by direct projection using a mobile projector or by visualizing AR on the screen of mobile PDA.

Due to the large expansion of heterogeneous networks, the scalability features of next generation video coding schemes have become a very important topic for future developments. These algorithms enable to generate a universal (temporally, spatially and SNR) scalable bit-stream so that the video may be decodable by any kind of device from a mobile phone with narrow bandwidth and small display to a PC with DSL connection and large TFT display. In this paper we present a new approach to improve the SNR scalability for scalable video schemes by combining motion information scalability and texture updates in order to get an optimal trade-off between motion and texture for a given bit-rate. Consequently we can better exploit the redundancy between successive frames than the current strategies based on requantization and/or bit-plane coding.

Scalable video coding (SVC) has a great advantage due to its very easy adaptation to unpredictable bandwidth variations and network conditions. However, in some scalable video coding schemes, either a single prediction is used, which leads to either drift or coding inefficiency, or a different prediction is obtained for each reconstructed version, which leads to add complexity. So, the new techniques have to be developed to improve the performance of the SVC scheme. Wyner-Ziv coding gives us the surprising insight that efficient data compression can also be achieved by exploiting source statistics-partially or wholly-at the decoder only. Based on the Wyner-Ziv framework, Xu and Xiong proposed a Layer Wyner-Ziv scheme (LWZ) by treating a standard coded video as a base layer, and building the bit-plane enhancement layer using Wyner-Ziv coding. The LWZ scheme has error robustness and channel adaptation. However, the coding efficiency gap between the nonscalable video coding and LWZ is very large. The reason is that only the current base layer and lower enhancement layers are used as side information, and the correlation between the temporal adjacent frames isn't utilized. In order to improve the coding efficiency of LWZ, a scalable video coding based on Wyner-Ziv (SVC-WZ) is proposed in this paper. In our SVC-WZ scheme, we try to use the correlation between temporal adjacent frames rather than always use the correlation between the enhancement layers and the current base layer as SI during the enhancement layer encoding. Using the correlation between temporal adjacent frames would make SI more accurate, thus it could improve the coding efficiency. Our experimental results show that the SVC-WZ scheme can achieve consistently better coding efficiency than the LWZ scheme while keeping all the properties of LWZ.

Scalable video coding (SVC) has become more and more important with the enrichment of multimedia data and the diversification of network and terminal devices. In current MPEG SVC activities, a scalable extension of H.264/AVC, called scalable video model (SVM), is proposed by HHI and has shown further coding efficiency improvement and scalability functionality. However, the SVM model doesn't provide an efficient rate control scheme now, and rate control is achieved through a full search for selecting a suitable quantization parameter (QP). That is very inefficient and much time-consuming. In this paper, an efficient rate control scheme is proposed for the SVM, which is derived from the state-of-the-art hybrid rate control schemes of JVT with some considerations for scalable video coding. In the proposed rate control scheme, the rate distortion optimization (RDO) involved in the step of encoding temporal subband pictures is only implemented on the low-pass subband pictures, and rate control is independently applied to each spatial layer. For each spatial layer, the rate control is implemented at GOP, picture and basic unit levels. Furthermore, for the temporal subband pictures obtained from the motion compensation temporal filtering (MCTF), the target bit allocation and quantization parameter selection inside a GOP could make full use of the hierarchical relations inherent from the MCTF. The proposed rate control scheme has been implemented into SVM3.0 and experiment results show that the proposed algorithm can achieve the target bit rate with little bit rate fluctuation and keep fine image quality at the same time, but the computation complexity is reduced heavily.

In this paper a new fully scalable - wavelet based - video coding architecture is proposed, where motion compensated temporal filtered subbands of spatially scaled versions of a video sequence can be used as base layer for inter-scale predictions. These predictions take place between data at the same resolution level without the need of interpolation. The prediction residuals are further transformed by spatial wavelet decompositions. The resulting multi-scale spatiotemporal wavelet subbands are coded thanks to an embedded morphological dilation technique and context based arithmetic coding. Dyadic spatio-temporal scalability and progressive SNR scalability are achieved. Multiple adaptation decoding can be easily implemented without the need of knowing a predefined set of operating points. The proposed coding system allows to compensate some of the typical drawbacks of current wavelet based scalable video coding architectures and shows interesting visual results even when compared with the single operating point video coding standard AVC/H.264.

High Definition Television is currently a hot topic that generates research and commercial interests in video industry. This paper relates to video coding devices and methods for easy and fast conversion of video formats for digital TV. The proposed algorithm provides a generic framework that allows any video format conversions in a scalable way. We focused particularly on solutions to extract Standard Definition video signals from High Definition ones. Practically, we address spatial scalability solutions between video formats that are not linked up by dyadic decomposition and/or istropic decomposition. The challenge is to generate a single scalable bitstream in a compliant way that permits any spatial resolution decodings while saving bandwidth compared to classical simulcast approaches. Practically, our scalable coder is based on a multi-layer approach and on frame scales and borders introduction. Moreover, to encode/decode the macroblocks of the high resolution pictures, usual video coding approaches exploit the inheritance knowing the macroblocks of the decoded low resolution pictures. Approaches for inter-layer prediction of motion data have already been proposed for dyadic decompositions but do not work in case of non dyadic inter-layer resolutions. Consequently we propose a different managing of the inter layer prediction. All these techniques have been successfully implemented in the MPEG-SVC reference software. Obtained results achieve good coding efficiency and are comparable to the simulcast state of the art while providing features induced by scalability.

It is well-known that the problem of addressing heterogeneous networks in multicast can be solved by simultaneous transmission of multiple bitstreams of different bitrates and by layered encoding. This paper analyzes the use of H.264/AVC video coding in simulcast and for layered encoding. The sub-sequence feature of H.264/AVC enables hierarchical temporal scalability, which allows disposal of reference pictures from a coded bitstream without affecting the decoding of the remaining stream. In this paper we extend the scope of the H.264/AVC sub-sequence coding technique to quality scalability. The resulting quality scalable coding technique is similar to conventional coarse-granularity quality scalability but fully compatible with the H.264/AVC standard. It is found that the proposed method drops bitrate consumption in the core network compared to simulcast up to 20%. However, the bitrate required for enhanced-quality reception for scalably coded bitstreams is considerably higher than that of non-scalable bitstreams.

In content based image retrieval similarity measurement is one of the most important aspects in a large image database for efficient search and retrieval to find the best answer for a user query. Color and texture are among the more expressive of the visual features. Considerable work has been done in designing efficient descriptors for these features for applications such as similarity retrieval. The MPEG-7 specifies a standard set of descriptors for color, texture and shape. In the Human Vision System (HVS), visual information is not perceived equally; some information may be more important than other information. The purpose of this paper is to show how the MPEG-7 descriptor based on human vision system can be efficiently utilized for image matching. We considered scalable color descriptor (SCD) and edge histogram descriptor (EHD) descriptors of MPEG7. To increase the matching performance we used Linde-Buzo-Gray (LBG) vector quantization algorithm and image splitting by magnitude weighting technique for efficient use of SCD and EHD, respectively. The proposed matching method is considered to be a more efficient image content-based retrieval than EHD and SCD. Experimental results support this claim. Experiments on 3000 images from Corel Photo collections show that the proposed method yields better retrieval performance especially for semantic similarity.

We present an application of image analysis techniques to automatically annotate biological images depicting gene expression patterns in developing embryos of fruit fly (Drosophila melanogaster), a model organism to study gene interaction. The aim is to determine the view (lateral versus dorsal/ventral [non-lateral]), orientation (anterior-left or anterior-right), and the developmental stage of the embryo. We employed contour curvature analysis, symmetry of the gene expression patterns, and shape differences at the anterior and posterior ends of the embryo, among others, for these purposes. An analysis of a pilot database of 3500 images indicates that view was correctly identified in 62%, orientation in 85%, and developmental stage in 73% of the images. We observed that correct inferences had better separation in feature space than incorrect inferences. This means that, although these methods do not exhibit very high classification accuracy, they could be employed to identify images which need manual intervention, thereby reducing the target set for biologists. The novelty in this work is in the integration of well-established image analysis with the biological knowledge for annotating the embryos. Our examinations show that features that provide discrimination ability among different views, different orientations, and different developmental stages are often restricted to certain regions of the embryo, which agrees with the longstanding knowledge in the developmental biological community.

This paper presents a fingerprinting method based on equivalence classes. An equivalence class is composed of a reference image and all its variations (or replicas). For each reference image, a decision function is built. The latter determines if a given image belongs to its corresponding equivalence class. This function is built in three steps: synthesis, projection, and analysis. In the first step, the reference image is replicated using different image operators (like JPEG compression, average filtering, etc). During the projection step, the replicas are projected onto a distance space. In the final step, the distance space is analyzed, using machine learning algorithms, and the decision function is built. In this study, three machine learning approaches are compared: orthotope, support vectors machine (SVM), and support vectors data description (SVDD). The orthotope is a computationally efficient ad-hoc method. It consists in building a generalized rectangle in the distance space. The SVM and SVDD are two more general learning algorithms.

In this paper a novel image classification system is proposed. Atmosphere serves an important role in generating the scene's topic or in conveying the message behind the scene's story, which belongs to abstract attribute level in semantic levels. At first, five atmosphere semantic categories are defined according to rules of photo and film grammar, followed by global luminance and hue features. Then the hierarchical SVM classifiers are applied. In each classification stage, corresponding features are extracted and the trained linear SVM is implemented, resulting in two classes. After three stages of classification, five atmosphere categories are obtained. At last, the text annotation of the atmosphere semantics and the corresponding features by Extensible Markup Language (XML) in MPEG-7 is defined, which can be integrated into more multimedia applications (such as searching, indexing and accessing of multimedia content). The experiment is performed on Corel images and film frames. The classification results prove the effectiveness of the definition of atmosphere semantic classes and the corresponding features.

In this paper, separated feature indexing was used for region-based image retrieval. In the beginning, three methods, such as K-means, CA-Clustering and Blobworld, were used for region generation in this paper. Comparing their results, the retrieval results using Blobworld perform better than the other two. Therefore, Blobworld is chosen to generate regions in the following experiments. After generating a few regions in an image, in this paper, a few heterogeneous features, such as color signature, moments, contrast, anisotropy, edge orientation histogram, edge density, perimeter, area, position, etc. were used for region matching. According to the variations of two or more query regions, the weights to each feature were automatically generated. Furthermore, the individual indexing was built and maintained for each heterogeneous feature, i.e., separated feature indexing. Once the retrieval results from separated feature indexing were obtained, two processes about normalization and combining distances from these heterogeneous features should be completed in order to achieve the final retrieval result. In our experiments, two functions were tested for normalizing heterogeneous distances and four functions for combining the normalized distances into a global measure. The experimental results show that the best result can be obtained by using the one based on norm1 and mean2 operations. We used the precision/recall and the average normalized modified retrieval rank (ANMRR) to evaluate the experiments in this paper.

A new fast template matching algorithm is proposed in this work. It extracts the local self-similarity as image feature. Feature matching is based on equal comparison and clustering. Its time complexity reduced to 0(size of search image), the theoretic lower limit for classical image template matching. The new algorithm is generalized to partial matching of two images. By partitioning images into blocks by nona-tree decomposition, it detects the matching regions between blocks and merges the matching results using first order Markov chain in horizontal direction. In the vertical direction merging is implemented directly according neighbor parts connected and keeping the consistent displacements of correspond parts in two image. The algorithm is robust for the linear transformation of image grey value and image noise.

Standard video compression techniques apply motion-compensated prediction combined with transform coding of the prediction error. In the context of prediction with fractional-pel motion vector resolution it was shown, that aliasing components contained in an image signal are limiting the prediction accuracy obtained by motion compensation. In order to consider aliasing, quantisation and motion estimation errors, camera noise, etc., we analytically developed a two-dimensional (2D) non-separable interpolation filter, which is calculated for each frame independently by minimising the prediction error energy. For every fractional-pel position to be interpolated, an individual set of 2D filter coe±cients is determined. Since transmitting filter coefficients as side information results in an additional bit rate, which is almost independent for different total bit rates and image resolutions, the overall gain decreases when total bit rates decrease. In this paper we present an algorithm, which regards the non-separable two-dimensional filter as a polyphase filter. For each frame, predicting the interpolation filter impulse response through evaluation of the polyphase filter, we only have to encode the filter coefficients prediction error. This enables bit rate savings, needed for transmitting filter coe±cients of up to 75% compared to PCM coding. A coding gain of up to 1,2 dB Y-PSNR at same bit rate or up to 30% reduction of bit rate is obtained for HDTV-sequences compared to the standard H.264/AVC. Up to 0,5 dB (up to 10% bit rate reduction) are achieved for CIF-sequences.

An improved intra-prediction method for AVS-M, which is developed by AVS (Audio Video Coding Standard Working Group of China) to meet the requirements of multimedia applications in next-generation mobile communications, is proposed in this paper. Utilizing the spatial and temporal correlation of the video sequences, the proposed method only processes a portion of normative 9 intra-prediction modes to decide the best mode, therefore, speeds up intra prediction with some degradation in the coding gain. Moreover, a complexity-scalable technique is presented to control the complexity of intra prediction, by which the video coder is matched to the limited computational resource of communication devices. The proposed methods are verified. The results show that the improved mode-decision method can significantly save encoding time at expense of little coding loss and the complexity-scalable technique works robustly and accurately. In addition, the proposed method also can be applied to other standards, such as H.264.

MPEG-4 supports object-level video coding. It is a challenge to design an optimal bit allocation strategy which considers not only how to distribute bits among multiple video objects (MVO's) but also how to achieve optimization between the texture and shape information. In this paper, we present a uniform framework for optimal multiple video object bit allocation in MPEG-4. We combine the rate-distortion (R-D) models for the texture and shape information of arbitrarily shaped video objects to develop the joint texture-shape rate-distortion models. The dynamic programming (DP) technique is applied to optimize the bit allocation for the multiple video objects. The simulation results demonstrate that the proposed joint texture-shape optimization algorithm outperforms the MPEG-4 verification model on the decoded picture quality.

Recursive wavelet filters and an alternative algorithm for implementing wavelet transform are presented in this paper. The recursive filters use previously calculated (past) wavelet coefficients as inputs to calculate the current wavelet coefficient, and provide the same transform results as convolutional FIR and lifting wavelet filters. The coefficients of the recursive filters are derived from those of conventional FIR wavelet filters. The wavelet transform with recursive filters requires a smaller amount of memory and is easy to implement in hardware. Another important advantage of the recursive filters is that perfect reconstruction can be easily achieved using recursive wavelet filters if a sequence of pixels to be transformed is extended by boundary pixel repetition. Boundary pixel repetition can be more efficient than the widely used method of symmetric extension for image and video coding.

Residual operations are introduced in many video compression algorithms to exploit temporal redundancy in video sequences. One of the most effective and computationally simple algorithms is 3D-DWT-SPIHT algorithm. Nevertheless, it only utilizes intra-GOF temporal redundancy. In order to eliminate inter-GOF redundancy, the Simple GOF Residual Operation (SGRO) algorithm was introduced, where residual operations are performed every two GOFs. However, when background mutations occur, the PSNR of reconstructed target GOF significantly drops; on the other hand, when video sequences are temporally stationary, SGRO algorithm fails to fully utilize inter-GOF temporal redundancy due to its imperative insertion of no-residual-operation GOFs. In this paper, we propose a new Adaptive GOF Residual Operation (AGRO) algorithm, based upon a criterion on residual operations, which is derived from an empirical formula of image complexity established by us. AGRO algorithm always selects the best residual operation manner according to the contents of video sequences: by detecting contents of video sequences, it cancels residual operations where background mutations happen, while encouraging residual operations where video sequences are temporally stationary. Therefore, AGRO algorithm prevents the significant drop in compression effects resulted from background mutations, and in the meantime, fully utilizes inter-GOF temporal redundancy. In addition, AGRO algorithm demonstrates an innate error-propagation-resistant property. Numerical results show that AGRO algorithm renders a significant PSNR increase over SGRO algorithm whenever background mutations occur or temporally stationary sequences dominate.

Compared to contemporary video coding standards, the H.264 standard achieves higher compression efficiency at the expense of high computational complexity at the encoder to carry out rate distortion optimization (RDO). In this paper, we investigated the encoding speed improvement technology of H.264 and focus on RDO to select the encode mode for fast coding of H.264. Based on the analysis of motion activity information of the macroblock to be coded in inter frame, we proposed a methodology to simplify the computation of encoder while maintaining the coding efficiency. At first, we established the relationships between INTRA mode and motion activity in inter frame. Then the INTRA mode skipping algorithm (IMSA) is proposed and applied to RDO procedure. If the motion activity of a macroblock is small enough, then IMSA adaptively skips the whole INTRA mode decision process because the encoder benefits little from such type of macroblocks. Simulation results demonstrate its valuable efficiency of considerably reducing 30%~60% encoding complexity while incurring little even no loss in quality and increase in bit rate. Additionally, it is easier than other published fast mode decision algorithms and more efficient when working together with them.

Audio Video coding Standard (AVS) is established by the Working Group of China in the same name. AVS-video is an application driven coding standard. AVS Part 2 targets to high-definition digital video broadcasting and high-density storage media and AVS Part 7 targets to low complexity, low picture resolution mobility applications. Integer transform, intra and inter-picture prediction, in-loop deblocking filter and context-based two dimensional variable length coding are the major compression tools in AVS-video, which are well-tuned for target applications. It achieves similar performance to H.264/AVC with lower cost.

Conventional video coding techniques make use of the most recently decoded reference frame(s) for motioncompensated inter prediction. However, it has been shown that to allow using reference frames in a flexible way such that not only the latest reference frames are used is beneficial. A typical use of flexible reference frame is feedback based reference picture selection, wherein error-free reference frames available in both the encoder and decoder sides are selected and used for inter prediction reference. This paper first overviews support of reference picture selection in different video coding standards, and then presents three specific feedback based reference picture selection methods using flexible reference frames. In addition, a novel simple reference frame management method that enables using of flexible reference frame is proposed. The reference frame management method enables much simpler video codec implementations compared to the complex reference frame management methods in H.263 Annex U and H.264/AVC. The proposed coding methods and some conventional methods are compared with each other. Simulation results show significantly improved error resiliency performance of the proposed reference picture selection methods compared to conventional methods. The effect on the performance imposed by feedback delay variation is also shown. Thanks to the merits, support of flexible reference frame and the reference frame management has been adopted to the AVS-M video coding standard.

AVS is a new digital audio-video coding standard established by China. AVS will be used in digital TV broadcasting and next general optical disk. AVS adopted many digital audio-video coding techniques developed by Chinese company and universities in recent years, it has very low complexity compared to H.264, and AVS will charge very low royalty fee through one-step license including all AVS tools. So AVS is a good and competitive candidate for Chinese DTV and next generation optical disk. In addition, Chinese government has published a plan for satellite TV signal directly to home(DTH) and a telecommunication satellite named as SINO 2 will be launched in 2006. AVS will be also one of the best hopeful candidates of audio-video coding standard on satellite signal transmission.

The new video coding standard known as H.264/MPEG-4 Advanced Video Coding (AVC), now in its fourth version, has demonstrated significant achievements in terms of coding efficiency, robustness to a variety of network channels and conditions, and breadth of application. The recent fidelity range extensions have further improved compression quality and further broadened the range of applications, and the recent corrigenda have excised the inevitable errata of the initially-approved versions of the specification. Patent licensing programs have begun, the standard has been adopted into a variety of application specifications, and products suitable for widespread deployment have begun to appear. New work toward the near-term development of scalable video coding (SVC) extensions is also under way. This paper does not attempt to review the details of the H.264/MPEG-4 AVC technical design, as that subject has been covered already in a number of publications. Instead, it covers only the high-level design characteristics and focuses more on the recent developments in the standardization community and the deployment status of the specification.

VC-1 is a state-of-the-art video compression format that provides highly competitive video quality, from very low through very high bit rates, at reasonable computational complexity. The technology includes several innovations to improve the rate-distortion performance envelope of the format and to provide additional tools for perceptual quality improvement. This paper describes some of the key innovations of VC-1 that contributes to its quality and complexity advantages. The focus of this paper is on those tools which are unique to VC-1, and differentiate it from other video compression standards.

With the increasing use of digital imaging in general consumer applications, there is a great deal of interest in developing new products that increase the value and enjoyment level of viewing digital images in consumers' living rooms. One way to enrich image viewing and sharing is to combine images with voice annotation and music. A picture VCD system (PVCD) was developed for multimedia authoring, centered around and driven by still photos, with an emphasis on composing still images with sound, including music and spoken annotations. We describe the overall system, as well as major enabling technology components, including multimedia authoring, semantic image classification, and cross-media indexing. The finished multimedia bit stream is primarily recorded on DVD or VCD to facilitate enriched enjoyment through a TV set, but can also be shared on a desktop/laptop, via email, or online.

Video motion analysis and its applications have been a classic research topic for decades. In this paper, we explore the problem of real time video semantics understanding based on motion information. The work can be divided into two segments: global / camera motion estimation and object motion analysis. The former involves optical flow analysis and semantic meaning parsing, and the latter involves object detection and tracking. Although each of these topics has been studied extensively in the literature, a thorough system combining all of them without human intervention, especially under a real time application scenario, is still worthy of further investigation. In this paper we develop our approach toward such a destination and propose an integral architecture. The usability and efficiency of the proposed system have been demonstrated through experiments. Results of this project have numerous applications in digital entertainment, such as video and image summarization, annotation, retrieval and editing.

We present a framework to detect the regions of singing voice in musical audio signals. This work is oriented towards the development of a robust transcriber of lyrics for karaoke applications. The technique leverages on a combination of low-level audio features and higher level musical knowledge of rhythm and tonality. Musical knowledge of the key is used to create a song-specific filterbank to attenuate the presence of the pitched musical instruments. This is followed by subband processing of the audio to detect the musical octaves in which the vocals are present. Text processing is employed to approximate the duration of the sung passages using freely available lyrics. This is used to obtain a dynamic threshold for vocal/ non-vocal segmentation. This pairing of audio and text processing helps create a more accurate system. Experimental evaluation on a small database of popular songs shows the validity of the proposed approach. Holistic and per-component evaluation of the system is conducted and various improvements are discussed.

With the proliferation of camera equipped cell phones and the deployment of the higher data rate 2.5G and 3G infra structure systems, providing consumers with video-equipped cellular communication infrastructure is highly desirable, and can drive the development of a large number of valuable applications. However, for an uplink wireless channel, both the bandwidth and battery energy in a mobile phone are limited for video communications. In this paper, we pursue an energy efficient video communication solution through joint video summarization and transmission adaptation over a slow fading wireless channel. Coding and modulation schemes and packet transmission strategy are optimized and adapted to the unique packet arrival and delay characteristics of the video summaries. In additional to the optimal solution, we also propose a heuristic solution that is greedy but has close to optimal performance. Operational energy efficiency-summary distortion performance is characterized under an optimal summarization setting. Simulation results show the advantage of the proposed scheme with respect to energy efficiency and video transmission quality.

Home video collections constitute an important source of content to be experienced within the digital entertainment context. To make such content easy to access and reuse, various video analysis technologies have been researched and developed to extract video assets for management tasks, including video shot/scene detection, keyframe extraction, and video skimming/summarization. However, one less addressed issue is to investigate how useful those assets are in helping consumers managing their video collections and the usage pattern of the assets. In this paper, we present Personal Video Manager, both as a home video management system and an explorative research platform to enable a systematic analysis and understanding of consumers' demand on video assets and video processing technologies. For understanding consumer's interest, PVM adopts database management technologies to model and archive how consumers identify video assets and utilize them for management tasks. The PVM mining engine performs data mining on such archived data to mine useful knowledge of consumer's preference on video assets and behavior on utilizing the assets. As revealed in the experiment, consumer's interaction embeds rich information to be leveraged in developing more effective video analysis technologies.

In this paper, an efficient method for human facial expression recognition is presented. Our method includes two major techniques: spatially maximum occurrence model (SMOM), which is used to describe the different facial expressions; and elastic shape-texture matching (ESTM), which is used to compute the similarity between two images. The combination of these two techniques, namely the SMOM-ESTM method, is used to classify the facial expressions. With our approach, the recognition rate based on the AR database is 94.5%.

This paper presents a complete functional system capable of detecting people and tracking their motion in either live camera feed or pre-recorded video sequences. The system consists of two main modules, namely the detection and tracking modules. Automatic detection aims at locating human faces and is based on fusion of color and feature-based information. Thus, it is capable of handling faces in different orientations and poses (frontal, profile, intermediate). To avoid false detections, a number of decision criteria are employed. Tracking is performed using a variant of the well-known Kanade- Lucas-Tomasi tracker, while occlusion is handled through a re-detection stage. Manual intervention is allowed to assist both modules if required. In manual mode, the system can track any object of interest, so long as there are enough features to track.

Non-frontal illumination of objects may cause specular reflections and strong self-shadowing. Those phenomena change the appearance of objects to such an extent that they may not be recognized properly. We propose a method to automatically discard the areas of the image which are degraded beyond recovery by adverse illumination conditions. The method is based on a comparison between local variances of image gradient and is computationally efficient. We show that proposed method, implemented with a face verification system based on local DCTmod2 features and a GMM classifier, reduces total recognition errors in the presence of changing directional illumination conditions. Consequently, we show that proposed segmentation method can be used as an automatic estimator of mismatch between the illumination conditions present during the acquisition of training and testing images. We propose an adaptive thresholding scheme that uses the mismatch estimate to further reduce the recognition error.

In this paper, a real-time method is proposed as a solution to the problem of facial expression classiffication in video sequences. The user manually places some of the Candide grid nodes to the face depicted at the first frame. The grid adaptation system, based on deformable models, tracks the entire Candide grid as the facial expression evolves through time, thus producing a grid that corresponds to the greatest intensity of the facial expression, as shown at the last frame. Certain points that are involved into creating the Facial Action Units movements are selected. Their geometrical displacement information, de ned as the coordinates' difference between the last and the first frame, is extracted to be the input to a six class Support Vector Machine system. The output of the system is the facial expression recognized. The proposed real-time system, recognizes the 6 basic facial expressions with an approximately 98% accuracy.

A novel technique that can implement face detection directly in the wavelet compressed domain is presented in this paper. The algorithm takes the entropy decoding and inverse quantized wavelet transform coefficients of JPEG2000 picture as input, and outputs the locations of the detected faces. The main contribution of this work is in proposing a multi-level gradient energy representation of face pattern based on wavelet compressed data, which permits pertinent high contrast facial parts, such as eyes, nose and mouth, to be highlighted in a compact mode no matter the face is big or small. A neural-network based classifier is designed to decide a gradient energy pattern as face or non-face. In contrast to the traditional spatial-domain techniques, the proposed compressed domain technique eliminates the unnecessary decompression step and thus has lower computational complexity. Moreover, compared with the previous methods based on DCT compressed domain, the proposed multi-level gradient energy presentation removes the complex spatial scaling operation in compressed domain and overcomes block quantization problem. Based on test results of a variety of pictures, the presented algorithm was found to be more efficient and effective than the previous related methods.

The illumination changes on face images make face recognition a very difficult task. In this paper, a human face representation scheme that is insensitive to illumination variation is proposed in order to deal with the problem. The variations in lighting over human faces are modeled by means of Principal Component Analysis (PCA) on a number of blurred faces under different lighting conditions. Then the 'difference image', which is the difference between the original image and the reconstructed image, is used for face recognition. We also propose an uncorrelated Linear Discriminant Analysis technique for face recognition based on the eigen-illumination representation scheme. This method can obtain the uncorrelated optimal discriminant vectors (UODVs) so that the extracted features are uncorrelated. Experimental results show that the proposed method is effective to deal with varying illimunation problem for face recognition.

Based on recent works on Wyner-Ziv coding (or lossy source coding with decoder side information), we consider the case with noisy channel and addresses distributed joint source-channel coding, while targeting at the important application of scalable video transmission over wireless networks. In Wyner-Ziv coding, after quantization, Slepian-Wolf coding (SWC) is used to reduce the rate. SWC is traditionally realized by sending syndromes of a linear channel code. Since syndromes of the channel code can only compress but cannot protect, for transmission over noisy channels, additional error protection is needed. However, instead of using one channel code for SWC and one for error protection, our idea is to use a single channel code to achieve both compression and protection. We replace the traditional syndrome-based SWC scheme by the parity-based one, where only parity bits of the Slepian-Wolf channel code are sent. If the amount of transmitted parity bits increases above the Slepian-Wolf limit, the added redundancy is exploited to cope against the noise in the transmission channel. Using IRA codes for practical parity-based SWC, we design a novel layered Wyner-Ziv video coder which is robust to channel failures and thus very suitable for wireless communications. Our simulation results show great advantages of the proposed solution based on joint source-channel coding compared to the traditional approach where source and channel coding are performed separately.

The JPEG2000 image coding standard defines two kinds of region of interest (ROI) compression methods--the general scaling based method and the maximum shift (maxshift) method. There are two major drawbacks of the two methods. First, they would significantly reduce the compression efficiency by increasing the dynamic range (or number of bitplanes) of wavelet coefficients. Second, they do not have the especial protection for the ROI against the bit errors in image communication application, i.e., remote medical treatment and Satellite Communication. When the bit errors occurred, the reconstruction quality of ROI and BG would reduce at the same time. In this paper, a new and flexible ROI image compression method is proposed according to the Wyner-Ziv theorem on source coding with side information. The reconstructed low quality ROI is regard as a noisy version of original ROI in the proposed method. So, along with the parity bits from the original ROI, the reconstructed low quality ROI as side information can be utilized to Turbo decoder to decode the high quality ROI. Our experimental results show that the proposed method applying the Wyner-Ziv coding to ROI compression can highly improve the compression efficiency as well as efficiently protect the ROI against the bit errors.

In distributed video coding, the complexity of the video encoder is reduced at the cost of a more complex video decoder. Using the principles of Slepian andWolf, video compression is then carried out using channel coding principles, under the assumption that the video decoder can temporally predict side-information that is correlated with the source video frames. In recent work on distributed video coding the application of turbo codes has been studied. Turbo codes perform well in typical (tele-)communications settings. However, in distributed video coding the dependency channel between source and side-information is inherently non-stationary, for instance due to occluded regions in the video frames. In this paper, we study the modeling of the virtual dependency channel, as well as the consequences of incorrect model assumptions on the turbo decoding process. We observe a strong dependency of the performance of the distributed video decoder on the model of the dependency channel.

There are several problems in video coding that can not be adequately addressed through the use of conventional source coding techniques. For instance, any video coding system that is based on a predictive framework will suffer from performance degradations if there is uncertainty in the predictions. In this paper, we propose a method that incorporates binning codes and channel decoding to address video coding problems that have multiple possible predictions at the decoder. We will propose solutions based on binning codes to three separate problems in video coding and show that the results are superior to pre-existing methods.

Motion Compensated Temporal Filtering (MCTF) has proved to be an efficient coding tool in the design of open-loop scalable video codecs. In this paper we propose a MCTF video coding scheme based on lifting where the prediction step is implemented using PRISM (Power efficient, Robust, hIgh compression Syndrome-based Multimedia coding), a video coding framework built on distributed source coding principles. We study the effect of integrating the update step at the encoder or at the decoder side. We show that the latter approach allows to improve the quality of the side information exploited during decoding. We present the analytical results obtained by modeling the video signal along the motion trajectories as a first order auto-regressive process. We show that the update step at the decoder allows to half the contribution of the quantization noise. We also include experimental results with real video data that demonstrate the potential of this approach when the video sequences are coded at low bitrates.

Both digital radioscopy and radiographic testing rely on human experts to perform manual interpretation of images, and the recognition of welding defects demands the inspector's vast experience. With computer detect defects in a weld image; many countries have been made seeking the development of automatic system of inspection of welding defects. Defects were detected and segmented is key step for automatic recognition from welding image. This paper presents an algorithm of defects detection and segmentation for weld radiographic inspection which is based on wavelet analysis. The detection algorithm based on waveform analysis classifies weld defects as trough-anomaly, peak-anomaly and slanttrough- anomaly three types firstly. Aiming at to trough-anomaly and slant-trough-anomaly, weld defects inside them can be classified as crack and non-crack according to the width of two peaks in following. Different segmentation algorithms are adopted to these two types defects sequently. The experimental results show that the algorithm is very effective.

The text in a video frame can help us to understand the semantics of video content directly. Although there are many approaches that can automatically detect and localize text a video, most of them use the original pixels of an image to find the text regions. In this paper, we present an approach to automatically localize captions in MPEG compressed videos. Caption regions are segmented from background by using their distinguishing texture characteristics. Unlike previously published ones which fully decompress the video sequence before extracting the caption regions or only extract text regions in Intra-(I-) frames, our approach detect and localize caption regions directly in the DCT compressed domain. Therefore, only very small amounts of decoding processes are required. Experiments show that a good caption detection rate can be obtained, and the average recalls of Intra- and Inter-frame detections are 97.77% and 97.84%, respectively.

In this paper, a novel method is proposed to locate one-dimensional barcodes in JPEG 2000 images. The barcode locating system consists of two parts: candidate barcode location detection, barcode location verification and refinement. Both parts are designed to work in the compressed domain of JPEG 2000 images. The locations of candidate barcodes are extracted from the header data and verified by examining part of the decoded coefficients of the JPEG 2000 file. Since only a small amount of the compressed data is used, this algorithm has a low complexity relative to algorithms which use all of the pixel data.

In this paper, we introduce a real time vision-based human motion capturing system using two CCD cameras. For realtime processing consideration, we propose some simple but effective methods to estimate the motion parameters (BAPs) of the human object. Our method analyzes the vertical projection profile and the horizontal projection profile in each view to identify different arm and leg postures. With the identified postures, we can apply the Kalman filter to track the motion parameters (joint angels). Our motion analysis method is divided into macro motion analysis and micro motion analysis. The former identifies certain well-defined postures and the latter traces the variation of joint angle or BAPs.

Label location and recognition has become a crucial task for today's Unmanned Aerial Vehicles. We proposed a morphology-based algorithm to locate and recognize labels. This algorithm is insensitive to scaling and rotation, and able to work at low resolution. The label positioning and recognition strategy we designed is divided into two steps. First, at the altitude of 10m or so, we apply dilation processing and edge detection on the images sent back by UAV. Then combining the current heading information of the vehicle, we are able to give the topology map of all labels. After that the vehicle is lowered to about 5m and we apply erosion processing on the returned image and then recognize each label using image measurement and image analysis methods. The validity of this algorithm is well verified at ARCC 2004.

The automatic target detection under natural backgrounds is an important topic in the field of automatic target recognition. Fractal dimension is generally related to the roughness of the surface. The fractal dimension of the man-made object is usually lower than the background's because mostly it is smoother than natural background. This feature can be used to detect the target automatically. In the computing of the fractal dimension, the irregular values often appear at the boundary of the different textures in the image. This phenomenon can be called 'edge effect'. It may result in the difficulty in the followed image processing such as thresholding and cluster segmentation. The main reason of the edge effect is the same contribution of the every pixel in the neighborhood of the pixel where the fractal dimension being calculated. In this paper, in order to weaken the 'edge effect' in the fractal dimension computation, a 2-D Parzon window is designed. The accuracy of fractal dimension calculated after multiplied by the Parzon window is discussed, and a new algorithm is proposed to apply in the automatic target detection. The proposed automatic target detection algorithm is adopted in the experiments in images under complex land or sea backgrounds. The correctly detection rate is above 95%. The robust of this algorithm is represented in the cases of the variety of the light, rotation, size changing and occlusion of the target.

We investigate and compare combined compression-encryption schemes. We assess the respective security, and we show how encryption affects the image coding efficiency. The techniques employ the wavelet-based compression algorithms JPEG2000 and SPIHT, and we randomly permute and rotate blocks of wavelet coefficients in different wavelet-subbands to encrypt image data within the compression pipeline. We identify weak points of the propsed encryption techniques, and possible attacks are highlighted. The investigated methods allow to trade off security for compression performance. The results also have interesting implications with respect to the significance of the zerotree-hypothesis as stated in the context of compression schemes.

VLC codeword concatenations are not secured in most video encryption algorithms because direct encryption of them will usually not result in valid VLC codeword concatenations, and then the encrypted video stream does not conform to standard format. So how to secure VLC codewords has become a difficult issue in video encryption. This paper presents a light-weight approach to solve this problem. In the proposed scheme, the VLC codewords of TCOEF (Transform coefficient) and MV (motion vector) are randomly substituted with other legal codeword. The substitute codewords are from the standard VLC table, thus the encrypted video stream is format-compliant as an ordinary compressed stream that can be decoded by a conventional decoder. In addition, our scheme not only encrypts the sigh bits of TCOEF and MV, but also encrypts INTRADC codewords (DC coefficients for INTRA blocks) using secret key cryptography. This can also disguise the video content more and still conform to the standard format. Therefore, the security of this scheme is significantly enhanced while the complexity overhead is slightly increased. The experimental results show that video stream can be encrypted in real-time and the video content is fully disguised.

This paper presents a pay-video scheme that manages video stream, key stream and payment stream efficiently. In our scheme, the owner segments a video into fragments and encrypts them with independent keys. The keys are generated with a novel concept called as hash interval, where each hash interval discloses a range of numbers without disclosing any information on numbers outside of the range. The video fragments are then broadcast on one or more channels. A buyer can purchase the keys to decrypt any fragments and, within each fragment, any desired quality level. The accompanying payment protocol is integrated with the key management protocol seamlessly and hence the computation cost is very low.

Fingerprinting schemes use digital watermarks to determine originators of unauthorized/pirated copies. Multiple users may collude and collectively escape identification by creating an average or median of their individually watermarked copies. We present a codebook generation scheme to enhance the Anti-Collusion fingerprinting (AND-ACC) scheme to make it scalable for supporting large number of users. We evaluated our proposed codebook generation scheme using HVS based watermarking scheme and our own fingerprints. The simulation results with standard test images show good collusion detection performance over average and median collusion attacks.