In this paper, we survey the techniques for image-based rendering. Unlike traditional 3D computer graphics in which 3D geometry of the scene is known, image-based rendering techniques render novel views directly from input images. Previous image-based rendering techniques can be classified into three categories according to how much geometric information is used: rendering without geometry, rendering with implicit geometry (i.e., correspondence), and rendering with explicit geometry (either with approximate or accurate geometry). We discuss the characteristics of these categories and their representative methods. The continuum between images and geometry used in image-based rendering techniques suggests that image-based rendering with traditional 3D graphics can be united in a joint image and geometry space.

A compression scheme for calibrated images depicting a static scene from arbitrary viewpoints is presented. 3D scene geometry is reconstructed, and view-dependent texture maps are generated from all images. Texture is wavelet-coded using the SPIHT coding scheme extended to 4D, exploiting correlations within as well as between texture maps. During decoding, all texture maps are simultaneously and progressively reconstructed. The coder provides 3D scene geometry and multiple texture maps, enabling the use of graphics hardware to accelerate the rendering process. Three image sets acquired from real-world objects are used to evaluate the model-based coding scheme. Coding efficiency is shown for geometry approximations of different accuracy.

In this paper, we introduce an efficient image based rendering system for concentric mosaics, which can render stereo pair images of real scene in real time, with improved image qualities. We will mainly focus on related user interface issues and three interpolation algorithms: point sampling, linear interpolation with infinite or constant depth.

The concentric mosaics offer a quick solution to the construction and navigation of a virtual environment. To reduce the vast data amount of the concentric mosaics, a compression scheme based on 3D wavelet transform has been proposed in a previous paper. In this work, we investigate the efficient implementation of the renderer. It is preferable not to expand the compressed bitstream as a whole, so that the memory consumption of the renderer can be reduced. Instead, only the data necessary to render the current view are accessed and decoded. The progressive inverse wavelet synthesis (PIWS) algorithm is proposed to provide the random data access and to reduce the calculation for the data access requests to a minimum. A mixed cache is used in PIWS, where the entropy decoded wavelet coefficient, intermediate result of lifting and fully synthesized pixel are all stored at the same memory unit because of the in- place calculation property of the lifting implementation. PIWS operates with a finite state machine, where each memory unit is attached with a state to indicate what type of content is currently stored. The computational saving achieved by PIWS is demonstrated with extensive experiment results.

Concentric mosaics have the ability to quickly capture a complete 3D view of a realistic environment and to enable a user to wander freely in the environment. However, the data amount of the concentric mosaics is huge. In this paper, we propose an algorithm to compress the concentric mosaic image array through motion compensation and residue coding, which we called reference block codec. A two-level index table is embedded in the compressed bitstream for random access. During the rendering, the entire compressed concentric mosaic scene is not fully expanded at any time. Instead, only the contents necessary to render the current view are decoded in real time. We denote such rendering scheme as just-in-time rendering. Four decoder caches are implemented to speed up the rendering.

In this paper, we propose an adaptive entropy coding method to improve the VLC coding efficiency of H.26L TML-1 codec. First of all, we will show that the VLC coding presented in TML-1 does not satisfy the sibling property of entropy coding. Then, we will modify the coding method into the local statistics adaptive one to satisfy the property. The proposed method based on the local symbol statistics dynamically changes the mapping relationship between symbol and bit pattern in the VLC table according to sibling property. Note that the codewords in the VLC table of TML-1 codec is not changed. Since this changed mapping relationship also derived in the decoder side by using the decoded symbols, the proposed VLC coding method does not require any overhead information. The simulation results show that the proposed method gives about 30% and 37% reduction in average bit rate for MB type and CBP information, respectively.

This paper considers single video object rate control for MPEG-4 and presents a new rate control algorithm based on the quadratic rate-distortion model. The major innovation are a novel constraint for the least mean square estimation of the model parameters of the rate-distortion function, a new measure for the encoding complexity, and an efficient frame skipping strategy. An extension of the proposed rate control scheme to sequences with multiple video objects is possible. Experiments that were carried out on different video sequences showed an increase in the average PSNR of 1.2 dB and, more significantly, a reduction in the number of skipped frames of about 25% compared to the rate control in the MPEG-4 video verification model. The likelihood of buffer overflows was also greatly reduced for our proposed technique.

Most previous research efforts on video transcoding have been focused on changing the bit-rate of one pre-encoded bit-stream to another. However, in many applications (e.g. streaming video over a heterogeneous network), it may require to transcode a pre-encoded video bit-stream to multiple bit-streams with different bit-rates and features to support multiple clients with different requirements. In this paper, we discuss the case of point-to-multipoint transcoding. We compare two transcoder architectures in terms of processing speed for H.263 and MPEG-2 transcoding. We show that for point-to-multipoint transcoding, a cascaded video transcoder is more efficient since some parts of the transcoder can be shared by the multiple clients.

In this paper, we presented a novel and effective optimization method to achieve better trade-off between rate and distortion. The optimization is done at the encoder side alone and is transparent to the decoder. That is the quantized DCT coefficient set it optimized with full syntax compliance and decoder compatibility. The proposed method determines both the positions and the retained values of quantized DCT coefficients according to the rate-distortion performance measured by the associating Lagrangian cost. Fast dynamic programming technique was developed to release the computation burden. All the experiments show that the proposed method consistently outperforms other existing optimization methods.

Wavelet video coding using motion vectors estimated simultaneously at the transmitter and receiver side from the transmitted image data have been reported to have good compression capabilities, comparable to the non-scalable version of H.263. When scalability is required, the comparison turns even more in favor of the wavelet coding scheme. This paper shows that it is possible to reduce the bit-rate further in backward motion estimation schemes by using the certainty of each estimated motion vector. In this paper we report a lowering in bit rate of about 20% by using the motion vector certainty as background information in the entropy coding/decoding process. We also propose a low- complexity algorithm which does not require motion estimation/compensation, but uses the motion vector certainty.

A system for automatic detection of eye movements in medical applications is presented. It is designed to register exactly the dynamic motions of human eyes. In principle, the system runs independently of specific applications and may be used for different medical diagnoses. The method for motion analysis is applied to video image sequences of the retina recorded by a scanning laser ophthalmoscope (SLO). A modified block-matching algorithm has been developed for tracking the ocular fundus in the SLO sequences. The image processing strategies implemented provide high reliability of the motion detection which is used to process off-line. Several modifications have been developed in order to take into account the video quality of the SLO sequences. Additionally, some improvements have been implemented to increase the speed of the software. The system includes a phase of automatic motion detection followed by a subsequent phase of manual quality control. In this latter phase, an efficient tool for error detection and correction guarantees high quality and security of movement detection, which is of great importance in clinical applications.

This paper presents an efficient way of indexing and searching based on object-specific feature for video retrieval at different semantic levels. By tracking individual objects with segmented data, we generate motion trajectories with moving trails of objects and set a model using polynomial curve fitting. The trajectory model is used as an indexing key for accessing each object in the semantic level. The proposed searching system supports various types of queries including query-by-example, query- by-sketch and query on weighting parameters for event-based retrieval. When retrieving the interested video clips, the system returns the best-matched events in the similarity order. In addition, we implement a temporal event graph for direct accessing and browsing a specific event in the video sequence.

Early algorithms detecting camera shot transitions were based on the calculation of the difference between a single parameter of two consecutive frames. For many applications, only MPEG video is available; and in this situation, performing previous decompression in order to apply pixel- based algorithms may not be strictly required to achieve reasonable results: an alternative to the pixel-based approaches is to work with frame differences directly extracted from the MPEG compressed sequence. This paper intends to prove that shot detection on this domain can be as precise as that on the uncompressed one. After a discussion on the parameters of the MPEG stream best adapted to calculate an inter-frame difference, we present a distance-independent cut detection method, based on cut modeling and clustering, which obtains results similar to those obtained in the uncompressed domain.

In this paper we propose a thresholding technique for change detection in digital video. The technique assumes that the difference image generated from two frames of a video sequence has a trimodal Gaussian distribution and a computationally efficient fitting criterion is employed to find the best match between the data and model. Results show that the technique is capable of detecting true motion in very images.

In this paper, we propose an efficient shot boundary detection algorithm for direct processing of MPEG-2 video sequences. The proposed algorithm utilizes the hierarchical structure of the compressed bitstreams and characteristics of the coded parameters, for example, picture coding type and macroblock coding mode, thus greatly reducing computational requirement compared to pixel domain processing with full decompression. Occurrence of a shot boundary is checked first in a sub-GOP level, and if the result is affirmative it is checked again in each picture. And, to solve selection of appropriate threshold values, we use the hypothesis test with statistical characteristics of the coded parameters, for example, the ratio of intra-coded macroblocks to inter-coded macroblocks, the ratio of the backward macroblocks to bi-directional macroblocks, and the ratio of forward macroblocks to bi-direction macroblocks.

In this paper, we investigate the effects of errors on different parts in each slice of coded pictures of each type in MPEG-2 video sequences. We conclude, based on the simulation results, that the bits at the beginning of each slice are far more important than those near the end of the slice. Thus if the bits at the beginning of each slice are better protected while letting errors corrupt other bits in the slice, better video quality could be achieved while minimizing the overhead due to forward error control. We then propose a new unequal error protection (UEP) scheme, which gives better video quality than the existing schemes. In our scheme, each slice is first split into a number of segments. Then the segments at the beginning of each slice are better protected, while other segments in the slice are less protected. Our simulation results show remarkable (up to 4 dB improvement in peak signal to noise ratio) improvement in video quality, compared to the equal error protection and one existing UEP schemes, without bandwidth expansion. We conclude that the proposed UEP scheme is robust for MPEG-2 video transmission over frequency- selective Rayleigh fading channels.

This paper provides a system level description of a mixed video/data CDMA transmission model. We propose the combination of rate compatible punctured convolution (RCPC) code for the wireless physical (PHY) layer and CDMA RAKE receiver for wireless mixed video/data traffic which can be sent over indoor wireless channels. Through numerical analysis, using a frequency selective multipath Rayleigh fading channel model, it is shown that RCPC coding at the wireless PHY layer can be effective in providing reliable wireless CDMA network. And the proposed scheme that combine RCPC coder and CDMA RAKE receiver provides a significant gain in the BER and PER performance over multipath fading channels.

A mandatory requirement for future wireless multimedia communication is the availability of error resilient media codecs. This work discusses error detection and concealment techniques for a layered scalable video decoder, based on the SNR scalability option of the H.263 standard. A concealment method is proposed which uses error frequency and error location statistics for efficient hiding of transmission errors. The designed decoder allows error robust decoding with acceptable image quality even for highly corrupted video sequences up to bit error rates of 10^-3, optimizing the image quality specifically for two-layer coded sequences. Simulation results show that scalable video coding outperforms single-layer coding under typical wireless conditions even if no priority or increased protection is applied to the base layer. The proposed two- layer concealment method yields a consistent improvement of up to 5 dB in image quality.

In this paper we assess the independence of the optimization of source and channel coding parameters. We propose a method to separate the source and channel coding optimization as much as possible while maintaining the possibility of joint optimization. We theoretically derive key parameters that must be passed through an interface between source and channel coding. This separation greatly reduces the complexity of the optimization problem and enhances the flexibility.

An alternative method to H.263 for encoding of moving images at bit rates below 64 kbit/s is presented using adaptive spatial subsampling, mesh based interpolation and node tracking. This is why we call this new coding algorithm Mesh Based Interpolative Coding. Data compression is achieved by representing the image content by a number of non- equidistant sampling points (nodes) for luminance and color difference signals. The decoder reconstructs the image by interpolating the transmitted sampling points. For a given number of nodes which corresponds approximately to the total bit rate, the coder generates the node positions by minimizing the mean square error between the original picture and the interpolated picture. For moving images, each node is associated with a motion vector for node tracking. Simulation results of a complete encoder and decoder show that this method can provide lower bit rates than conventional schemes at a given picture quality for sequences with moderate movement.

In this paper, we propose a zerotree wavelet image/video coding technique with resilience to transmission errors which typically occur on noisy channels. A key tool that we employ is the bit partitioning algorithm and the bit reorganization algorithm which is called the EREC (Error Resilient Entropy Code). In order to take full advantage of the bit reorganization algorithm, the bit partitioning algorithm composes the data as separate code-blocks although the zerotree wavelet coding algorithm is not block-based compression technique. The bit reorganization algorithm requires a very low redundancy for the sequential transmission of variable length blocks that offer virtually guaranteed code and block synchronization. We present simulation results verifying the error resiliency of the proposed algorithm both for image coding which uses the wavelet transform and for video coding which uses the 3D wavelet transform. Experimental results show that the proposed coders outperform the existing error resilient coders for both noise-free channels and noisy channels. In addition, we confirm that the proposed algorithm is more error-resilient than the previously reported error-resilient coders for various channel error conditions.

With the recent expansion of multimedia applications, video coding systems are expected to become highly scalable, that is to allow partial decoding of the compressed bit-stream. Encoding techniques based on subband/wavelet decompositions offer a natural hierarchical representation for still pictures and their high efficiency in progressively encoding images yields a scalable representation. The multiscale representation can be extended to video data, by a 3D (or 2D+t) wavelet analysis, which includes the temporal dimension within the decomposition. Progressive encoding of video data represented by a 3D-subband decomposition was recently proposed as an extension of image coding techniques exploiting hierarchical dependencies between wavelet coefficients. In most of the previous image and video coding techniques, the compression is performed independently for luminance and chrominance coordinates. In this paper we propose a new coding technique for the chrominance coefficients, which not only delivers a bit-stream with a higher degree of embedding, but also takes advantage of the dependencies between luminance and chrominance components to provide an effective compression.

Quad-tree structured motion-compensation technique effectively utilizes the motion content of a frame as opposed to fixed size block motion compensation technique. In this paper, we propose a novel quad-tree-structured region-wise motion compensation technique that divides a frame into equivalent triangle blocks using the quad-tree structure. Arbitrary partition shapes are achieved by allowing 4-to-1, 3-to-1 and 2-1 merge/combine of sibling blocks having the same motion vector. We propose an optimal code scheme and a temporal predictive coding for the quad- tree. Simulation results show that our techniques reduce the bit rate by 40% as compared to other methods.

Multi-hypothesis prediction extends motion compensation with one prediction signal to the linear superposition of several motion-compensated prediction signals. These motion- compensated prediction signals are referenced by motion vectors and picture reference parameters. This paper proposes a state-of-the-art video codec based on the ITU-T Recommendation H.263 that incorporates multi-hypothesis motion-compensated prediction. In contrast to B-Frames, reference pictures are always previously decoded pictures. It is demonstrated that two hypotheses are efficient for practical video compression algorithms. In addition, it is shown that multi-hypothesis motion-compensated prediction and variable block size prediction can be combined to improve the overall coding gain. The encoder utilizes rate- constrained coder control including rate-constrained multi- hypothesis motion estimation. The advanced 4-hypothesis codec improves coding efficiency up to 1.8 dB when compared to the advanced prediction codec with ten reference frames for the set of investigated test sequences.

This paper presents a marker-free methodology for facilitating the analysis and tracking of human motion during gait. It consists of recognition and reconstruction of the legs of a walking human. In order to study the gait, a video system composed of three synchronized CCD cameras has been devised. It provides three different grey level image sequences of the same scene. First of all, a model of the human, based on tapered superquadric curves, has been defined; the leg can be divided into three parts: the thigh, the calf, and the foot. The whole methodology can be split following this scheme: determination of the boundaries of the human in motion by using successively an optical flow process and a crest lines extraction algorithm, prediction of the location of the human body in the 3D space, direct reconstruction of each part of the leg with a Least Median of Squares regression, and finally application of a spatial coherence process. The method described has been tested on synthetic images (mean error of about 1.2 mm and a maximal error of about 5 mm along the coordinate axes) and on image sequences of a walking human.

MIME (Mime Is Manual Expression) is a computationally efficient computer video system for recognizing hand gestures. The system is intended to replace the mouse interface on a standard personal computer to control application software in a more intuitive manner. The system is implemented in C code with no hardware-acceleration and tracks hand motion at 30 fps on a standard PC. Using a simple 2D model of the human hand, MIME employs a highly- efficient, single-pass algorithm to segment the hand and extract its model parameters from each frame in the video input. The hand is tracked from one frame to the next using a constant-acceleration Kalman filter. Tracking and feature extraction is remarkably fast and robust even when the hand is placed above difficult backdrops such as a typical cluttered desktop environment. Because of the efficient coding of the gesture tracking software, adequate CPU power remains to run standard application software such as web browsers and presentation software.

In recent years considerable progress has been made in the area of face recognition. Through the development of techniques like eigenfaces, computers can now compte favorably with humans in many face recognition tasks, particularly those in which large databases of faces must be searched. Whilst these methods perform extremely well under constrained conditions, the problem of face recognition under gross variations in expression, view, and lighting remains largely unsolved. This paper details the design of a real-time face recognition system aimed at operating in less constrained environments. The system is capable of single scale recognition with an accuracy of 94% at 2 frames-per- second. A description of the system's performance and the issues and problems faced during it's development is given.

Temporal profiles of point-like dim targets and the extended cloud pixels provide useful information in detecting the targets. Among the recent methods that utilize temporal profile of pixel in detecting point target are Triple Temporal Filter (TTF) and Continuous Wavelet Transform (CWT). TTF uses two damped sinusoidal filters, an exponential averaging filter with six appropriate coefficients to deal with different aspects of clutter. TTF is recursive and efficient in detecting point targets without applying any threshold techniques. The performance of CWT is comparable to TTF but all the frames in a sequence need to be stored. Therefore it is computationally complex algorithm.

An adaptive algorithm to adjust the Intra update rate of a video encoder is presented. As optimization criterion, the PSNR at the decoder after lossy transmission is used. Based on a model for the video encoder and error propagation at the video decoder the optimal Intra rate can be calculated for a given transmission channel analytically. In this paper it is shown how the parameters of these models can be measured adaptively during encoding. Thus, the Intra rate can be adjusted to changing channel conditions and sequence statistics. It is shown that a practical robust system can be built with the presented algorithm.

Delivering temporally-constrained multimedia streams in heterogeneous environments, offering no guarantee in terms of bandwidth, packet loss, or delay, is a very challenging problem faced today by both the networking and the coding community. Layered coding is often proposed as a solution for rate-based congestion control of video transmission in heterogeneous environments. The problem addressed more specifically here is the design of a responsive mechanism for rate allocation in each layer, that would guarantee the best bandwidth usage for all the receivers. After a review of solutions for congestion and loss control in unicast video communications, the paper describes a rate-based congestion control mechanism for multicast layered video transmission.

IN this paper, we report implementation of encryption of the signal in networked environment using synchronized chaos. The science of chaos has successfully been explored and implemented in hardware for communication signals by Cuomo et al in 1993, although he claimed that the mathematics behind the phenomenon was still obscure and not very clear. We are utilizing this unpredictable, random phenomenon called chaos to code our information by superimposing it on a chaotic signal. We applied this technique to different formats of data and results were 100% in the sense that decoded files were run without error. This testing scenario encourages use of chaos in information coding across networks. It is shown that, in this environment, the security breach may not be possible even if sender and receiver have compatible scramblers and descramblers, and the key is also known to the receiver.

A camera used in video communication over Internet is usually placed on top of a monitor, therefore it is hard for an user to make a natural eye contact with the peer communicator since the user gazes at the monitor, not the camera lens. In this paper, we propose a single 3D mesh warping technique for gaze-correction. It performs 3D rotation of face image by a certain correction angle to obtain a gaze-corrected image. The correction angle is estimated in an unsupervised way by using invariant face feature, and a very simple face section model is used in 3D rotation instead of precise, but not easily attainable in most cases, 3D face models. The method is computationally simple enough to implement for real-time casual video communication applications.

In the defence organization, imagery represents an important information source for users in the tactical, operational and strategic environments. Its wider dissemination may rely on deployed communication networks that are often unstable with high bit error rates and outages. This paper presents efficient techniques for imagery dissemination using the user datagram protocol (UDP). The use of UDP is compared with the popular transmission control protocol and shown to be superior in performance for error prone IP networks. We have employed a wavelet based coder producing an embedded bit-stream. The packetization of the bit-stream is investigated and we show that it is better to tile an image into independent embedded bit-streams when the network performance is poor. Variable-size tiling is compared with, and shown to be superior to, a fixed-size tiling approach. Selective re-transmission of lost packets is implemented for efficient imagery dissemination using UDP. The selective re- transmission scheme is a function of network bandwidth, delay times, error rates and the significance of the packet.

The latest digital image coding technique known as the object-based coding is mainly relied on efficient segmentation algorithms. In this paper an edge detail detection technique has been proposed and it can segment the image into objects that are perceptually meaningful. It is a transform domain technique and it makes use of perceptual characteristics, such as frequency distribution decomposition and edge structure decomposition, of transform coefficients thus it detects the perceptually most significant edge details which are useful for segmentation in object-based coding and compression. An experiment has also been conducted to test its robustness to image compression.

In this paper, we present an object-based coding scheme using 3D shape-adaptive discrete wavelet transforms (SA- DWT). Rather than straightforward extension of 2D SA-DWT, a novel way to handle the temporal wavelet transform using a motion model is proposed to achieve higher coding efficiency. Corresponding to this transform scheme, we use a 3D entropy coding algorithm called Motion-based Embedded Subband Coding with Optimized Truncation (ESCOT) to code the wavelet coefficients. Results show that ESCOT can achieve comparable coding performance with the state-of-the-art MPEG-4 verification model 13.0 while having the scalability and flexibility of the bitstream in low bit-rate object- based video coding. And in relative higher bit-rate, our coding approach outperforms MPEG-4 VM 13.0 by about 2.5 dB.

In this paper, the performance of the extrapolation block transform (EBT) and the shape-adaptive DCT (SA-DCT) for arbitrarily shaped image segment coding is theoretically compared. The comparison indicates that the EBT approach yields better performance than the SA-DCT on the highly correlated image at low bit rates. Since the correlation maximizing extrapolation (CME) algorithm proposed in maximizes the correlation of the extrapolated block, it is considered to be one of the EBT techniques that can achieve the performance expected by the theoretical performance. Thus, a novel object-oriented video coder employing the CME algorithm for the texture coding is proposed in this paper, and its performance is compared with that of the MPEG-4 VM through intensive computer simulations. In the proposed video coder, the contour coding technique employing the two- stage motion compensation is adopted for the shape coding, and the rate-constrained hierarchical grid interpolation is adopted for the object-based motion compensation. The simulation results show that the proposed coder outperforms the MPEG-4 VM by about 0.84 approximately 1.29 dB depending on the test sequences at the same bit rate.

This paper presents an embedded zerotree wavelet coding technique for the compression of arbitrarily shaped 2D objects. The wavelet decomposition is carried out with an optimized, biorthogonal, shape adaptive discrete wavelet transform (SA DWT) which performs non-expansive multiresolution decompositions of arbitrary image regions. The proposed SA DWT is defined for even-length, symmetric wavelet filters such as the 6-10 filters. The processing at region boundaries is carried out via reflection, followed by an optimization stage which requires only a few operations per boundary pixel. The computationally inexpensive optimization results in an additional performance gain of up to 0.5 dB compared to the plain reflection based scheme.

An object-based analysis-synthesis coder for coding moving images at low data rates is investigated. The coder is based on the source model of articulated 3D objects. This model describes the real objects by means of model objects define by shape, motion and color parameters. The model objects may be articulated, i.e. they consist of several rigid object components linked to each other by joints. In this contribution a new algorithm for joint position estimation is presented. First the motion parameters of the connected object components are estimated at different times. For motion estimation a Maximum-Likelihood estimator is applied. Then the position of the joints is determined by evaluating the estimated motion parameters in the equations representing the constraints imposed for the joints on the relative motion of the connected object components. The algorithm was applied to a synthetic test sequence (CIF, 10 Hz). At a camera noise figure of 40 dB a joint position estimation average error of 0.94 pel can be achieved.

In this paper, we present a new stereo matching method with deformable windows to measure depth information more precisely. In order to correct the projective distortion between the left and right images, a window on the left image was deformed more than one window called deformable windows with a linear interpolation, and each deformable window keeps same texture pattern with different size. Stereo matching are performed simply by computing the normalized sum of squared difference (SSD) between the left and right images with deformable windows. The correspondence is determined by evaluating the matching score with a minimum of normalized SSD. The algorithm has been tested with real stereo image, and applied to measure the 3D-shape of human face. The results demonstrate the effectiveness of the stereo matching method with deformable windows.

To consider the quality of service for stereoscopic image through the network, it is necessary to develop a quality evaluation method for coded stereoscopic image. We propose a quality evaluation model of the coded stereoscopic color image. This evaluation model considers not only the distortions of the edge region and smooth region but also the texture features of the left image. In addition, this model takes into account the disparity information between the left and right images. Instead of the disparity compensated coded image, we employ the JPEG coded image for the subjective assessment test. As the results, the evaluation model is useful for coded stereoscopic image.

This paper focuses on the correspondence field estimation and utilizes the estimation performance to compress stereoscopic images. This paper proposes the dense correspondence estimation with new probabilistic diffusion algorithm based on maximum a posteriori (MAP) estimation. The MAP-based correspondence field estimation including occlusion and line field is derived with reflecting the probabilistic distribution of the neighborhoods, and is applied to the compression of stereoscopic images. The proposed probabilistic diffusion algorithm considers the neighborhoods in Markov random field with their joint probability density, which is the main difference from the previous MAP-based algorithms. The joint probability density of neighborhood system is implemented by using the probabilistic plane configuration model. And, the paper derives the upper and lower bounds of the probabilistic diffusion to analyze the applied to quadtree-decomposed blocks.

This paper tackles the 3D model-based representation of video sequences of real scenes and focuses on sequences derived from a camera moving in static scenes. 3D models can be used advantageously for compression purposes. Moreover, they are quite adapted to interactive viewpoint synthesis. The object is here to build a compact 3D-based representation of a given video sequence. It first requires establishing correspondence between images from which viewpoint parameters and depth maps are estimated. The representation is then built by selecting in the image sequence the data that are necessary and sufficient to reconstruct the sequence at a given quality level. This paper presents this second part of the system. Our representation is a structured view-dependent 3D model composed of an ordered set of rectangular patches describing 2.5D regions (flag+texture+depth for each pixel) with attached viewpoint parameters sets.

A method for high quality stereo panorama mosaicing is presented. The surrounding scene is captured by our original 3-camera system as stereo moving picture sequences, and the images are stitched after improvement of the texture quality. The multi-purpose 3-camera system features accurate frame synchronization between 3 channels, and can be used outdoors through battery operation. For registration of panorama stitching, under the newly investigated distortion- free condition. Affine parameters are estimated through the overlapped areas by the steepest descent algorithm. The texture improvement has two steps, one is vertical resolution recovery by field integration, and the other is image enhancement by 2D quadratic Volterra filter which satisfies Weber-Fechner's law. The presented method enables high quality stereo mosaicing with accurate mutual disparities between the channel and without visible distortion of textures.

In this paper, we present a multiple hierarchical protection scheme applied in for Rayleigh fading channels using multilevel coding (MLC) system in which both capacity rule and block partitioning have been taken into account. As a result, the non-uniform set-partitioning design trades the performance of the less important data for better performance of the most important ones, and thus improves the multiple hierarchical protection ability of MLC schemes.

Digital video broadcasts can be redistributed over a local area network using the packet based HIPERLAN (High Performance Radio Local Area Network) protocol. An example of the application of this technology is wireless communication within the home environment. This paper addresses the problem of transmitting MPEG-2 video over HIPERLAN and describes error resilient video transcoding as a means of handling channel errors. The implementation of two transcoding techniques on both PC and DSP platforms are discussed.

The MC-DCT based video coding has been generally used for its efficiency and easy implementation, but it is very vulnerable in error-prone environments. In this paper, we present an error-resilient video coding using multiple reference frame, based on the long-term memory motion compensated prediction (LTMP), and an error concealment technique associated with the proposed technique. The rate- distortion optimization of the LTMP is extended in order to yield the improved error-resilience and error concealment capabilities. Also, the proposed algorithm confines effectively the temporal error propagation using the negative acknowledgement in feedback channel, in which the corrupted area by channel errors as well as the propagated is estimated and removed from the search region for the motion compensation. Thus, the proposed algorithm yields the similar performance to the forced intra update (FIU) method in terms of the PSNR, but avoids the abrupt increase in the bitrate, resulting in more efficient network utilization than the FIU. By the computer simulations, we shall demonstrate that the proposed technique provides an acceptable performance both subjectively and objectively in error-prone environments, as compared with the H.263 and LTMP, with or without the feedback messages.

Advertising in TV broadcasting has shown that multimedia is a very effective means to present merchandise and attract shoppers. This has been applied to the Web by including animated multimedia banner ads on web pages. However, the issues of coupling interactive browsing, shopping, and secure transactions e.g. from inside a multimedia banner, have only recently started to being explored. Currently there is an explosively growing amount of back-end services available (e.g., business to business commerce (B2B), business to consumer (B2C) commerce, and infomercial services) in the Internet. These services are mostly accessible through static HTML web pages at a few specific web portals. In this paper, we will investigate the feasibility of using interactive multimedia banners as pervasive access point for the B2C, B2B, and infomercial services. We present a system architecture that involves a layer of middleware agents functioning as the bridge between the interactive multimedia banners and back-end services.

This paper presents a new scheme for efficient MPEG4 video transmission over the Internet that provides differentiated services (diff-serv). The scheme has three new elements: (1) smart packetization scheme which distinguishes different types of MPEG4 data streams; (2) object-based bit rate control and adaptation corresponding to users' interactions and selections; (3) selective dropping of packets that takes advantage of differentiated classes in diff-serv. Experiment was conducted on the Microsoft's IPv6 platform where an adaptive MPEG4 video streaming system was developed to verify our proposed approach. Experimental results show that our proposed scheme provides must improved QoS and capability for object-based user interactivity.

This paper presents a novel algorithm to accelerate the encoding procedure of fractal image compression. We develop an indexing technology to access candidate domain blocks. The location of maximal gradient is adopted as the key for indexing. Only those blocks whose positions of maximal gradients matching that of a given range block are rested. In our experiments, the new algorithm promises good performance. It takes few seconds to encode a 512 by 512 image on a Pentium II 450 PC with a slight loss of decoded image fidelity.

This paper presents an improved fractal block coding scheme for still images. The proposed scheme employs a new technique which we call `sub-block luminance level shifting.' In fractal block coding, an input image is first partitioned into range blocks. Each range block is encoded by a set of contractive affine transformations of its corresponding domain block. One of the coded data for each range block is an average pixel value of the range block, which is used for luminance level shifting between the range block and the contracted domain block. In our proposed method, a range block is further partitioned into sub-blocks in some cases and an average value of each sub-block instead of the range block is used for luminance level shifting. We have proposed an improved fractal block coding scheme applying this sub-block luminance level shifting adaptively block-by-block basis and also combining this method with adaptive range block size fractal coding. The computer simulation results show that the proposed fractal coding scheme gives higher SNR (Signal-to- Noise Ratio) values and better image qualities compared to the conventional fractal block coding scheme.

Low bit rate image coding is essential for many visual communication applications. However, it yields visually annoying artifacts that highly degrade the perceptual quality of image data. In this paper we proposed a novel method for reducing the blocking artifact that occurs when using DCT image coding at low bit rates. The blocking artifact has varied visibility in different region, that it, it is more visible in a smooth region than detailed region such as textures and edges. Hence, the proposed method is based on image segmentation and then thresholding the coefficients of different regions with adaptive thresholds. The proposed method gives consistent improvement over the previous methods for deblocking in term of peak signal-to- noise ratio, edge variance, and visual quality.

Adaptive interpolation with double-interpolator is used for image coding. The double-interpolator means that two adaptive prediction stages are used. The outer loop is an adaptive FIR predictor. The inner loop is an texture based bias estimation. The bias means a content dependent estimated prediction error. By assuming the signals to consist of polynomials of various degrees, the predictor of the outer loop is constructed by linearly combining a set of maximally flat filters. The maximally flat filter is the filtering implementation of the Lagrange interpolation. The prediction is done block by block. Within a block, similar with the lifting scheme, a hierarchical multiresolution prediction is used starting from the lowest resolution 2 by 2 sub-block. The least square prediction error criterion is used to derive the weighting coefficients of the predictor. To further reduce the prediction error, an inner loop to estimate the prediction error is included. Within the inner loop, the pixel to be predicted is classified into groups according to the neighborhood condition. The accumulated mean of the prediction errors of all the pixels within the same group is considered as the bias of the prediction error. This bias is then extracted from the actual prediction error to reduce entropy.

In recent years, there has been much interest in object- oriented video coding schemes as an alternative to conventional video coding schemes, particularly for very low bit rate applications such as the coding of head and shoulder sequences for video conferencing. Automatic face detection is desirable in object-oriented video coding schemes that seek to code head and shoulder video sequences. However automatic face detection is not a trivial problem when complex backgrounds are present. We propose a novel method of adequately characterizing skin color by exploiting the luminance and chrominance information found in the first frame of a sequence. The skin color characterization is then used to facilitate skin segmentation in subsequent frames, and thereby automatic face detection. To aid the adaptive skin characterization process, the head is constrained to a frontal, upright position in the first frame of the sequence. However no constraints are placed on the position of the head in the subsequent frames. The proposed scheme was tested on a number of common H.263 and MPEG-4 head and shoulder video sequences. Some experimental results are presented.

Human face images form the important database in banks, security kiosks, police departments and they are also found in abundance in day-to-day life. In these databases the important content, of course is the face region. In this paper, we present two highly efficient methods for compression face images. The first method, which is lossy, detects the human faces in the wavelet domain for discriminative quantization to achieve high perceptual quality content-based image compression. This method gives superior subjective performance over JPEG standard without sacrificing the performance in the rate-distortion spectrum. The second method, which is lossy as well as lossless, also detects the human faces in the wavelet domain so that lossy/lossless mode can be selected dynamically for compression. This method gives the best solution to applications which can not tolerate losses for face regions. The method improves the overall coding efficiency by adapting lossy mode for non-face regions instead of coding entire image in lossless mode.

In this paper, we address a few important image analysis problems, which are fundamental to the design of Perceptual User Interface. We use an inexpensive stationary desktop camera to collect the video streams and use them as input to the system. We present an algorithm for segmenting moving foreground object of interest from a complex, but stationary background. This algorithm can cope with illumination changes due to shadows, Automatic Exposure Correction and long term illumination changes in the environment. The segmentation is done real time and works well for both indoor and outdoor scenes. The detected foreground is recognized as human being based on head shoulder profile.

In this paper a new algorithm for joint detection and segmentation of human faces in color images sequence is presented. A skin probability image is generated using a model for skin color. Instead of a binary segmentation to detect skin regions, connected operators are used to analyze the skin probability image at different threshold levels. A hierarchical scheme of operators using shape and texture simplifies the skin probability image. For the remaining connected components, the likelihood of being a face is estimated using principal components analysis. To track a detected face region through the sequence, the connected component that represent the face in the previous frame is projected into the current frame. Using the projected segment as a marker, connected operators extract the actual face region from the skin probability image.

Because of the complexity, variability and cost of subjective video quality testing there is a strong requirement for an instrument or system which will make objective quality estimations as close as possible to the subjective ones. This paper includes a description of such Video Quality Analyzing System. It works off-line, on Sun's Ultra-10 platform, and we think that its distinctions from other well-known systems such as Tektronix PQA200, Rhode & Schwarz DVQ etc. are significant--functionality and algorithmically. This is a result of intensive three-years- experience, and we hope that will be clear that the work `analyzing' is not accidental here: there is the main distinction.

In this paper, a perceptual blocking distortion metric for block-based transform coded digital video is proposed. The metric is based on a spatio-temporal multichannel vision model to calculate the Just Noticeable Distortion (JND) map. The blocking dominant regions are segmented after the spatio-temporal decomposition and the JNDs in these regions are summed up to form an objective measure of the blocking artifact. Subjective and objective tests have been conducted and the results show a strong correlation between the objective blocking rating and the mean opinion score.

In today's television competitive environment, quality of service is critical. For network providers, the delivered quality of service is usually a matter of technical signal quality. However, with the current development of digital broadcasting, the quality of service that is provided to the end-user is one of the most important performance criterion. Thus, perceived image and sound quality are important and must be monitored. This paper addresses the issue of video quality monitoring in digital television networks. Many approaches to objective video quality assessment have been proposed and are being evaluated by normalization bodies. In order to build and apply in-service measurement, we propose a global model describing the whole approach and the different technical solution. Our technical solution is a double-ended with reduced reference method. Video quality assessment of digital television signals in a broadcasting network sets strong technical constraints. The reduced reference, a comparative approach, is well adapted since it has a moderate technical complexity and because it informs on any distortion in the network. The proposed method has been implemented and tested in a range of situations on simulated and real DVB networks. Three main applications are presented, network monitoring, DVB-T coverage area determination, and laboratory tests. This allows concluding that in-service measurements are the only valid measurements for end-to-end transmissions.

In this paper we discuss work on quantification of video impairments resulting from MPEG compression, their role, and their scope of application for objective quality assessment. Three important metrics, blocking artifacts level, ringing artifact level, and corner outlier artifact level have been used to create a combined impairment metric. The relevance of this metric to develop an objective quality assessment has been investigated, as well as the issues facing the creation of a no-reference quality metric. The main issues are overall metric completeness, and performance of the individual metric components. The impairment metrics that we have studied appear to be key components for future no- reference type of objective quality metrics. Impairment metrics are also of great importance because they allow closing the detect-measure-correct loop that is necessary to improve image quality in real time. Applications of single- ended quality metrics include multimedia home terminals, STBs, digital TV, and low bit-rate video applications such as IP videotelephony and video streaming over IP.

We propose a PQS_video (Picture Quality Scale for moving image), a method of objective quality assessment for coded moving images. We expect that the proposed PQS_video approximates subjective assessment well. In PQS_video, we define essential distortion factors considering not only global distortions (such as random noise) but also distortions on local features. Then, we describe each distortion metrically considering human visual perception. The PQS_video is given by a linear combination of define each essential distortion factor, utilizing the principal component analysis method and the multiple regression analysis method between quantity of each essential distortion factor and MOS (Mean Opinion Score) obtained by assessment test. We have confirmed that the PQS_video approximates MOS successfully.

The objective of this work is to reconstruct high quality gray-level images from bi-level halftone images. We develop optimal inverse halftoning methods for several commonly used halftone techniques, which include dispersed-dot ordered dither, clustered-dot ordered dither, and error diffusion. At first, the least-mean-square (LMS) adaptive filtering algorithm is applied in the training of inverse halftone filters and the optimal mask shapes are computed for various halftone techniques. In the next step, we further reduce the computational complexity by using lookup tables designed by the minimum mean square error (MMSE) method. The optimal masks obtained from the LMS method are used as the default filter masks. Finally, we propose the enhanced MMSE inverse halftone algorithm. It normally uses the MMSE table lookup method for its fast speed. When an empty cell is referred, the LMS method is used to reconstruct the gray-level value. Consequently, the proposed method has the advantages of both excellent reconstructed quality and fast speed. In the experiments, the error diffusion yields the best reconstruction quality among all three halftone techniques.

In this paper, we new sub-region search algorithm is proposed for fast encoding of image vector quantization. In the proposed scheme, we will only search a portion of the codebook to locate the best match vector of each input vector according to the state of its neighboring blocks. A block transition criterion is developed for selecting the subset of codebook for searching. Simulation results show that the encoding time required by the proposed algorithm is just about 10 - 20% of that required by the full search with almost the same output image quality.

Block Truncation Coding is one of the oldest known forms of image compression algorithms, its main attraction being its simple underlying concepts and ease of implementation. In this paper we present a new Predictive Absolute Moment Block Truncation Coding scheme which improves the performance of the existing Block Truncation Coding schemes. The proposed scheme is based on selectively predicting the reconstruction values of a block based on the corresponding values in the neighboring blocks, as well as predicting the bitplane from the bitplanes of the corresponding blocks in other color components.

This study is to improve double predictor differential pulse code modulation (DP-DPCM) algorithm for image data compression. A variable block-size double predictor DPCM (VBDP-DPCM) image coding system operates on an image that has been preprocessed into segments of variable size, square blocks, and each block is separately encoded by a DP-DPCM system. Quadtree segmentation algorithm is utilized to divide a given real-world image into variable size image blocks. The detail regions of a given image is segmented into blocks with smaller block size, and the background regions of the image will be assigned larger block size to the image blocks. After quadtree segmentation procedure, the differential values between the nearby pixels within an image block are reduced. Therefore, we can decrease the distribution range of the prediction error as well as reduce the quantization levels and the bit rate. We then adopt the double predictor DPCM image coding system to reduce the effect from the fed-back quantization error and not to increase the system complexity. The source coding performance of this proposed variable block-size DP-DPCM image encoder/decoder system is about 5 dB (or greater) coding gain in Signal-to-Noise Ratio than that of a conventional DPCM system.

In this paper, we propose a multi-mode linear prediction (MM_LP) scheme for the compression of multi-spectral satellite images. This scheme, extending our prior work on block-based single mode linear prediction, discards the prediction residuals and transforms the traditional residual-encoding problem into another mode-map encoding problem. The increase in the extra storage for more coefficients is nearly negligible and the compression of mode-map might be expected to have a higher efficiency than the residuals can achieve. We also propose an alternative scheme to hide the mode information in the LSB (least significant bit) of the residual data, which are then encoded to give a nearly lossless compression with PSNR larger than 51 dB (error variance (sigma) ² equals 0.5/per pixel). Comprehensive experiments justify performance of our MM_LP schemes and recommend that MM_LP (k >= 2) is suitable for PSNR less than 41.5 dB; single-mode LP (k equals 1) is for PSNR between 41.5 dB and 50 dB, while 2-mode mode- embedding approach is for PSNR > 50 dB.

In this paper we address the user-navigation through large volumes of image data. A similarity-measure based on MPEG-7 color histograms is introduced and Multidimensional Scaling concepts are employed to display images in two dimensions according to their mutual similarities. With such a view the user can easily see relations and color similarity between images and understand the structure of the data base. In order to cope with large volumes of images a modified version of k-means clustering technique is introduced which identifies representative image samples for each cluster. Representative images (up to 100) are then displayed in two dimensions using MDS structuring. The modified clustering technique proposed produces a hierarchical structure of clusters--similar to street maps with various resolutions of details. The user can zoom into various cluster levels to obtain more or less details if required. The results obtained verify the attractiveness of the approach for navigation and retrieval applications.

This paper describes an approach to image segmentation that is based on an integrate-and-fire operation. An analog cell is described that will fire current pulses when the input light intensity to the cell exceeds a given threshold. The firing of one cell can induce the firing of neighboring cells, so contiguous object regions are formed using both individual and neighborhood information. The proposed cell is relatively simple in structure and is capable of incorporating a range of advanced autonomous functions by adapting threshold levels to various local and global conditions.

The block-based image segmentation method is known to alleviate the over-segmentation problem of the morphological segmentation methods. In this paper, we improve the previous block-based MAP segmentations. First, to reduce the execution time, we try to reduce the number of undecided blocks. That is, as the block size is reduced, we define new monotone regions with the undecided blocks to decrease the number of undecided blocks and to overcome the undersegmentation problem. Second, to improve the segmentation accuracy, we adopt two different block sizes. For texture block clustering process, we use a large block- size. On the contrary, for monotone and edge block classification, it is more efficient to use a small block- size. The proposed segmentation method is applied to natural images with monotone and texture regions. Experimental results show that the proposed method yields large segments for texture regions while it can also pick up some detail monotone regions to overcome the under-segmentation problem.

For the past decade, the region-based approach, that combines object segmentation and optical flow estimation, has emerged as the only one likely to provide automatically, at a reasonable computational cost, higher-quality descriptions of 2D apparent motion in video sequences, as compared to conventional pixel-based motion estimation. Within this framework, a hybrid algorithm, embedding classical defense motion field estimation and color-based spatial segmentation, is presented. Per each, arbitrarily shaped, color-homogeneous region, a polynomial motion- parameter set is robustly estimated from pixel displacement vectors. Following a graph-based approach and starting from the initial color partition, neighboring regions are iteratively merged according to their mutual motion similarity. The obtained motion-homogeneous regions are eventually temporally tracked along the sequence. The region-based motion estimation algorithm is described in details and its computational complexity is loosely evaluated through processing time statistics on a workstation. The partition maps and modeled motion fields obtained on three well-known test sequences--`Table Tennis', `Mobile and calendar' and `Flower Garden'--are displayed. Alternative approaches in the literature are then assessed, their results being compared with the above ones. Application of such an automatic `mid-level' image analysis tool to object-based representation, manipulation and coding as well as indexing of video is outlined at last.

This paper presents the preliminary results of our investigation on using singular value decomposition transform to extract effective image features. It proposes using the singular values of subimages to discriminate different classes of imagery. The proposed new features have been very effective when applied to target detection and background discrimination in low resolution SAR imagery.

Shunting inhibition is a model of early visual processing which can provide contrast and edge enhancement, and dynamic range compression. An architecture of digital Shunting Inhibitory Cellular Neural Network for real time image processing is presented. The proposed architecture is intended to be used in a complete vision system for edge detection and image enhancement. The present hardware architecture, is modeled and simulated in VHDL. Simulation results show the functional validity of the proposed architecture.

Many fast search block-matching motion estimation (BMME) algorithms have been developed in order to minimize the search positions and speed up the computation but they do not consider how they can be effectively implemented by hardware. In this paper, we propose a new regular fast search block-matching motion estimation algorithm named Two Step Search (2SS). The 2SS BMME will then be implemented by 8 Xilinx XC6216 fine-grain, sea-of-gate FPGA chips. The experimental and simulation results shows that it can have better algorithmic performance and can be implemented by FPGA chips very cost-effectively for video compression applications. Also, the 30 frames per second real time 2SS BMME video compression can be obtained by using eight Xilinx XC6216 FPGAs.

To increase memory bandwidth of Synchronous DRAM (SDRAM) that is commonly employed as external memory in video applications, a memory address translation method is proposed for minimizing the number of overhead cycles needed for row-activations and precharges. The features of SDRAM and memory access patterns of video processing applications are considered to find a suitable address translation. Experimental results show that the proposed method increases the memory bandwidth by 42% over that of the conventional linear translation.

In this paper, we present a novel video signal processor (VSP) architecture, named VS-VSP, which combines the very long instruction word (VLIW) control with the single instruction multiple data (SIMD) processing technology. The SIMD architecture provides high performance for the computation-intensive tasks, and the VLIW control introduces satisfactory flexibility to the whole system. In addition, a hierarchical memory organization is employed in VS-VSP to tackle the high data-bandwidth requirements in the video signal processing application. The proposed VS-VSP architecture can be exploited for implementing a variety of video coding algorithms efficiently.

To retrieve efficiently a specific image in their voluminous image database, users need of appropriate tools. That is the reason why, over the last years, content-based image retrieval systems have been developed. In these systems, users formulate their queries from both visual and textual descriptions. Consequently, these features must be described in a well-suited representation in order to satisfy the efficiency and relevant retrieval criterion. In the sequel, we will only dwell on one of the most important visual features, namely the shape feature. This feature is essential as it corresponds to region of interest in images. As far as the shape is concerned, an interesting representation can not be reached without efficient synergy between image analysis techniques, mathematics, and database technology. In this paper, the development of our theoretical model describing shape representation for content-based retrieval is presented as well as the numerical results computed for different simulations. These results are used to show the usefulness of the considered Fourier descriptors (first order and second order interpolation). Under MATLAB, a powerful mathematical software, we analyzed and easily validated the proposed method. To conclude, the relation between Moment theory (another interesting shape representation) and Fourier analysis is discussed.

Although global color histogram and moments have been proven to be very useful for image indexing, they do not take color-based spatial information into account. Thus when the image collection becomes very large, many false hits frequently occur. The latest work, so called spatial color indexing, attempts to characterize finer details of the color distribution. However the conventional methods have to undergo multi-step processes of partitioning and indexing the image blocks. This paper proposes an efficient image description method for image retrieval using circular scanning pattern.

In recent years 3D wavelet video coding has become a popular research area due to its low coding complexity. As the emerging MPEG-4 video compression standard places a strong emphasis on content base manipulation and compression, there is a need to implement the content coding feature within the 3D video coding algorithms. In this paper we propose a very simple algorithm to separate the foreground and background pixels for content based video coding. This algorithm exploits the 3D wavelet transform characteristics and only adds as small amount of overhead to the existing 3D video compression algorithms.

In this paper, we present a mesh-based motion estimation scheme for image sequence. Nodal motion vectors optimization is performed by using a multi-resolution differential method. Because our final aim is mesh tracking throughout a video sequence with optimized reconstruction, neither backward tracking nor forward tracking is well suited. One motivation of our work is to take advantage of both forward tracking (which enables tracking) and backward tracking (for its efficiency) in a `backward in forward' method. For the optimization of the nodal motion vectors, we also propose a novel approach with multi-resolution and several hierarchy levels, which, in addition, makes it possible scalable representation. This is achieved with a progressive representation defined according to a rate distortion criterion. Results are presented to illustrate the proposed methods.

IN this paper, we propose a new scene change detection (SCD) algorithm, and also provide a novel video-indexing scheme for fast content-based browsing and retrieval in video databases. We detect scene changes from the MPEG video sequence, and extract key frames to represent contents of a shot. Then, we perform the video indexing by applying the rosette pattern to the extracted key frames, and retrieve them. Our SCD method is better than the conventional ones in terms of the SCD performance. Moreover, by applying the rosette pattern for indexing, we can remarkably reduce the number of pixels required to index and excellently retrieve the video scene.

Compressed video processing for the sake of content based retrieval saves time of the expensive decoding. In this paper, we process the compressed MPEG video data for the motion analysis of its contents. Two motion components are differentiated from each other. Firstly, the object's motion; the change in object's co-ordinates throughout consecutive frames. Secondly, the camera motion resulting from the camera effects such as zooming in and out, and panning right and left, ... etc. A trajectory is constructed for each object and represented by a spatio-temporal representation. Video objects are indexed by the actual motion of the objects, independent from the moving camera motion.

This paper presents a technique for detecting edges of a multi-surface object placed in a background having grayscale intensity same as that of the object surface, i.e. in a homogeneous background. The various surfaces of the object also have the same intensities. For these types of objects an intensity gradient does not exist across the image and thus common intensity based edge detectors such as Sobel, Prewitt and Canny are unable to detect the edges successfully. In the proposed technique, sinusoidally-coded structured light is projected obliquely onto the object surface that is viewed from a normal position. The different illumination and viewing directions cause the projected light pattern (fringes) to break at the edges of the object due to the presence of a height difference between the various surfaces of the object and its background. By locating the positions of the intensity break points in the image it is possible to locate the various edges of the object. The main drawbacks of this technique is that the accuracy of the edge detection is greatly affected by noise. However, by using the phase-shift method and processing the resulting phase image to extract edges it was found that the accuracy is almost unaffected by the presence noise. The algorithms for the proposed edge detection technique were developed based on simulated images and later applied to real images with suitable modifications.

The problem of tracking of triangulated meshes for Video Object Planes in video sequences is considered. Triangulated meshes are constructed on the basis of hierarchical region- based models of VOP. The mesh is articulated, that is each polygonal region in a VOP is Delaunay triangulated separately and all partial meshes are connected in the global triangulation. Nodal optical flow is computed from motion parameters of associated regions according to an affine motion model using a hierarchy of region-based models. The tracking of such a VOP along video sequence is based on a full region based tracking of a hierarchical region-based model of a VOP. A forward tracking scheme and a backward motion compensation for a hybrid coding are proposed. Experimental results for a complex articulated VOP are presented.

This paper describes an algorithm for multiple object tracking that takes a new occlusion reasoning approach. In order to track individual objects under occlusion conditions, we design a 2D token-based tracking system using Kalman filtering. The proposed tracking system consists of two parts: object detection and tracking, and occlusion reasoning using feature matching. The object detection and tracking part finds moving objects from their background. For object detection, we develop an adaptive background update technique. By tracking individual objects with segmentation information, we generate motion trajectories. Computer simulation of the proposed scheme demonstrates its robustness to various occlusion conditions for several test sequences.

Texture segmentation has been an important problem in image processing. Filtering approaches have been popular, and recent studies have indicated a need for efficient, low- complexity algorithms. In this paper, we present a texture segmentation scheme based on the Dual-Tree Complex Wavelet Transform (DT-CWT). The advantage of the DT-CWT over other approaches is that it offers a partially redundant representation with strong directionality. The texture segmentation scheme presented here consists of three steps: feature extraction, conditioning, and classification. A number of feature smoothing windows have been tested. Classification is performed using a modified K-NN clustering algorithm. The proposed scheme consistently achieves error rates of less than 10%.

We present a critical review of the representation available for model-based object recognition. The choice of representation has a direct impact on the ability of the system to match objects, particularly in environments where noise or occlusion are presented. This review provides a set of criteria for object representation in recognition systems and this forms the basis upon which a qualitative critical review has been formulated. Initially well established techniques for object representation are reviewed. In particular object centered representations and viewer centered representations are addressed. The primary issue which afflicts these representations is the domain of objects which they can effectively represent. Free-form representations, which overcome this problem, are also examined. In particular, parametric representations including splines, implicit algebraic surfaces and finite element techniques, and geometric models which are based on differential geometry are considered. Ultimately, only representations, which possess an elegant blend of efficiency, accuracy and large representational domain are successful in providing a recognition system with the information required to achieve the level of scene understanding required in modern computer vision.

This work presents a geometry based vision system for aircraft recognition and pose estimation using single images. Pose estimation improves the tracking performance of guided weapons with imaging seekers, and is useful in estimating target manoeuvres and aim-point selection required in the terminal phase of missile engagements. After edge detection and straight-line extraction, a hierarchy of geometric reasoning algorithms is applied to form line clusters (or groupings) for image interpretation. Assuming a scaled orthographic projection and coplanar wings, lateral symmetry inherent in the airframe provides additional constraints to further reject spurious line clusters. Clusters that accidentally pass all previous tests are checked against the original image and are discarded. Valid line clusters are then used to deduce aircraft viewing angles. By observing that the leading edges of wings of a number of aircraft of interest are within 45 to 65 degrees from the symmetry axis, a bounded range of aircraft viewing angles can be found. This generic property offers the advantage of not requiring the storage of complete aircraft models viewed from all aspects, and can handle aircraft with flexible wings (e.g. F111). Several aircraft images associated with various spectral bands (i.e. visible and infra-red) are finally used to evaluate the system's performance.

This paper proposes a new and efficient algorithm for block matching motion estimation. It reduces search times and improves performance of motion estimation. Instead of using the fixed first search point to be alike previous search algorithms to search motion vector, the proposed method finds more accurate first search point as to compensating search area using temporal correlation of motion vector. Therefore, the proposed method could reduce search times and improve the accuracy of motion estimation. The main idea of proposed method is based on the property of consistent directivity and center-biased distributions of motion vector. Simulation results show that the PSNR values are improved up to the 3.4 dB as depend on the image sequences and advanced about 1.5 dB on an average. The results of the comparison show that the performance of the proposed algorithm is better than those of other fast search algorithms whether the image sequence contains fast or slow motion, and is similar to the performance of the Full Search algorithm. Simulation results also show that the performance of the proposed scheme gives better subjective picture quality than the other fast search algorithms.

A new block based motion estimation and compensation technique named overlapped multi-resolution motion compensation (OMRMC) is proposed. The algorithm employs the overlapped block motion compensation algorithm in wavelet domain to reduce the blocky artifacts in the predicted frames using the multi-resolution motion estimation (MRME) technique. Simulation results showed the use of OMRMC reduced up to 26% of the displaced frame difference energy compared with MRME.

In this paper, a fast motion compensation algorithm is proposed that improves coding efficiency for video sequences with brightness variations. We also propose a cross entropy measure between histograms of two frames to detect brightness variations. The framewise brightness variation parameters, a multiplier and an offset field for image intensity, are estimated and compensated. Simulation results show that the proposed method yields a higher peak signal to noise ratio compared with the conventional method, with a greatly reduced computational load, when the video scene contains illumination changes.

Typical motion estimation for block-based video coding schemes consists of two parts: the one at integer pixel accuracy and the other at half pixel accuracy. In this paper, integer pixel motion estimation algorithms are first discussed in terms of three technical categories: search step, search pattern and decision of the initial motion vector. With the development of the efficiency of integer pixel motion estimation, the computation load of widely used full half pixel search becomes relatively higher. In order to further improve the speed of half pixel search, this paper proposes a paraboloid prediction based fast half pixel search algorithm. Experimental results show that variable search step, search pattern with less points and predicted initial motion vector helps to improve the performance of fast integer pixel search and the proposed fast half pixel search increases the speed of half pixel search with almost not affecting the image quality.

For MPEG-II and other hybrid MC/DPCM/DCT based video coding standards, it is very important to reconstruct the predicted frames based on the block motion information. In case of transmission over unreliable channels, error concealment methods are introduced to recover the lost or erroneous motion vectors. In this paper, a novel side motion estimation method is proposed to recover the lost motion vectors by selecting from a candidate motion vector set. The outer boundary of the lost block is used to perform motion estimation and the recovered motion vector is the one that minimizes the squared error of the block boundary pixels between two consecutive frames. The method takes advantage of the same motion direction of most blocks and their boundaries. It releases the boundary pixel gray level continuity assumption of traditional boundary match/side match approaches so that better estimation result can be achieved. Overlapped block motion compensation is also incorporated in the proposed method to reduce the blocking artifacts. By reducing the number of motion vectors in the candidate set, the performance of the proposed algorithm can be further improved.

A new method to remove impulse noise from images using lifting wavelet filters which contain free parameters is presented. High frequency components obtained by wavelet decomposition are large around impulse noise. Lifting wavelet filters are designed by tuning their free parameters so as to vanish high frequency components at the location of impulse noise. The designed filters have features of impulse noise. Detection of impulse noise can be done by applying the learnt filters to corrupted images. Reduction of impulse noise from the image is carried out using a wavelet reconstruction formula.

A joint optimization method for design of filter banks and related dithered quantizers by using time-domain formulation of filter banks is provided. Mathematical analysis and experiments show that in image subband coding by appreciate choice of synthesis filter bank and dithering, it would be possible to replace signal-dependent artifacts with signal- independent white noise. This process improves the perceptual quality of reconstructed image, especially at low bit-rates and makes it more practical for a noise-removal algorithm to remove the output noise. Experiment results illustrate the effectiveness of dithering for lower frequency bands and modification of synthesis filter bank for higher frequency bands.

We investigate the scenario of using the Automatic Repeat reQuest (ARQ) retransmission scheme for two-way low bit-rate video communications over wireless Rayleigh fading channels. We show that during the retransmission of error packets, due to the reduced channel throughput, the video encoder buffer may fill-up quickly and cause the TMN8 rate-control algorithm to significantly reduce the bits allocated to each video frame. This results in Peak Signal-to-Noise Ratio (PSNR) degradation and many skipper frames. To reduce the number of frames skipped, in this paper we propose a coding scheme which takes into consideration the effects of the video buffer fill-up, an a priori channel model, the channel feedback information, and hybrid ARQ/FEC. The simulation results indicate that our proposed scheme encode the video sequences with much fewer frame skipping and with higher PSNR compared to H.263 TMN8.

This paper introduces some preliminary results of the standardization process of ITU-T's H26L project. This forthcoming video coding standard will not only significantly improve the coding efficiency, but it will also introduce new concepts such as network friendliness, which are not common in current video coding approaches. The paper focuses on the high level syntax which resides hierarchically above the macroblock layer. Data partitioning techniques are used to separate data of different types from each other. The partitions are arranged in packets of data with different importance for the reproduced picture quality. Along with unequal error protection, which can either be a function of the underlying network or implemented on the application layer, the error resilience and thus the reproduced picture quality in error prone environments is greatly improved. To verify the findings, simulation results for an Internet/RTP environment, based on real-world observations of the current Internet that do not assume network-based quality of service, are included.

Many methods for video transmission error control have been proposed recently, especially in relation to transmission over bursty-error channels. However, a thorough, structural taxonomical framework for analysis and design of such methods seems lacking. Such a framework helps clarify in thought and aids in inspiring new error control methods or combinations thereof. We present a framework for classification of the various transmission error control techniques. We then consider error control for H.263. Several techniques are presented from the viewpoint of the proposed analysis framework and they illustrate how different techniques can be integrated coherently to achieve enhanced error resilience in the overall system. In particular, we employ slotted multiplexing at the multiplex level to reduce synchronization errors in variable-length coded data and to randomize the locations of the remaining error-corrupted image areas. At the source level, we introduce two standard-compatible schemes, called length- based intra refresh and motion vector pairing, respectively, which further limit spatial-temporal error propagation.

The performance analysis of unequal error protection codes for image transmission over noisy channels and AWGN channels is considered in this paper. The International Standards Organization has proposed the JPEG standard for still image compression. JPEG standard not only provides the basic feature of compression as baseline algorithm but also provides the framework for reconstructing images in different picture qualities and sizes. These features are referred to as SNR and spatial scalability, respectively. The ability of calculation of the bit sensitivity of the compressed image data enables one to perform a highly efficient unequal error protection for image transmission over noisy channels. A channel code is proposed to protect progressively compressed and packetized image data that is transmitted across noisy channels. A group of unequal error protection codes using Trellis Coded Modulation (TCM) for the JPEG compressed images is considered. Computer simulated results of two levels of UEP with TCM and four levels of UEP with TCM are presented.

A new trellis decoding method of block codes based on the GAC structure is applied to Rayleigh fading channels in this paper. Performance of this decoding technique with hard- decision and soft-decision is studied respectively. Researches are also carried out to introduce this maximum- likelihood decoding method to narrow-banded mobile image transmission systems.

Past approaches on motion estimation use iterative algorithm to produce dense motion fields, which is modeled by the energy functions. The optimization strategy such as simulated annealing or iterated conditional mode reorganize the motion fields slowly. This paper introduces adaptive block matching and multiscale smoothing as an initial motion fields for bayesian based motion estimation. The adaptive block matching is a local intensity matching procedure, which gives a unique matching results. The results are smoothed by multiscale smoothing algorithm. This algorithm is based on kalman filter, but the time domain of this filter becomes the scale domain. The result shows that this strategy can give a more global motion fields than the result of single resolution bayesian motion estimation method. This multiscale smoothing algorithm have numerous possibility to enhance the speed as well as strategy to produce better motion fields.

Motion estimation represents the main computational burden of every hybrid video encoder. Various solutions have been proposed in order to reduce the number of operations needed for this task, trying to keep good quality of the estimation and of the relative encoded video. In this paper we propose an algorithm that, exploiting the statistical properties of the motion field, searches a number of points dynamically related to the evolution of the sequence. A subsampling pattern of the macroblock is also proposed to reduce the overall impact of the motion estimation in an MPEG encoder.

In this paper a new fast motion estimation algorithm is presented. The algorithm, named as Predictive Diamond Search, is actually based on the Diamond Search (DS) algorithm, which was recently adopted inside the MPEG-4 standard. The DS algorithm, even though faster than most known algorithms, was found not to be very robust in terms of quality for several sequences. By introducing a new predictive criterion and some additional steps in DS, our simulation results show that the proposed algorithm manages to have similar complexity with the DS algorithm, while having superior and more robust quality, similar to that of the Full Search algorithm.

A hybrid algorithm for estimating true motion fields is proposed in this paper. This algorithm consists of three steps: block-based initial motion estimation, image segmentation, and wrong motion vector correction based on objects. The hierarchical block-matching algorithms are improved for the initial motion estimation. The improved algorithm uses an adaptive technique to propagate motion vectors between hierarchical levels. It produces accurate motion field everywhere, except in the areas of motion occlusion. In order to correct wrong motion vectors in the areas of motion occlusion, the current image is segmented into objects and an object-based method is proposed to process the estimated motion fields. With the object-based method, wrong motion vectors are detected by approximating the estimated motion field in each object with a motion model, and are corrected using an object-adaptive interpolator. The object-adaptive interpolator is also used to increase the density of the motion field. Experimental results show that the improved hierarchical block-matching algorithm outperforms the conventional hierarchical block- matching algorithms. The proposed algorithm results in dense motion fields that are smooth within every object, discontinuous between objects of different motion, and very close to the true motion fields.

In this paper a hierarchical spatial scalable wavelet video coder is presented. The scheme employs backward motion compensation, therefore no motion vectors have to be transmitted. The coarser levels of the wavelet decomposition of the current frame are used for motion estimation and motion compensation of the lowpass band of the next finer level. The lowpass band of the coarsest level has to be coded separately, e.g. using DPCM.

This paper describes how to integrate encryption in an algorithm which uses wavelets to compress data. Wavelets are more useful in image compression than other methods. This is because the definition about which data should be left out is more flexible. This adaptation of wavelets to specific features of images leads to acceptable results. The use of computers in image transfer will increase further and it will therefore become necessary to transfer image data quickly. In addition, a protection of the image against unauthorized access will become necessary. Image compression increases the amount of time needed for transferring an image. Regarding the area of telemedicine, there is a demand for the protection of images. The reason for this demand is the need to prevent the relationship between patient data and an image from being determined. Therefore, the approach intends to describe a combination of image compression by using wavelets and to integrate encryption into the wavelet- transform and -compression procedure.

The wavelet transform is usually performed on a whole image. However when the amount of memory available for the transformation is limited, the input image is partitioned into non-overlapping blocks and then each block is processed independently. Quantization, that typically follows the transformation procedure in a compression system, inevitably introduces distortion, which becomes especially pronounced along the boundaries of the blocks. In this paper we show that a significant reduction in these block boundary artifacts can be achieved by choosing odd block sizes given by 2^N + 1 rather than the conventional even block sizes given by 2^N. We show that, for the same coefficient entropy, an image compressed using 17 X 17 blocks of wavelet coefficients has at least 1 dB higher PSNR than an image compressed using 16 X 16 blocks of wavelet coefficients.

A distributed multimedia computing system consists of sources, processing units and presentation devices in which each component operates autonomously. This independency can be exploited for optimization of individual component performance. Flexibility in performance improvement is enhanced by using independent clock domains. This paper presents a Video I/O model for such a multimedia system. This model provides an asynchronous communication interface for independent clock domains with the ability to synchronize a video display to one of the video sources. The communication interface has been used for the design of I/O modules in a multimedia system, which are briefly outlined.

This paper presents a flexible architecture for motion estimation and compensation using a 1D pipelined systolic array. It has been specifically designed to implement the four-step search algorithm but can easily be adapted to a wide range of other reduced-complexity search algorithms. The intention is for the architecture to be incorporated into the digital compression unit of a single-chip video camera, the target application of which is as a device enabling people to communicate using sign-language over a standard phone line. The complete architecture has been implemented as register transfer level VHDL code and its functionality has been verified by simulation. The final VLSI layout will be a combination of synthesized and custom- designed cells.

The most crucial backend algorithm of the new MPEG-4 standard is the computational expensive rendering of arbitrary shaped video objects to the final video scene. This co-processor architecture presents a solution for the scene rendering of the CCIR 601 video format with an arbitrary number of video objects. For the very high data bandwidth rate a hierarchical memory concept has been implemented. The total size of all rendered objects for one scene may reach two times the size of the CCIR 601 format. Running at 100 MHz clock frequency, the co-processor achieves a peak performance at about two billion multiply- accumulate operations. The co-processor has been designed for a 0.35 micrometers CMOS technology. About 60% of the overall area of 52 mm² is used for on-chip static memory. The power consumption of the co-processor has been estimated with 1 W.

There are huge bit-level operations in shape coding algorithms for MPEG-4. These bit-level operations make it difficult to implement real-time MPEG-4 coding in the general-purpose processor without any hardware support. This paper first gives the analysis of computation complexity in MPEG-4 shape coding algorithms. Then a dedicated VLSI architecture to accelerate shape coding, which is called Shape Engine, is proposed. Shape Engine is composed of three dedicated but flexible processors element (PE) namely PB-PE, Filtering-PE and CAE-PE. The combination of RISC core and Shape Engine leads to a great speed up over pure software implementation of MPEG-4 shape coding.

In this paper, a CMOS circuit for flexible read-out of imagers is proposed allowing random access, sequential access and window based access to the pixels. The circuit has been implemented within a CMOS imager using 0.7 micrometers technology. It is shown that this versatile read-out technique is obtained with only 8% increase in the silicon area as compared to the often used sequential read-out technique. The read-out circuit is fully digital, which makes it more robust against sizing mismatch. The circuit operates at a maximum frequency of 50 MHz which makes it very attractive for real time applications.