This paper proposes a method of error detection and recovery by hiding specific information into video bitstream using fragile watermarking and checking it later. The proposed method requires no additional bits into compressed bitstream since it embeds a user-specific data pattern in the least significant bits of LEVEL's in VLC codewords. The decoder can extract the information to check whether there is an error in the received bitstream. We also propose to use this method to embed essential data such as motion vectors that can be used for error recovery. The proposed method can detect corrupted MB's that usually escape the conventional syntax-based error detection scheme. This proposed method is quite simple and of low complexity.

We propose a new motion estimation (ME) algorithm adequate for temporal error concealment. To increase spatial motion vector (MV) correlation for efficient temporal error concealment, the proposed algorithm utilizes an enlarged macroblock (MB) for ME, which includes pixels in the adjacent MBs in addition to an original MB. It is proved that the algorithm considerably improves error concealment performance, while minimizing the degradation of encoding performance.

This paper presents an embedded video compression with error resilience and error concealment using three dimensional SPIHT (3-D SPIHT) algorithm. We use a new method for partitioning the wavelet coefficients into spatio-temporal (s-t) blocks to get higher error resilience and to support error concealment. Instead of grouping adjacent coefficients, we group coefficients in some fixed intervals in the lowest subband to get interleaved trees. All of the sub-blocks correspond to a group of full frames within the image sequence, because each group of interleaved trees has coefficients dispersed over the entire frame group. Then we separate the stream into fixed length packets and encode every one with a channel code. Experiments show that our proposed method brings higher error resilience in noisy channels since the decoded coefficients associated with early decoding error would be spread out to the whole area along with the sequence, and we can conceal the lost coefficients with the surrounding coefficients even if some of substreams are totally missing. In addition to that, our proposed method gives higher coding performance in noseless channels than the conventional grouping method of grouping contiguous trees.

In this paper, a further study based on the results of is conducted to accomplish a near-optimum design of stream-shuffling and error concealment for error-resilient transmission of wireless video. In particular, a two-phase stream-shuffling scheme is systematically scheduled in conjunction with embedded channel codes, while a new error detection method and adaptive error concealment algorithm are contrived to effectively mitigate the effects of residual errors on visual quality. Comparison with the algorithms proposed in, the new methodology may provide better understanding of optimum design of stream-shuffling in a systematic way, which yielding improved performance in wireless video communication.

With the emerging of the third generation (3G) wireless technology, digital media, like image and video, over wireless channel becomes more and more demanding. In this paper, the measure metrics for the wireless image is proposed and a Qos-guarantee error control is presented, combining UEP with Forward Error Correction (FEC) and Automatic Repeat reQuest (ARQ), aiming to high quality image transmission with short delay and little energy. Simulation results show that our scheme can achieve good reconstructed image with few retransmission times and small bit budget under different channel conditions, which can reduce the energy consumed in the network interface.

An enhanced network service model for layered video multicast applications is presented for differentiated service (Diffserv) networks in this work, which consists of enhanced active queue management (AQM) and hierarchial priority marking. It is found from experiments that a straightforward application of the receiver-driven layer multicast (RLM) model to the Diffserv network may perform poorer than to the best-effort Internet, since RLM applications could behave like unresponsive flows and conflict with Diffserv active queues. To fix this problem, we present three sets of extended RED/RIO parameters as the enhanced AQM model, which is designed to be generally applicable to service protection as well as to support RLM traffic in Diffserv networks. Furthermore, to take advantage of the priority service in Diffserv networks, we present a priority marking model of RLM video layers that conform to the priority marking architecture of Dffserv networks. It is shown by simulation experiments that the proposed network service enhancement model improves the stability of the join experiment and reduces the packet loss rate to support RLM traffic in Diffserv networks.

In this research, a video delivery system is proposed by integrating the layered codec and the proposed corruption model. In the system, it is assumed that the base layer packets are delivered without loss since they are assigned the most reliable service level. Then, packets in the enhancement layer are analyzed with the corruption model and categorized for differentiated delivery services. The corruption model for generating the relative priority index (RPI) is extended to layered bitstreams. In addition to taking into account the initial error and the propagation error due to packet loss, the dependency between the same and different layers is reflected in the modified macroblock corruption model. In order to facilitate the prioritized delivery of the enhancement layer, the RPI-based corruption model for each packet as well as coordinated delivery of packetized video over QoS networks are investigated. Finally, a per-packet optimization framework with unequal error protection (UEP) is realized to consider both end-to-end video performance and pricing. The performance of the proposed solution is verified with extensive simulations.

This work proposes and implements a system to transmit in real-time slow video signals - in particular video conference signal, that is head and shoulder sequences - over the public network. The proposed codec is based on a multiple description approach that gives N equally important and independent flows. The advantage of this codec is an intrinsic robustness to the transmission errors and to the packet loss, as the simulation results have proved. This feature results very suitable not only for IP-network, but also for every packet network like the next generation mobile systems. The approach is important also for the end user scalability since each device can decide the information to receive according to the resolution of the display and the bandwidth of the connecting link using the same source data stream.

TCP-friendly layered video transmission for the Internet multicast application is investigated in this research. To adapt the available bandwidth in a TCP-friendly manner, a receiver-driven rate adaptation approach based on layered video is used. The receiver adjusts its number of subscription layer based upon the TCP-throughput estimation obtained from our previous work known as smooth and fast rate adaption mechansim(SFRAM). Since SFRAM mitigates unnecessary rate fluctuation and responds only to distinct network variations, a receiver can add or drop a certain number of video layers in a very stable manner. Furthermore, we apply the priority dropping mechanism (PDM) to protect the video layer selectively. To achieve fairness among different flows with PDM, we propose a new packet classification scheme (PCS) to assign the priority to each layer based upon the target rate. Since proposed PCS can also distinguish the priority for packets even within the same layer, it is suitable for Internet video multicast. With PCS, we implement PDM on the random early dropping (RED) queue to drop packets of the lowest priority layer so that end users may avoid instant error propagation. We conduct NS simulations to examine the proposed layer adaptation behavior and the effect of PDM. Our results show that SFRAM is effective enough to direct the right number of subscription layer stably while it behaves in a TCP-friendly manner. Furthermore, PDM on RED can protect layered video of the highest priority effectively with proposed PCS.

This paper presents an effective VOD transmission scheme under RCBR network that is suitable for the statistically non-stationary video traffic since it supports the bandwidth renegotiation during data transmission. The proposed system includes two parts: the VOD server architecture and traffic smoothing/admission control algorithms. The proposed VOD server system consists of video DB and metadata DB that includes compression history information including the quantization parameters versus output rate profile, encoded frame positions and encoding types, etc. And, the proposed traffic smoothing and admission control algorithms are performed based on the aforementioned metadata and control the traffic properties of the selected VOD through the communication between them to be accommodated under the current network situations. And, the experimental results are provided to demonstrate the efficiency of the proposed system.

In network delivery of compressed video, packets may be lost if the channel is unreliable. Such losses tend to occur in burst. In this paper, we develop an error resilient video encoding approach to help error concealment at the decoder. We introduce a new block shuffling scheme to isolate erroneous blocks caused by packet losses. And we apply data hiding to add additional protection for motion vectors. The incorporation of these scheme adds little complexity to the standard encoder. Experimental results suggest that our approach can achieve a reasonable quality for packet loss up to 30% over a wide range of video materials.

A synchronization scheme that can specify the start position of each macroblock in a compressed video bitstream for low data rate wireless applications is proposed by extending the Error Resilient Entropy Coding (EREC) method. Our scheme is implemented in the form of a transcoder, placed before and after the channel, of a MPEG-4 Simple Profile bitstream. We compare our proposed technique using reconstructed video quality (measured in PSNR) and the length of the extra redundancy bits incurred by the transcoder with the error resilient tools of H.263⁺. A simple syntax-based codeword repair method is also proposed so that the transcoder generates a MPEG-4 compliant bitstream which can then be decoded with a standard MPEG-4 decoder such as MoMuSys (FDIS V1.0).

In this paper we present a method of creating domain-based multiple descriptions of images and video. Descriptions are created by partitioning the domain of the signal into sets whose points are maximally separated from each other. This property enables simple error concealment methods to produce good estimates of lost signal samples. We present the approach in the context of Internet transmission of subband-coded images and scalable motion compensated 3-D subband-coded video, but applications are not limited to these scenarios.

A block-interlaced codewords realignment (BICR) scheme is proposed for error resilience enhancement video transmission over burst error channels. The video coder is compatible with the MPEG-4 coding standard. Specifically, based on the error propagation property of the variable length coding (VLC) and reversible variable length coding (RVLC), the proposed scheme realigns the codewords within a minimum data unit according to the loss probability and the source significance information. This realignment automatically achieves an unequal error protection for different codewords of the coded video bit-stream with different importance, increasing the error resilience of the video coding without any penalty in transmission efficiency. Simulation results confirm that this realignment can increase the quality of the reconstructed video frames remarkably over channels with burst errors in terms of PSNR criterion.

In this paper we proposed a temporal error concealment scheme utilizing the overlapping approach for the block-based video coder. In the proposed algorithm, each pixel of the lost block is recovered by overlapping from four nearest available neighboring blocks. The weighting coefficients used for overlapping have been carefully designed to address the suitable weights on different pixel locations inside the lost block. The weighted extended boundary match is proposed to replace the popularly used boundary match for recovering the lost motion vectors, such that the recovered motion vector can provide the best lost pixel prediction for the overlapping error concealment process. The experimental results were shown by integrating our algorithm into the H.263+ coder.

We present a low cost high performance scheme for video transcoding. Our scheme includes the information look-ahead structure and the transcoding algorithm. It is this look-ahead mechanism that greatly facilitates the use of information embedded in the bitstream for informative and intelligent transcoding. Furthermore, our transcoding algorithm has four new features: 1) picture complexity look-ahead for efficient rate allocation and control; 2) dynamic bit allocation weighting on picture types based on the scene complexity and bit rate; 3) adaptive nonlinear rate control on macroblocks based on complexity distribution in the picture; 4) Intra frame requantization adjustment to reduce the transcoding distortion. The software simulation results have proven that the performance of our transcoding scheme is significantly better than the reencoding scheme and comparable to the direct encoding scheme.

In video transcoding, useful coding statistics become available when the input video stream is decoded. It is possible to utilize these statistics to improve the transcoding performance. In this paper we investigate how to use these statistics to perform the picture bit-allocation for transcoding a pre-encoded video bit-stream. We propose a scheme to estimate the complexities of the pictures of the output video using the coding statistics computed from the input video stream. Based on the estimated picture complexity, we present a picture bit-allocation algorithm for the rate-control of the transcoding process. The algorithm is simple to compute, and effectively improves the video quality and reduces the picture quality variations.

In this paper, we work on the problem of inserting logo into part of a previously compressed video sequence while transcoding it to a video stream with lower rate. In most of the transcoding architecture, it has been widely accepted to reuse the motion information extracted from the original incoming bit stream. For our transcoding and logo insertion process, we preserve the same motion information as the incoming video stream for the parts that were unaffected by logo, and propose an efficient algorithm to code logo areas. We first find the proper range of pictures that might be affected by logo insertions, analyze logo-affected parts of those pictures, and perform corresponding algorithms depending on the picture types. In addition, we develop a method to achieve better bit allocation during logo insertion. The efficiency of our algorithm is demonstrated by simulation.

This paper discusses the problem of reduced-resolution transcoding of compressed video bitstreams. An analysis of drift errors is provided to identify the sources of quality degradation when transcoding to a lower spatial resolution. Two types of drift error are considered: a reference picture error, which has been identified in previous works, and error due to the non-commutative property of motion compensation and down-sampling, which is unique to this work. To overcome these sources of error, four novel architectures are presented. One architecture attempts to compensate for the reference picture error in the reduced resolution, while another architecture attempts to do the same in the original resolution. We present a third architecture that attempts to eliminate the second type of drift error and a final architecture that relies on an intra block refresh method to compensate all types of errors. In all these architectures, a variety of macroblock level conversions are required, such as motion vector mapping and texture down-sampling. These conversions are discussed in detail. Another important issue for the transcoder is rate control. This is especially important for the intra refresh architecture since it must find a balance between number of intra blocks used to compensate errors and the associated rate-distortion characteristics of the low-resolution signal. The complexity and quality of the architectures are compared. Based on the results, we find that the intra refresh architecture offers the best trade-off between quality and complexity, and is also the most flexible.

MPEG-2 and MPEG-4 are two most popular international video compression standards. They will coexist in different systems and networks for a long time. For compatibility of these two video standards, this paper put forward three transcoding algorithms on the basis of detailed analysis of computational complexity. In the paper, the principle and implementation of two efficient transcoding algorithms are emphasized. Both of them are tradeoff between the computational complexity and reconstructed video quality. One is motion-information reusing algorithm, which meets the highest video quality requirement of studio and post-processing environments. The other is real-time low complexity transcoding algorithm, which is proper for low delay requirement of the network server and user terminal with limited computational performance. The simulation results of many test sequences verified the high performance of the algorithms.

Current rate control schemes in video coding standards do not have efficient frame-level bit allocation due to the limitation of real-time encoding. In this paper, by taking advantage of offline video encoding, we proposed a two-pass video encoding scheme for low bit rate streaming applications. Specifically, in the first pass, we generate the feature information of video sequences, including rate- distortion (R-D) functions and scene change boundaries. Then, in the second pass, according to the available channel bandwidth, by exploiting the feature information, we are able to implement frame-level bit allocation in an optimal way so that video sequences can be coded at low bit rate with an improved quality. Experimental results show the proposed scheme is able to achieve not only the improved PSNR results but also much smoother visual quality.

Streaming media over heterogeneous lossy networks and time-varying communication channels is an active area of research. Several video coders that operate under the varying constraints of such environments have been proposed recently. Scalability has become a very desirable feature in these video coders. In this paper, we make use of a leaky-bucket rate allocation method (DBRC) that provides constant quality video under buffer constraints, and extend it in two advantageous directions. First, we present a rate control mechanism for 3D wavelet video coding using DBRC. Second, we enhance the DBRC so that it can be utilized when multiple sequences are multiplexed over a single communications channel. The goal is to allocate the capacity of the channel between sequences to achieve constant quality across all sequences.

Proxy cache located at base station is helpful for improving performance of wireless multimedia streaming. In this paper we propose a cooperative proxy cache architecture, MobileCache, for cell-based wireless environment. The proxies cooperate to provide seamless streaming for wireless client even when the client crosses the cell boundary. In order to improve the efficiency for MobileCache, we develop a cache cooperation management scheme, including cache replacement, cache migration and cache replication, which maximize the revenue for MobileCache. Simulation results confirm that, by employing the cache cooperation management, MobileCache achieves higher performance than other cache management schemes.

Streaming video over IP is studied by integrating the error resilient scalable source codec with constant quality rate adaptation and prioritized network transmission. The scalable codec is based on the standardized ISO/IEC MPEG-4 fine granular scalable (FGS) coding scheme, which allows arbitrary truncation according to the given rate budget. To maximize the end-to-end visual quality, constant quality video rate control (CQVRC) is proposed to preserve smooth video quality when there is a significant amount of variation in the video source, the available bandwidth or both. The proposed CQVRC attempts to provide smooth video quality for both the base layer (BL) and the enhancement layer (EL). The video stream is then prioritized for priority dropping, where rate adaptation is dynamically performed to meet the time-varying available bandwidth. This kind of prioritized stream can benefit from QoS networks such as the IP differentiated service (DiffServ). All key components, including FGS encoding, CQVRC based rate adaptation and packetization, error resilient decoding, and differentiated forwarding, are seamlessly integrated into one system. By focusing on the end-to-end quality, we set both source and network parameters properly to achieve a superior performance of FGS video streaming.

This paper proposes a new content-based rate shaping algorithm for Fine Granular Scalable scheme (FGS). FGS has been adopted by the MPEG-4 video standard as the core compression tool for video streaming over Internet, and has enabled a wide range of multimedia applications. However, most of current video streaming technologies only protect the video stream by rate allocation or CRC means, based on the bandwidth of the network. As a result, the video quality that user expected is unrelated with the content of video. Our approach is innovative in that it is based on video content analysis and extraction of the information of video content. Firstly, we evaluate the importance of the video sequence by using NFL (Nearest Feature Line) method. Then we drop the enhancement layer in term of the importance, which is decided by the bits sent out that meet the current bandwidth of network. The experimental results indicate that our layer dropping method not only improves the performance of FGS to 0.2 dB, but also enhances the subjective quality of video effectively.

In this paper, we investigate the object placement problem in distributed cooperative proxy systems and build a mathematics model. We prove that the problem is a NP-Hard problem and two novel greedy heuristic placement algorithms MWLC and MWGB are proposed to get the near-optimal answer in a polynomial complexity. We perform simulation and the results reported in this paper show that these two algorithms could achieve better performance than the traditional MFU algorithm (about 13.05% and 14.13% respectively). By the analysis of the experiment results, we find that the two algorithms can efficiently reduce the average request cost and balance the load of proxies. In general, the technologies proposed in this paper could be used to develop effective distributed cooperative proxy systems such as CDSP and digital library and so on.

Illumination variability has a considerable influence on the performance of computer vision algorithms or video coding methods. The efficiency and robustness of these algorithms can be significantly improved by removing the undesired effects of changing illumination. In this paper, we introduce a 3-D model-based technique for estimating and manipulating the lighting in an image sequence. The current scene lighting is estimated for each frame exploiting 3-D model information and by synthetic re-lighting of the original video frames. To provide the estimator with surface normal information, the objects in the scene are represented by 3-D shape models and their motion and deformation are tracked over time using a model-based estimation method. Given the normal information, the current lighting is estimated with a linear algorithm of low computational complexity using an orthogonal set of light maps. This results in a small set of parameters which efficiently represent the scene lighting. In our experiments, we demonstrate how this representation can be used to create video sequences with arbitrary new illumination. When encoding a video sequence for transmission over a network, significant coding gains can be achieved when removing the time varying lighting effects prior to encoding. Improvements of up to 3 dB in PSNR are observed for an illumination-compensated sequence.

A view-dependent progressive mesh coding algorithm is proposed in this work to facilitate interactive 3D graphic streaming and browsing. First, a 3D graphic model is split into several partitions. Second, each partition is simplified independently to generate a base model that can be efficiently encoded. Third, topological and geometrical data are reorganized to enable the view dependent transmission. Before the transmission, the server is informed of the viewing parameters. Then, the server can accordingly transmit visible parts in detail, while cutting off invisible parts. Experimental results demonstrate that the proposed algorithm reduces the required transmission bandwidth, and provides an acceptable visual quality even at low bit rates.

In the field of compression, the type of 3D models traditionally considered is that of polygonal meshes, for which several efficient compression techniques have been proposed in the recent years. Nowadays, an increasing proportion of 3D models are created by a synthesis or modeling process, instead of captured from the real world. Such models are most often given as parametric surfaces, which have several advantages over polygonal meshes, such as resolution independence and a more compact representation. This paper proposes a method to code parametric surfaces, given as Non-Uniform Rational B-Splines (NURBS). The coding scheme consists in coding the NURBS parameters (knots and control points) using a predictive scheme, coupled with uniform quantization and entropy coding. The multiplicity of knots is preserved by decomposing the knot vectors in a break vector (the values) and a multiplicity map. The rate-distortion of the proposed scheme is evaluated and compared against compressed triangular meshes. The results show that a considerable compression ratio is achievable under visually lossless conditions, that outperforms by far triangular meshes. In addition of having a better rate-distortion performance, the coding scheme enables the efficient transmission of synthesized 3D models retaining their resolution independence.

In this paper, we propose a new algorithm to code the animated three-dimensional (3-D) mesh model. After we classify the mesh frames in the animation model into intra- and inter-meshes, each mesh frame is decomposed into linear triangular strips, which in turn are partitioned into several fixed-length segments. The intra-mesh is compressed by differential coding of vertex coordinates. The inter-mesh is motion-compensated segment by segment from the previous meshes and residual errors are transformed by 1-D DCT. The transform coefficients are then entropy coded. We demonstrate that the proposed algorithm yields an improved coding gain than the MPEG-4 SNHC codec.

A jointly designed source-channel coding technique that protects video signals against transmission errors is proposed in this work. We develop a low-complexity source model which can estimate the bitrate and the quantization error for each packet. Also, the channel distortion is estimated based on the packet importance and the packet loss rate. The packet importance is measured as the mean square error between the error-free reconstruction and the concealed reconstruction, and the packet loss rate is determined by channel conditions. According to the informed channel condition, the encoder adaptively assigns the quantization parameter and the channel code rate to each packet so that the expected mean square error due to source and channel distortions is minimized subject to a constraint on the overall bitrate. Simulation results show that the proposed adaptive system provides acceptable image quality even in a high bit error rate environment.

Dynamic throughput estimation for wireless multimedia transmission is examined in this research. A novel dynamic mapping scheme from measurements of a real-world wireless environment to a discrete Markov model is first proposed to simulate wireless video packet-based transmission. Then, the available throughput is estimated by using the derived discrete Markov model together with the number of negative acknowledgments (NACK) that are fed back from the receiver to the transmitter. The estimated throughput is used to perform the adaptive region-of-interest (ROI) rate control based on the multiple constraint optimization. The performance of the channel mapping scheme, the throughput estimation, and the region-of-interest rate control is demonstrated by simulating a micro-cell environment and ITU-T H.263+ video standard. Performance variation due to different estimation of throughput is compared and the performance improvement due to the multiple constraint optimization is demonstrated both in objective and subjective quality.

Video transmission over the multi-path fading wireless channel has to overcome the inherent vulnerability of compressed video to the channel errors. To effectively prevent the corruption of video stream and its propagation in spatial and temporal domain, proactive error controls are widely being deployed. Among possible candidates, turbo code is known to exhibit superior error correction performance over fading channel. Ordinary turbo codes, however, are not suitable to support the variable-size segment of the video stream. A version of turbo code, byte-aligned variable-length turbo code, is thus proposed and applied for the robust video transmission system. Protection performance of the proposed turbo code is evaluated by applying it to GOB-based variable-size ITU-T H.263+ video packets, where the protection level is controlled based on the joint source-channel criteria. The resulting performance comparison with the conventional RCPC code clearly demonstrates the possibility of the proposed approach for the time-varying correlated Rayleigh-fading channel.

This article presents a novel unequal error protection (UEP) for efficient multimedia services over mobile cellular environments, in particular for the case that runs delay- sensitive (e.g., video streaming) and delay-insensitive (e.g., ftp, http) applications simultaneously. In cdma2000 1x system, for example, one Physical Layer Packer (PLP) can obtain multiple streams from multiple information/application sources. However, each PLP is error-protected evenly without consideration f different QoS attributes of different information/application sources. The key idea to be presented in this paper is that more protection is employed to the short delay-insensitive application packets to ensure very low error rate and less protection is given to relatively long delay-sensitive application packets to get almost the same error rate as the normal case - this is using the concept of inter-media QoS classification taking advantage of Turbo coding gain and penalty. By doing this, entire rate of successful transmission for delay-insensitive application is increased so that total packet connection time is reduced with comparable quality of delay-sensitive applications. The scope of this paper is to illustrate the proposed concept, its performance gain and some identified key future works. Section 1 contains some preliminary aspects of the recent mobile cellular systems for high-speed packet access applications. Section 2 describes the key concept of the proposed scheme and its associated simulation results are presented in section 3. Conclusions are drawn in section 4 followed by the final section to identify some possible future works.

A call admission control (CAC) scheme and a resource-reservation estimation (RRE) method suitable for the interference-based wireless system, such as wide-band code division multiple access (W-CDMA), are proposed in this work. The proposed CAC scheme gives preferential treatment to high priority handoff calls by pre-reserving a certain amount of interference margin called the interference guard margin (IGM). The amount of guard margin is determined by the measurement performed by the RRE module in base stations. Each RRE module dynamically adjusts the level of guard margin by considering traffic conditions in neighboring cells based upon handoff requests. A service model is adopted to support multiple services, which includes mobile terminal's data rate, different levels of priorities, mobility and rate adaptivity characteristics. Simulations are conducted with OPNET to study the performance of the proposed scheme in terms of the objective function, blocking probabilities and system utilization under different traffic conditions.

In this paper, we present a motion compensated two-link chain coding technique to effectively encode 2-D binary shape sequences for object-based video coding. This technique consists of a contour motion estimation and compensation algorithm and a two-link chain coding algorithm. The object contour is defined on a 6-connected contour lattice for a smoother contour representation. The contour in the current frame is first predicted by global motion and local motion based on the decoded contour in the previous frame; then, it is segmented into motion success segments, which can be predicted by the global motion or the local motion, and motion failure segments, which can not be predicted by the global and local motion. For each motion failure segment, a two-link chain code, which uses one chain code to represent two consecutive contour links, followed by an arithmetic coder is proposed for efficient coding. Each motion success segment can be represented by the motion vector and its length. For contour motion estimation and compensation, besides the translational motion model, an affine global motion model is proposed and investigated for complex global motion. We test the performance of the proposed technique by several MPEG-4 shape test sequences. The experimental results show that our proposed scheme is better than the CAE technique which is applied in the MPEG-4 verification model.

This paper presents a region-based scalable coding technique that can be used in interactive transmission of images over networks. This method has a capability of near lossless coding for a specific region of interest (ROI), while the rest of the region is coded with a high quality lossy codec. The enhancement layers add refinement to the quality of the images that have been reconstructed using the basic layer of the intra frame. The proposed coding technique uses multiple quantizers with thresholds (QT) for layering and it creates a bit plane for each layer of both the intra and residual frames. The bit plane is then partitioned into sets of small areas to be coded independently. Run-length and entropy coding are applied to each of the sets to provide scalability for the entire image sets resulting in high picture quality in the end user-specified ROI. We tested this technique by applying it to various test image sequences and we consistently achieved a high level of performance.

This paper describes a bit allocation algorithm that can achieve a constant bit rate when coding multiple video objects (MVOs), while improving the rate-distortion (R-D) performance over the reference method for MPEG-4 object-based rate control. In object-based coding, bit allocation is performed at the object level and temporal rates of different objects may vary. The proposed algorithm in this paper deals with these two issues. We pay particular attention to maintenance of buffer occupancy levels and propose a new method for spatio-temporal trades-offs for object-based coding. In order to improve the coding efficiency, we also consider several new R-D coding modes. These modes are available at the encoder to determine the best R-D performance under different coding conditions and are chosen automatically. Simulation results demonstrate moderate improvements at low and high bit rates. One important key aspect of the proposed algorithm is that the actual coded bits in the proposed algorithm are similar to the target bits over a wide range of bit rates. Consequently, the proposed algorithm has not experienced the buffer overflow/underflow over the entire range of bit rates.

In this paper, we propose a modified shape adaptive discrete wavelet transform (SA DWT). In conventional SA DWT method, after several levels of decomposition, the number of coefficients that has no parent in coarser scale is usually large and uncontrollable. If a tree-structure coding method is applied to the wavelet coefficients, the coding efficiency will be decreased by these coefficients. In our proposed method the boundary of object needs to be extrapolated first and then symmetric reflection is applied. By using this kind of procedure, the number of coefficients without parent can be reduced. The proposed SA DWT is also non-expansive. We establish a modified SPIHT coding algorithm to test the new method's contribution to coding efficiency. Experiment results show that the proposed method can produce more regular LL band figure and achieve a better rate-distortion curve than that of a conventional one.

Facial Animation Parameters (FAPs) are defined in MPEG-4 to animate a facial object. The algorithm proposed in this paper to extract these FAPs is applied to very low bit-rate video communication, in which the scene is composed of a head-and-shoulder object with complex background. This paper addresses the algorithm to automatically extract all FAPs needed to animate a generic facial model, estimate the 3D motion of head by points. The proposed algorithm extracts human facial region by color segmentation and intra-frame and inter-frame edge detection. Facial structure and edge distribution of facial feature such as vertical and horizontal gradient histograms are used to locate the facial feature region. Parabola and circle deformable templates are employed to fit facial feature and extract a part of FAPs. A special data structure is proposed to describe deformable templates to reduce time consumption for computing energy functions. Another part of FAPs, 3D rigid head motion vectors, are estimated by corresponding-points method. A 3D head wire-frame model provides facial semantic information for selection of proper corresponding points, which helps to increase accuracy of 3D rigid object motion estimation.

Efficient MPEG encoding begins with the selection of each frame's coding type. Depending on the constraints of a given application, the options available for individual frame types differ. These options are further limited by the encoding environment, in particular the frame buffer size and computational power available. In this work we assume an off-line encoding environment, with its implicit unlimited frame buffer and arbitrarily fast computations. Such an environment enables us to examine ideal frame types for a variety of situations, with changing emphasis between encoded bits and motion estimation error. We find that the optimal frame types vary dramatically as focus shifts from bit allocation to motion estimation error, even for ``natural'' video sequences. When additional video effects such as fades and scene cuts are introduced, the frame types become even more interesting, deviating significantly from the M and N fixed-spacing scheme used by many real-time MPEG encoders.

Common motion compensated hybrid video coding standards such as H.263, MPEG-1, MPEG-2, MPEG-4 are based on a fractional-pel displacement vector (DV) resolution of 1/2-pel. Recent approaches like MPEG-4 (ACE-profile) and H.26L use higher DV resolutions of 1/4-pel and 1/8-pel. In order to estimate and compensate fractional-pel displacements, the image signal has to be interpolated. Especially for higher DV resolutions an efficient interpolation filter is necessary. Thus, in MPEG-4 (ACE-profile) and H.26L 6 or 8-tap Wiener interpolation filters are applied. These Wiener filters were designed to interpolate the image signal while reducing spacial aliasing components that deteriorate the motion compensated prediction. In we presented an improved interpolation filter that allows to interpolate an aliasing affected image more accurate then the conventional Wiener filter. This filter is a combination of the Wiener filter and a motion compensated interpolation filter. Up to now these filters are based on invariant filter coefficients. The same coefficients are applied for all sequences and for all images of a sequence. In we presented an adaptive interpolation filter that improves the coding efficiency by reducing the impact of displacement estimation errors and by compensating the aliasing components. The interpolation scheme of is based on filter coefficients that are adapted once per frame to the non-stationary statistical features of the sequence. This paper introduces a motion compensated adaptive interpolation filter that combines the frame-adaptive approach of and the motion compensated interpolation approach of. Due to the motion compensated adaptive interpolation filter, the coding efficiency of an H.26L codec is improved up to 0.8 dB.

An encoder-based rate smoothing technique is developed to reduce the rate burstiness in video coding. At first, we propose a new source model to accurately estimate the rate-distortion (R-D) function of each video frame with very low computational complexity. Based on the estimated R-D function, in the second step, we propose a rate smoothing algorithm to make a tradeoff between the constant bit rate and the constant picture quality. The tradeoff forces the output video stream to have both a smoothed rate shape and a consistent video presentation quality, which is highly desirable in video coding and transportation.

Video decoding at reduced resolution with resizing embedded in the decoding loop saves computational resources such as memory, memory bandwidth and CPU cycles. Key to such embedded resizing is proper filtering/scaling of DCT data, and motion compensation at the reduced resolution. Although MPEG-2 video decoding with embedded resizing has been investigated in the past, little work has been reported on solving problems associated with interlaced video undergoing decoding with embedded resizing. In particular, annoying artifacts may occur in moving areas of interlaced video due to improper scaling or motion compensation. In this paper, we introduce the notion of the Local Interlacing Property for interlaced moving areas and propose algorithms to detect and process data with the Local Interlacing Property properly in the context of decoding with embedded resizing. Specifically, we demonstrate that 1) vertical high frequency in interlaced moving areas should be preserved during downscaling, and 2) phase shift must be added for motion compensation in interlaced moving areas under certain circumstances. Experimental results show that our method effectively removes artifacts in interlaced moving areas, making MPEG-2 video decoding with embedded resizing a practical tradeoff for interlaced video.

Complexity scalable algorithms are capable of trading resource usage for output quality in a near-optimal way. We present a complexity scalable motion estimation algorithm based on the 3-D recursive search block matcher. We introduce data prioritizing as a new approach to scalability. With this approach, we achieve a near-constant complexity and a continuous quality-resource distribution. While maintaining acceptable quality, it is possible to vary the resource usage from below 1 match-error calculation per block on the average to more than 5 match-error calculations per block on the average.

Existing fast block motion estimation algorithms, which reduce the computation by limiting the number of search points, utilize the motion vector (MV) characteristics of high spatial correlation as well as center-biased distribution in predicting an initial MV. Even though they provide good performance for slow motion sequences, they suffer from poor accuracy for fast or complex motion sequences. In this paper, a new fast and efficient block motion estimation algorithm is proposed. To find an initial search point, in addition to the predictors of zero MV and neighboring MVs, the algorithm utilizes another predictor obtained from one-dimensional feature matching using selective integral projections. This low complexity procedure enables the selection of a better initial search point so that a simple gradient descent search near this point may be enough to find the global minimum point. Compared to recent fast search algorithms, the proposed algorithm has lower computational complexity and provides better prediction performance, especially for fast or complex motion sequences.

A method for scalable motion compensated up-conversion of video sequences is proposed. By computing suitable objective quality estimates,(which show a high correlation with subjective quality scores), the method allows to predict the resources needed to up-convert a given input video sequence achieving a certain visual quality of the output video sequence. This method permits to dynamically change the resource utilization in a way which is optimal for the system, e.g. for a programmable platform for media processing.

Long term memory motion compensation (LTMC) is an approach to extend the temporal motion search range by using multiple decoded frames as reference frames. By employing multiple reference frames, LTMC reduces the residual frame energy significantly. However the computational complexity of motion estimation for LTMC increases significantly as well. Therefore reduction of the required computational complexity is one of the most challenging issues for LTMC. Also, if we locate motion search windows at fixed locations in a frame buffer for a given macro-block, it is highly possible that the oldest frames in a frame buffer do not contain matching blocks due to the reduced correlation between the current frame and the reference frames located further in the frame buffer. Therefore, if we can locate the motion search window at a good position adaptively in a frame buffer, we can enhance the gain performance of LTMS. In this paper, we propose a novel motion estimation algorithm for LTMC to reduce significantly the required computation complexity, and to enhance the performance of LTMC. For the proposed motion estimation algorithm, we introduce a directed search strategy. Also, we propose to employ hypothesis testing fast matching (HTFM) as a fast matching criterion. The goal of a directed search strategy is to let the location of the motion search windows change adaptively as the search proceeds to older frames in the frame buffer. The main benefit over standard, fixed window, approaches is that the algorithm can track larger motion and therefore, we can reduce the residual frame energy. In addition, because the directed search strategy keeps track of best matched blocks, we can reduce the computational complexity significantly by reducing the motion search window area in a frame buffer. Simulation results show that by employing the directed search with reduced motion search window, we can reduce the computational complexity approximately 30%-40%, and that by employing HTFM, we can reduce the computational complexity as additional 40%-50% as compared to the result of a full search algorithm which employs a non-directed search strategy for LTMC. The proposed algorithm provides particularly significant gains for video sequences which contain large motions. In the case of high motion sequences directed search also results in slight gains in PSNR over non-directed search.

In this paper, a large range motion estimation algorithm based on hierarchical motion vector estimation is proposed. Aimed at objects move in large area in image sequences, the proposed algorithm encounter with problems caused by large range search, such as huge computational complexity and relative bad performance. The computer simulation results are also given for various test sequences.

3D modeling of human face has been an active research topic in computer vision and computer graphics communities. In this paper we develop a face-modeling system from calibrated stereo video input. The stereo configuration is chosen according to the characteristics of the face structure. For data capturing the subject sits in front of the calibrated stereo video cameras and rotates his/her head from front view to side view. Then we can create his/her 3D face model with realistic texture mapped, which is ready for MPEG-4 compatible facial animation, by adapting a generic wireframe model to the input. In deciding 3D face geometry we utilize two kinds of 2D information. One is the correspondence between stereo pair; the other is self-occluded contour. After some 3D points in the specific face are obtained from both image clues in different frames, RBF (Radius Basis Function) data interpolation is performed to decide its shape. Afterwards texture image is generated from the face geometry and video. Experiments show that satisfactory results can be reached with our system.

Recent deployments of digital cameras and fast PCs have led real-time interactive systems to adopt vision-based interfaces. However, 2D vision-based interfaces, due to the lack of 3D information, have inherent weakness in tracking 3D objects in real world. In this paper, we propose a vision-based 3D interface for interactive systems that allows an object tracking on the fly in 3D by exploiting multi-view images. Due to the characteristics of interactive systems, requiring real time processing, the main challenge is a simple and robust estimation of 3D information from the images captured by asynchronous digital cameras. The proposed vision-based 3D tracking consists of three steps: (i) dynamic stereo calibration (ii) simple object segmentation (iii) robust 3D movement tracking. To show its effectiveness of the proposed framework, we applied the proposed 3D interface to an interactive 3D Gumdo simulation. Due to the simplicity and robustness of the proposed framework, it can be applied to various real-time interactive applications.

This paper presents a video watermarking scheme for copyright notification and protection. Each video frame is clustered into high motion and low motion regions. Dual watermarks are designed by exploiting video motion, perceptual characteristics, and some other informative cues. They are embedded into certain macroblocks in these two regions respectively. The watermark in the low motion region is resilient to frame-level attacks such as frame dropping, frame averaging and frame reshuffling. The watermark in the high motion region is robust to statistical attacks. The detection does not need the original frame or any a priori index information of frames in video sequence. Good invisibility as well as strong robustness can be obtained. The experimental results validate the effectiveness of the system.

In this paper we present a new method to extract both superimposed and embedded graphical texts in a freeze-frame of news video. The algorithm is summarized in the following three steps. For the first step, we convert a color image into a gray-level image and apply contrast stretching to enhance the contrast of the input image. Then, a modified local adaptive thresholding is applied to the contrast-stretched image. The second step is divided into three processes: eliminating text-like components by applying erosion, dilation, and (OpenClose + CloseOpen)/2 morphological operations, maintaining text components using (OpenClose + CloseOpen)/2 operation with a new Geo-correction method, and subtracting two result images for eliminating false-positive components further. In the third filtering step, the characteristics of each component such as the ratio of the number of pixels in each candidate component to the number of its boundary pixels and the ratio of the minor to the major axis of each bounding box are used. Acceptable results have been obtained using the proposed method on 300 news images with a recognition rate of 93.6%. Also, our method indicates a good performance on all the various kinds of images by adjusting the size of the structuring element.

In this paper, we present a novel technique for classification of face images that employs Curvature-based Multi-scale Morphology (CMM). Multi-scale Morphology is an image analysis technique that employs mathematical morphology with structuring elements whose spatial dimensions are scaled successively. This scale-space representation of images has proven to be an efficient technique for indexing a large database of images. A majority of the existing techniques for multi-scale morphology employ regular and symmetrical structuring elements like cylinders, hemispheres or circular poweroids. The shape of these structuring elements is controlled only by the scaling parameter. In this paper, we propose the use of a structuring element whose shape is a function of both the scaling factor and the principal curvatures of the intensity surface of the face image. A high-dimensional feature vector is obtained by applying the CMM technique to the face images. The dimensionality of the feature vector is reduced by using the PCA technique, and the low- dimensional feature vectors are analyzed using an Enhanced FLD Model (EFM) for superior classification performance. Experimental results have shown that the proposed CMM technique outperforms existing approaches based on multi- scale morphology.

We develop a method for automatic segmentation of natural video sequences. The method is based on low-level spatial and temporal analyses. It features three designs to help facilitate good region segmentation while keeping the computational complexity at a reasonable level. Firstly, a preliminary seed-area identification and a final re-segmentation process are performed on each video frame to help region tracking. Secondly, a simple way to measure homogeneity of texture in a region is devised and the segmentation tries to locate object boundaries at where the texture shows significant changes. And thirdly, a reduced-complexity motion estimation technique is used, so that dense motion fields can be computed at a reasonable complexity. The overall method is organized into four tasks, namely, seed-area identification (for each frame), initial segmentation (only for the first frame in the sequence), motion-based segmentation (for all later frames), and region tracking and updating (also for all later frames). Some examples are provided to illustrate the performance of this method.

Background registration technique is useful to solve still object problem and uncovered background problem for video segmentation. However, it is hard to automatically decide the threshold of the frame difference for background registration to make it more feasible for real-time applications. Many previous works made efforts on automatic threshold decision for change detection. In this paper, we propose a new method of automatic threshold decision algorithm in a totally different viewpoint. Not only change detection but also the quantization effect in discrete domain is concerned. A Gaussianity test is first applied to find the standard deviation of Gaussian noise from the camera. Then, the quantization effect in discrete domain is taken into consideration to derive the relation between the standard deviation and the optimal threshold value. A couple MPEG-4 sequences and experimental sequences are tested as examples. Simulation results show that the calculated threshold values are suitable for background registration to give good segmentation results.