Optimal complexity scalable H.264/AVC video decoding scheme for portable multimedia devices

Abstract. Limited computing resources in portable multimedia devices are an obstacle in real-time video decoding of high resolution and/or high quality video contents. Ordinary H.264/AVC video decoders cannot decode video contents that exceed the limits set by their processing resources. However, in many real applications especially on portable devices, a simplified decoding with some acceptable degradation may be desirable instead of just refusing to decode such contents. For this purpose, a complexity-scalable H.264/AVC video decoding scheme is investigated in this paper. First, several simplified methods of decoding tools that have different characteristics are investigated to reduce decoding complexity and consequential degradation of reconstructed video. Then a complexity scalable H.264/AVC decoding scheme is designed by selectively combining effective simplified methods to achieve the minimum degradation. Experimental results with the H.264/AVC main profile bitstream show that its decoding complexity can be scalably controlled, and reduced by up to 44% without subjective quality loss.


Introduction
The proliferation of ubiquitous communication infrastructures makes various video services increasingly popular on portable multimedia devices.Although recent technological advancements have made real-time video playback possible on many portable multimedia devices, when it comes to high resolution and/or high quality video content, especially on mobile devices, there are still impending issues in realtime playback due to limited resources of portable devices in battery capacity, processor speed, and memory.
The problem addressed in this paper is slightly different from the conventional usage scenario of video decoders since it addresses what to do if an H.264/AVC video decoder is given a compressed bitstream exceeding its level specification.The level in a video coding standard specifies the minimal resources with which a given standard-conformant decoder shall be equipped.Encountering video contents exceeding a decoder's level specification is not rare in reality due to the recent ubiquity of various communication networks and the coexistence of portable devices with wide variability in their computation capacity.That is because video content available on mobile networks is accessible basically to every sort of devices, and some content may have a higher level specification of H.264/AVC than that of a receiving decoder.In such cases, currently a conventional decoder just refuses to decode.However, it would be much friendlier to users if the decoder were able to show the decoded pictures at a slightly lower quality that its available resources could provide with the best efforts.In order to do this, a video decoder should be able to flexibly decode the bitstream exceeding the level specification according to its available computing resources.This kind of simplified decoding capability is also quite essential when a decoder knows a priori that fully compliant decoding is not necessary, e.g., in fast-forwarding of video, making thumbnails, or skimming through a video playback.As a whole, the issue in this problem is complexity scalable video decoding in accordance with the available resources in a decoder.
The aforementioned video playback capability itself is already implementable in a sense if the play-back quality is not concerned much.However, such a less careful approach would be practically useless due to the significant degradation of both objective and subjective quality resulting from the processing mismatch between the encoder and the decoder.A quality distortion caused by a complexityreduced decoding process is propagated to subsequent pictures, and pretty soon, the quality degradation will become unbearable.Therefore, it is very important to carefully design a complexity scalable decoding algorithm which can manage optimal complexity control depending on the resource availability of the device.
][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19] A major approach to complexity scalable decoding is to control the computational complexity of one or two decoding processes.Peng 1 proposed a discrete cosine transform (DCT)-based complexity scalable video decoder which controlled the decoding complexity by pruning out some DCT data in order to skip the inverse discrete cosine transform (IDCT) process.Peng and Zhong 2 proposed a selective B-residual decoding method based on computational resources and the energy level of B-residual blocks.Chen et al. 3 realized the complexity scalability by using both IDCT pruning and a simpler interpolation filter based on the frame type.Lei et al. 4 proposed a complexity scalable algorithm for the audio and video coding standard in China (AVS) by using a loop filter along with a luminance interpolation filter scaling method.In this approach, the encoder sends complexity information about the loop filter and interpolation filter to the decoder for the complexity control.Meanwhile, Ji et al. 5 developed an energy scalable video decoding strategy for multimedia devices.Lee et al. 6 also worked towards complexity scalability by controlling the complexity of motion compensation and of the deblocking filter.Mahjoub et al. 7 proposed a complexity reduction method of the deblocking filter, and Lu et al. 8 optimized the context adaptive variable length coding (CAVLC) lookup tables to reduce the decoding complexity.de Oliveira et al. 9 optimized the inverse transform (IT) matrix for each frame, as a function of both content and quantization noise.
][12][13] They include complexity models of H.264/AVC decoding parts such as motion compensation, 10,11 entropy decoding, 12 or the whole H.264/AVC decoder. 13s another approach to the complexity scalable algorithms, Park et al. 14 reduced the energy consumption of the MPEG-4 decoding process by using a re-quantization process that reduced the amount of data to process.Nam et al. 15 proposed a method using spatial downsizing decoding.
7][18][19] Chao et al. 16 designed an optimized IT architecture to support multistandard video coding applications, and Wei et al. 17 proposed parallel decoding algorithms for multicore processors.Tsai et al. 18 designed a parallel level decoding method of CAVLC, and Sze and Chandrakasan 19 proposed a parallel context adaptive binary arithmetic coding (CABAC) decoding method.However, these hardware perspective approaches are not easy for achieving flexible control because of their hardwired characteristics.
In this paper, an algorithmic complexity scalable video decoding scheme for the H.264/AVC decoder of portable multimedia devices is investigated.First, each decoding element of H.264/AVC was studied in terms of both its complexity versus degradation performance from the viewpoint of complexity scalability and its complexity control parameters.By adjusting the complexity control parameters in a complexity-distortion (C-D) optimized way, the optimum complexity control levels which give the best performance in complexity reduction with minimal quality degradation were found.
The rest of this paper is structured as follows.Section 2 investigates the video decoding elements from the viewpoint of complexity control and develops a complexity reduction method.Section 3 presents the proposed complexity scalable decoding method of each control element and the optimal complexity control level.Section 4 presents experimental results, and Sec. 5 concludes with discussion and some directions for possible future work.

Complexity Control of Video Decoding Elements
The H.264/AVC decoder is composed of several decoding elements as depicted in Fig. 1: a variable length decoder (VLD), an inverse quantizer (IQ), an IT, motion compensation, intra prediction, reconstruction, and deblocking filter.In previous research, 6 complexity of the H.264/AVC decoding elements in the main profile was evaluated, and its complexity profiling result 6 is summarized in Table 1 to show that the most complex decoding element is the motion compensation and variable length decoding (CABAC decoding), and that the second major complex element is the deblocking filter."Others" in Table 1 represent operating system overheads such as file I/O for reading bitstreams from the file system. 20ased on this profiling result, the motion compensation and deblocking filtering were chosen in this paper as targets for complexity control.
However, by noting the lossless nature of the entropy decoder, simplification of CABAC is not considered since even small degradation from compromised entropy decoding can result in fatal decoding errors.Since the complexity control range using the motion compensation and deblocking filter were not sufficient, a macroblock (MB) decoding skipping method was further developed to provide more flexibility in the complexity control.

Complexity Reduction of the Motion Compensation
In the H.264/AVC motion compensation, motion predicted values at half-pel samples were generated by horizontal or vertical one-dimensional 6-tap finite impulse response (FIR) interpolation filtering, while quarter-pel samples were generated by averaging the nearest half-pel and integer-pel samples. 21Figure 2 depicts quarter-pel samples for luma components.Since the interpolation process accounts for most of the complexity of motion compensation, its complexity for the luma component is modeled as where C mc luma is the total computational complexity of the motion compensation in a decoder; C Int pel , C Half pel , and C Quarter pel are the computational complexity for generating integer-pel samples (i.e., for sample copy from its reference picture), 6-tap FIR interpolation filtering to generate half-pel samples, and an additional averaging process of quarter-pel samples after generating half-pel samples, respectively.Table 2 shows the interpolation filtering process based on the quarter-pel sample positions in Fig. 2 and their normalized complexity with respect to C Int pel . 6Note the nonidentical computational complexity: it depends on the sample positions.Quarter-pel positions at f, i, k, q are the most complex since they require seven 6-tap filtering and one 2-tap filtering.On the other hand, those half-pel samples labeled b, h are the least complex onesthey require just one time of 6-tap filtering.This observation suggests that the motion compensation complexity depends critically on the complexity of the interpolation filter.Therefore, to reduce the complexity, the 6-tap interpolation filter can be replaced by 2-tap or 4-tap filter, depending on the subpel position, with filter coefficients as shown in Table 3.Note that the filter coefficients of the 4-tap filter are used in a scalable extension of H.264/AVC 22 for inter-layer intra prediction, and those of the 2-tap filter are generated by simplifying the H.264/AVC interpolation filter with adjacent int-pel samples. 6rediction values for chroma sample positions are generated by bilinear interpolation of four neighboring integer samples using (2)     Bs is determined as one integer between 0 and 4 according to the rules 23 based on whether or not it is a MB boundary, whether its blocks have intra/inter prediction mode, whether it has nonzero DCT coefficients, its motion vector, its reference picture index, etc. Subsequently, the actual filtering process is applied to each 4 × 4 block boundary depending on the selected value of Bs.In case of Bs ¼ 4, a special filter is applied according to the specific condition. 23When Bs is from 1 to 3, a normal filter is applied. 23Therefore, the computational complexity of the deblocking filter is modeled as where C deblocking filter is the computational complexity of the deblocking filtering, C Bs decision is the computational complexity of the Bs decision process, and C filtering is the computational complexity of the actual pixel filtering process.
The complexity reduction of deblocking can be designed in two waysby reducing C Bs decision or C filtering .
In order to reduce C Bs decision , a simplified Bs decision process in the previous research 6 was proposed based on some observations. 6,24The proposed simplified Bs decision process contained the same Bs decision rules as the previous research, 6 but different Bs values according to the rules as depicted in Fig. 3.
A slightly different filtering method compared to the previous research 6 was also designed by adjusting filtering tap size and the number of samples to be filtered according to Bs to reduce C filtering .When Bs ≥ 3, the proposed simplified filtering method is the same as that of H.264/AVC.On the other hand, if Bs ¼ 2, only one nearest pixel from each side of the block boundary (i.e., p 0 and q 0 ) is filtered 23 that is, unlike the H.264/AVC, the second immediate pixels from the block boundaries (p 1 and q 1 ) are not filtered. 23In the Bs ¼ 1 case, the filtering complexity is reduced by using a 2-tap FIR filter which is applied only to the p 0 and q 0 samples.

Complexity Reduction by MB Decoding Skipping
For further complexity scalability, a MB was skipped from the whole decoding process after executing VLD.6][27][28] Reduction in temporal resolution due to the frame skipping may cause noticeably perceivable motion jerkiness and consequently significant subjective quality loss can follow.To prevent such mishaps, existing approaches selectively skip frames that satisfy some conditions, for examples, scene change, 25 motion activity, 26,27 or motion continuity. 28Such skipping methods were also used for the proposed complexity scalable video decoder.That is, the MB decoding in a B slice was skipped when the MB satisfied three conditions of MB coding type, coded block pattern (cbp) value, and motion activity.If an MB (in inter-coded slice or picture) was determined to be intracoded, its correlation with the blocks in reference picture (s) must be low.Therefore, an intra-coded MB should not be skipped from the decoding since otherwise its pixel values cannot be faithfully reproduced.On the other hand, if an inter-coded MB has no nonzero coded coefficients (that is, cbp ¼ 0), then it was safe to assume that the MB was highly correlated to its reference block, and it was possible to estimate the MB quite faithfully from its reference.Motion activity can indicate whether the motion of the current MB is fast or slow.If an MB with fast motion is skipped from decoding, its estimated reconstruction is highly likely to have noticeable motion jerkiness.Therefore, the decoding skipping also needs to check the motion activity of the current MB which can be calculated by mv MA ¼ 1 16 where mv MA is the motion activity of current MB, and ðmv n x ; mv n y Þ is the motion vector of the nth 4 × 4 block (n ¼ 0 ∼ 15) inside a current MB.In the proposed method, an inter-coded MB having cbp ¼ 0 and motion activity (mv MA ) less than a threshold T MA was skipped from further decoding after VLD.In the picture play-out, the skipped MB was generated by a very simple reconstruction method whose complexity was much lower than the actual decoding process.0][31] Those skipped MBs were reconstructed by motion compensation using motion vectors and the reference index of the current MB.For example, if the current MB was predicted to be from list 0, its reconstructed MB was motioncompensated from the reference slice in the list 0 memory.If the current MB was bi-predicted, the reconstructed signal was formed by averaging the motion-compensated signals from the list 0 and list 1 memory.

Proposed Complexity Scalable Decoding
Scheme The complexity scalable video decoding scheme should satisfy the following optimality criterion max Cðx 1 ; x 2 ; : : : ; x n Þ subject to min Dðx 1 ; x 2 ; : : : ; x n Þ; (5) where Cðx 1 ; x 2 ; : : : ; x n Þ and Dðx 1 ; x 2 ; : : : ; x n Þ denote, respectively the complexity reduction and consequent quality loss when the decoding complexity is controlled by the complexity control parameters x 1 ; x 2 ; : : : ; x n .By controlling the complexity control parameters, users can control the amount of complexity reduction.Under the constraints on available computing resources, the proposed complexity scalable video decoder can achieve the minimum quality loss while maximizing complexity reduction by controlling the selected control parameters.To find an optimal control level of those parameters, the C-D performance with selected complexity control parameters is evaluated first.Next three decoding complexity control parameters are discussed: motion compensation, the deblocking filter, and the MB decoding process skipping method.

Complexity Scalable Method for the Motion Compensation
To reduce the complexity of motion compensation, simplified interpolation filtering methods were developed as depicted in Sec. 2. The most effective complexity scalability for motion compensation can be achieved by selectively using the simplified interpolation filtering methods according to a specified scalability level.The proposed four motion compensation complexity reduction levels (MCR Level ) are shown in Table 4.When MCR Level ¼ 0, the motion compensation had the maximum complexity, which is the same as the conventional H.264/AVC motion compensation.When MCR Level ¼ 1 or 2, simplified methods were applied to the B slice only.This gives the minimal degradation on the video quality since the degradation in the B-slice is not propagated to other slices unless the stored B-slice is used.When MCR Level ¼ 3, simplified motion compensation methods were used for the P-slices (4-tap filter) and the B-slices (2-tap filter).Therefore, encoder-decoder mismatch could occur, however, in return, the decoder achieved a significant reduction in complexity.For chroma components, when MCR Level > 0, the simplified method from Sec. 2.1 for chroma was applied.

Complexity Scalable Method for the Deblocking Filter
To control the complexity of the deblocking filter, a simplified Bs decision and deblocking method were applied as shown in Table 5, based on the six deblocking filter complexity reduction levels (DFR Level ).When DFR Level ¼ 0, the conventional H.264/AVC deblocking filter was used without any complexity reduction.As the DFR Level increased, simplified methods of deblocking in Sec. 2 were applied one by one.
When DFR Level ¼ 4, the deblocking filtering was applied only to the MB boundary with the simplified Bs decision and filtering method in Sec. 2. To achieve the maximum complexity reduction in the deblocking filter, the deblocking  filter process to whole slices was switched off when DFR Level ¼ 5.

Complexity Scalable Method for MB Decoding Skipping
The MB decoding process skipping has three levels which are described in terms of their MB decoding reduction level (MDR Level ), as in Table 6.If MDR Level ¼ 1, those MBs satisfying the skip conditions (inter-coded MB, cbp ¼ 0, mv MA < T MA ) were not decoded, that is, were skipped from decoding.On the other hand, if MDR Level ¼ 2, the intercoded MBs were skipped from the decoding process, thus saving a tremendous amount of computation.

Proposed Complexity Scalable Decoding Scheme
The previous subsections discussed how to realize decoding complexity scalability individually for each key decoding element.In this subsection, the same problem is discussed but focusing specifically on the best scalable control of total video decoding by optimally adjusting the three control parameters MCR Level , DFR Level , and MDR Level together.
A block diagram of the proposed complexity scalable video decoder is shown in Fig. 4. Let us define a total complexity reduction level (TCR Level ) as a function of the three levels, where fð•Þ is a complexity control function with all three individual control levels as input.Note that each level has different effectiveness in complexity control and that consequentially the degradation varies in quality.To find the optimal combination of the control parameters to maximize the total complexity reduction subject to a constraint on the minimum quality degradation, the C-D performance 6,32 was compared by using AST and ΔPSNR which are respectively defined as ASTð%Þ ¼ DTðAnchorÞ − DTðproposedÞ DTðAnchorÞ × 100; (7) where ASTð%Þ is an average saving time and DTð•Þ is a total decoding time of a decoder.ΔPSNR [dB]is the difference in objective quality as measured in the peak signal-to-noise ratio (PSNR) between the proposed method and the anchor.
Here the "anchor" represents the conventional H.264/AVC decoder which corresponds to the maximum complexity (i.e., TCR Level ¼ MCR Level ¼ DFR Level ¼ MDR Level ¼ 0), and the "proposed" represents the complexity-reduced decoding scheme according to the selected control level.The C-D curves are shown in Fig. 5; each curve represents the complexity reduction versus quality degradation obtained by controlling the individual level parameter independently (MCR Level in Table 4, DFR Level in Table 5, and MDR Level in Table 6, respectively).By controlling each parameter, the decoder complexity is reduced by up to 13.6% by MCR Level , 12.9% by DFR Level , and 25.9% by MDR Level .The quality degradations compared to the maximum complexity reduction are −1.66,−1.84, and −3.19 dB, respectively.As in Fig. 5, it was verified that these three parameters were suitable for complexity control of a decoder.
The joint C-D curve in terms of all three control parameters together (MCR Level , DFR Level , and MDR Level ) is shown in Fig. 6.The optimal complexity control points which have the maximum complexity reduction subject to minimum distortion are specified by a line in Fig. 6.Therefore, if a decoder adjusts the control parameters following the line, the optimal decoding complexity scalability can be attained.Table 7 shows the control level of each parameter according to TCR Level , while its relative complexity reduction and quality degradation are shown in Fig. 6.As shown in Table 7, the expected maximum complexity reduction was up to 41% compared with TCR Level ¼ 0. Since the choice of TCR Level ¼ 15 in Table 7, compared to TCR Level ¼ 14, reduced the complexity by not more than 1% but incurred additional quality degradation of about 0.6 dB, the maximum complexity reduction level was set to 14 instead of 15.Therefore, the complexity of a decoder was adjusted in up to 15 steps.When TCR Level was 0, the decoder has no complexity reduction, that is, the same complexity as conventional H.264/AVC decoder.As the TCR Level increased, the complexity control level of each parameter was adjusted according to Table 7.

Experimental Results
In order to assess the performance of the proposed scheme, it was implemented on JM18.0 H.264/AVC reference software.Bitstreams for experiments were coded for the H.264/AVC main profile with group of pictures (GOP) size ¼ 60 under IBPBP structure (Here, I, B, and P represents Intra, Bi-predictive, and Predictive picture, respectively).The number of reference frames was 5, and one picture was coded as one slice.The quantization parameter (QP) was set to 22, 27, 32, and 37.The video sequences used for performance evaluation were Bigships, City_corr, Night, and Crew (1280 × 720@60 fps).The performance of the proposed scheme was measured by AST (%) in Eq. ( 7) and ΔPSNR [dB] in Eq. ( 8).
Figure 7 shows how the subjective quality changes when each control parameter of complexity (MCR Level , DFR Level , and MDR Level ) is individually adjusted.Figure 7(a) corresponds to the maximum complexity level (TCR Level ¼ 0, i.e., MCR Level ¼ 0, DFR Level ¼ 0, and MDR Level ¼ 0) while the others, Fig. 7 7 shows that while the maximum complexity reduction of deblocking filtering (DFR Level ¼ 5) in Fig. 7(c) gives similar subjective quality as the no complexity reduction case in Fig. 7(a), the sharpness of the picture is degraded when MCR Level or MDR Level are adjusted to those of the minimum complexity level (MCR Level ¼ 3, MDR Level ¼ 2).The sharpness degradation appears in all regions in Fig. 7(b), but it is less apparent in the low motion area in Fig. 7(d) (see the middle building in the picture).As depicted in Fig. 7, although control parameters had some degradation in the minimum complexity level, they still maintained proper overall subjective quality.
Table 8 shows experimental results of the proposed scheme, indicating that it attains complexity scalability.The proposed scheme reduces the decoding complexity by up to 44% (in City_corr sequence, TCR Level ¼ 14) as compared   with the conventional H.264/AVC decoder.In Table 9, C target represents the target complexity reduction and TCR Level is the complexity control level to achieve the complexity reduction up to C target .Reduced complexity for the TCR Level is similar to the C target in most cases.However, the reduced complexity does not reach the target complexity C target in the Crew sequence.This is because the proposed scheme does not consider an intra-coded MB as a candidate for complexity control, therefore, there may exist a problem of insufficiency in the adjustable range of complexity especially if there are many intra MBs.The many intra-coded MBs (boxed areas) in the Crew sequence are shown in Fig. 8 in an inter slice.This behavior is due to flash lights).This is one area for future extension of the proposed method.Table 9 shows the distortion and complexity reduction performance of the proposed method compared to the    previous research. 4,6For a fair comparison, previous research 4,6 was implemented on the JM18.0H.264/AVC reference software.In Table 9, methods 1 and 2, respectively represent methods of research from Refs. 4 and 6.The reduced complexity according to the C target and the relative distortion compared with the conventional H.264/AVC decoder were measured.Since method 2 reduces the complexity by up to 25% and method 1 exploits the B picture decoding skip method to reduce the complexity by more than 30%, C target was only considered for 10% and 20% in this experiment.Compared to methods 1 and 2, the proposed method is better in objective quality loss.Since those methods reduce the complexity of the interpolation filtering process regardless of picture typethat is, through the P picture or stored-B picture which are used as references of following pictureserror will be propagated to the following pictures and the objective quality loss was much higher than the proposed method.Complexity control of the proposed method was also more accurate than methods 1 and 2, and the reduced complexity of proposed method was more similar to C target than these two methods.Figure 9 shows a subjective quality comparison according to the TCR Level represented in Table 8 from the 118th (the last decoded frame in one GOP) frame in City_corr sequence coded with QP 22 (high quality).Even if the TCR Level increases, the decoded pictures still have acceptable subjective quality, although the objective quality degradation becomes larger.
To verify the proposed scheme on mobile devices, it was also implemented on a mobile device.Figure 10 shows subjective quality with the maximum complexity (TCR Level ¼ 0) and the minimum complexity (TCR Level ¼ 14) on a mobile device.Since the mobile device is not able to show the original resolution of the video sequence, it only displays a downsized picture after performing a downsizing process.As shown in Fig. 10, the subjective quality of the decoded picture compared to the one of maximum complexity was acceptable even with the minimum complexity.

Conclusion
This paper presented a complexity scalable H.264/AVC decoding scheme for portable multimedia devices.The proposed method controls the motion compensation, deblocking filtering, and MB decoding skipping process by adjusting these three complexity control parameters.Its C-D performance was evaluated according to the controlling parameters, and the optimal complexity control levels for each parameter were sought.The proposed scheme can control the decoding complexity with variable complexity control levels without significant subjective quality loss.Since the current scheme can adjust the decoding complexity of inter MBs only, future work may extend it to include also the decoding complexity controlling capability of intra MBs.

Fig. 2
Fig. 2 Fractional sample positions for quarter sample luma interpolation.
scheme was measured by AST (%) in Eq.(7) and ΔPSNR [dB] in Eq.(8).Figure7shows how the subjective quality changes when each control parameter of complexity (MCR Level , DFR Level , and MDR Level ) is individually adjusted.Figure7(a) corresponds to the maximum complexity level (TCR Level ¼ 0, i.e., MCR Level ¼ 0, DFR Level ¼ 0, and MDR Level ¼ 0) while the others, Fig.7(b)-7(d), respectively, correspond to when each control parameter is changed to its maximum reduction value individually [Fig.7(b): MCR Level ¼ 3; Fig. 7(c): DFR Level ¼ 5; Fig. 7(d): MDR Level ¼ 2].Figure7shows that while the maximum complexity reduction of deblocking filtering (DFR Level ¼ 5) in Fig.7(c) gives similar subjective quality as the no complexity reduction case in Fig.7(a), the sharpness of the picture is degraded when MCR Level or MDR Level are adjusted to those of the minimum complexity level (MCR Level ¼ 3, MDR Level ¼ 2).The sharpness degradation appears in all regions in Fig.7(b), but it is less apparent in the low motion area in Fig.7(d)(see the middle building in the picture).As depicted in Fig.7, although control parameters had some degradation in the minimum complexity level, they still maintained proper overall subjective quality.Table8shows experimental results of the proposed scheme, indicating that it attains complexity scalability.The proposed scheme reduces the decoding complexity by up to 44% (in City_corr sequence, TCR Level ¼ 14) as compared

Figure
scheme was measured by AST (%) in Eq.(7) and ΔPSNR [dB] in Eq.(8).Figure7shows how the subjective quality changes when each control parameter of complexity (MCR Level , DFR Level , and MDR Level ) is individually adjusted.Figure7(a) corresponds to the maximum complexity level (TCR Level ¼ 0, i.e., MCR Level ¼ 0, DFR Level ¼ 0, and MDR Level ¼ 0) while the others, Fig.7(b)-7(d), respectively, correspond to when each control parameter is changed to its maximum reduction value individually [Fig.7(b): MCR Level ¼ 3; Fig. 7(c): DFR Level ¼ 5; Fig. 7(d): MDR Level ¼ 2].Figure7shows that while the maximum complexity reduction of deblocking filtering (DFR Level ¼ 5) in Fig.7(c) gives similar subjective quality as the no complexity reduction case in Fig.7(a), the sharpness of the picture is degraded when MCR Level or MDR Level are adjusted to those of the minimum complexity level (MCR Level ¼ 3, MDR Level ¼ 2).The sharpness degradation appears in all regions in Fig.7(b), but it is less apparent in the low motion area in Fig.7(d)(see the middle building in the picture).As depicted in Fig.7, although control parameters had some degradation in the minimum complexity level, they still maintained proper overall subjective quality.Table8shows experimental results of the proposed scheme, indicating that it attains complexity scalability.The proposed scheme reduces the decoding complexity by up to 44% (in City_corr sequence, TCR Level ¼ 14) as compared

Fig. 5
Fig. 5 Complexity-distortion curve of each individual control level.

Table 2
Complexity comparison of interpolation in H.264/AVC decoding.

Table 3
Complexity reduction method for interpolation filtering.are the integer-pel samples.xFrac c and yFrac c are the fractional offsets of the predicted sample.To reduce the complexity for chroma interpolation filtering, the predicted chroma sample is just copied from the nearest neighboring integer samples.

Table 6
Complexity reduction (MDR) levels for macroblock decoding processing skipping.
Fig. 4 Block diagram of the proposed complexity scalable H.264/AVC decoder.

Table 8
Experimental results of the proposed scheme.

Table 9
Performance comparison of the proposed scheme with previous methods (Refs.4,6).