Open Access
6 February 2014 Algorithm for JPEG artifact reduction via local edge regeneration
Author Affiliations +
Abstract
Transform coding using the discrete cosine transform is one of the most popular techniques for image and video compression. However, at low bit rates, the coded images suffer from severe visual distortions. An innovative approach is proposed that deals with artifacts in JPEG compressed images. Our algorithm addresses all three types of artifacts which are prevalent in JPEG images: blocking, and for edges blurring, and aliasing. We enhance the quality of the image via two stages. First, we remove blocking artifacts via boundary smoothing and guided filtering. Then, we reduce blurring and aliasing around the edges via a local edge-regeneration stage. We compared the proposed algorithm with other modern JPEG artifact-removal algorithms. The results demonstrate that the proposed approach is competitive, and can in many cases outperform, competing algorithms.

1.

Introduction

Image and video compression continue to be in high demand. The block discrete cosine transform1 is the most popular and widely used method in image/video compression standards, including JPEG2 for images, MPEG3 for videos, and H.2614 for videophone/teleconference applications, because of its compaction property and ease of implementation. For JPEG, one widely known shortcoming is that at low bit rates, the compression can leave discontinuities of intensities between adjacent blocks (known as blocking artifacts). JPEG can also lead to other visual artifacts such as degraded textures, blurring, and distortion of edges. In general, decreasing the bit rate will increase the severity and prevalence of these visual artifacts.

Over the past several decades, numerous algorithms have been proposed to improve the visual quality of JPEG images by attempting to remove the artifacts. Two approaches are generally adopted: encoder-based methods and postprocessing-based methods. The encoder-based approaches work by making changes to the encoder, such as transform-domain methods,57 interleaved block transform,8 interactive methods,9 lapped transform,10 combined transform,11 or wavelet-based filtering.12 However, the drawback of this approach is that the resulting compression algorithms no longer conform to the JPEG standard.

Postprocessing attempts to improve the visual quality by removing artifacts via processing of the image after decoding. This approach does not require any modifications to the encoder or decoder, and can thus be used on existing JPEG images. Accordingly, most artifact-reduction algorithms follow this latter approach. Postprocessing can roughly be divided into spatial-domain techniques,1320 DCT-domain techniques,2126 projections onto convex sets (POCS),2731 and block-shift filtering.3241

The spatial-domain techniques process the compressed image based on some prior knowledge and information about the original image, such as intensity smoothness or block boundaries of images, to improve the image quality. For instance, in Ref. 13, Reeve and Lim proposed a symmetric two-dimensional (2-D) Gaussian spatial filtering method to reduce the blocking artifacts. Other different methods of spatial-domain techniques, such as those based on gradients/thresholds and the histogram-based methods used in Refs. 1520, classify the blocks as either high frequency or low frequency, and use filtering methods to remove artifacts and thus improve image quality.

In DCT-domain postprocessing algorithms, blocking artifacts are reduced by directly manipulating DCT coefficients. For example, in Ref. 23, Zeng proposed a DCT-domain method for blocking reduction by applying a zero-masking to the DCT coefficients of some shifted image blocks. By using the fact that visible boundaries between two adjacent blocks in the coded image are primarily oriented along the horizontal and vertical directions, Zeng generated a new data block which is modeled as a 2-D step function contaminated by an independently and identically distributed noise with zero mean and a small variance. Then, by performing a DCT on such blocks, Zeng reported that there always exist AC components of significant energy in a few fixed positions. Accordingly, the author zeroed out some of these AC components and demonstrated that doing so can make blocking artifacts much less visible. However, a loss of edge information caused by the zero-masking scheme can be noticed in his method. In Ref. 24, Jeon and Jeong proposed a postprocessing method to reduce discontinuities of pixel values over block boundaries by compensating for the loss of coefficients’ accuracies in the transform domain. They defined the block boundary discontinuity as the sum of the squared differences of pixel values along the block boundary. More recently, in Ref. 25, Chen et al. proposed an algorithm based on three filtering modes in terms of the activity across block boundaries. They considered the masking effect of the human visual system and integrated adaptive filtering into the deblocking process.

There are also some methods which use both spatial-domain and DCT-domain approaches. For instance, in Ref. 26, Singh et al. proposed an adaptive postfiltering algorithm to remove blocking artifacts. They classify the boundary regions between the blocks as smooth, nonsmooth, or intermediate regions. Then, blocking artifacts in the smooth and nonsmooth regions are removed by modifying selected DCT coefficients while an edge preserving smoothing filter is applied to the intermediate regions.

In addition, there are typical postprocessing iterative methods based on the theory of projection onto convex sets (POCS)27 and a maximum a posterior probability approach.28 POCS based on recovery algorithm is a DCT-domain filtering approach. The basic idea is to optimize the value of the quantized transform coefficients, subject to some smoothness and quantization constraints. In Ref. 31, Reeve and Lim introduced a method based on the theory of POCS and proposed a postprocessing technique to reduce blocking artifacts in decoded images. They assumed that the input image is highly correlated so that similar frequency characteristics are maintained between adjacent blocks. The major drawback of this approach is the high computational complexity.

Block-shift filtering is an adaptive filtering algorithm for reducing image artifacts (see e.g., Refs. 40 and 41). Some algorithms have been proposed which attempt to reduce blockiness by using a quad-tree (QT) decomposition and block-shift filtering. QT decomposition3235 is a multiresolution image segmentation technique which can partition an image into many homogeneous regions based on some predefined rules. The variable block sizes generated by the QT decomposition facilitate the later block-shift filtering with low computational cost. In Ref. 38, Luo and Ward proposed an adaptive approach which reduced blocking artifacts in both the spatial-domain. For smooth regions, this method took advantage of the fact that the original pixels in the same block provide continuity. In Refs. 40 and 41, Zhai et al. proposed algorithms to preserve the image’s details and reduce the effect of quantization noise. They integrated QT decomposition with the block-shift filtering.

Although previous algorithms can effectively suppress blockiness, JPEG images suffer from more than just blocking. At low bit rates, the compressed image also suffers from blurring and aliasing artifacts around the edges. In Refs. 12, 40, and 41, the authors have taken deblocking further, and shown attempts to compensate for degraded textures in the compressed image. However, this approach does not address blurring and aliasing around the edges. Together, all of these artifacts degrade the quality of JPEG images, and there is no single algorithm that can remove all of these artifacts significantly.

In this paper, we propose a technique to improve the visual quality of JPEG images via a two-stage approach. Our algorithm removes blocking artifacts by using existing techniques,38,42 and it reduces blurring and aliasing artifacts around the edges by means of local edge regeneration. First, we remove blocking artifacts via boundary smoothing38 and Guided filtering.42 Then, we reduce blurring and aliasing around the edges via a local edge-regeneration stage. The main contribution of this work is a technique to enhance the quality of JPEG images not only by removing blocking artifacts, but also by reducing blurring and aliasing artifacts around the edges and via edge regeneration.

This paper is organized as follows: In Sec. 2, we provide details of the two stages of the proposed algorithm. In Sec. 3, we present and discuss the results of the proposed algorithm. General conclusions are presented in Sec. 4.

2.

Algorithm

The flow chart of our proposed algorithm is shown in Fig. 1. In our algorithm, given a JPEG image as the input, two stages are used to enhance the quality of the image. The first stage is designed to remove blocking artifacts. The second stage is designed to remove blurring and aliasing artifacts around the edges in an attempt to make the edges appear sharper.

Fig. 1

Flow chart of our proposed algorithm.

JEI_23_1_013018_f001.png

2.1.

Deblocking

A very well-known problem with JPEG images is blocking artifacts. This stage serves to remove blocking artifacts by smoothing the boundaries of blocks and by using guided filtering.42 The flow chart of this stage is shown in Fig. 2. Although the technique employed for this deblocking stage is a combination of existing tools, as we will demonstrate later it is quite effective.

Fig. 2

Flow chart of the deblocking stage.

JEI_23_1_013018_f002.png

At low bit rates, there will be discontinuities between block boundaries due to independent quantization. We first reduce discontinuities between the neighboring blocks by using the method presented in Ref. 38 for smooth areas.

As explained in Ref. 38, their technique for smooth areas first identifies pairs of neighboring blocks whose shared boundary is not due to a genuine change in the intensities at that position. This condition is met if: (1) the two blocks share similar horizontal/vertical frequency properties and (2) a third block centered on the boundary is otherwise relatively smooth. After identifying such blocks, the block-to-block discontinuities are suppressed via DCT-based filtering, i.e., via strategic blending of the DCT coefficients of the two blocks and of the third block encompassing the boundary so as to remove blockiness but not introduce artifacts (for further details, see Ref. 38). By operating only on blocks which satisfy the above two criteria, the technique avoids modifying strong textures and edge regions, and thus attempts to preserve important information, true edges, and textures in the image.

After reducing the discontinuities between block boundaries, we apply a guided filter42 on the image to reduce the blocking artifacts. According to the authors in Ref. 42, the guided filter utilizes the structures in the image and has a fast linear-time algorithm. The guided filter preserves the image’s edges during deblocking. Unlike other edge-preserving filters (e.g., the bilateral filter), the guided filtering is computationally efficient [O(N), linear time], and it does not introduce gradient-reversal artifacts found via the bilateral filter.

If we apply the guided filter without smoothing the boundaries, the image may still have blocking artifacts around the boundaries of 8×8 blocks [see Fig. 5(b)]. We assist the guided filtering by explicitly toning-down the block boundaries in the first stage of our deblocking scheme. We have found this first stage to be a simple, yet effective technique of boosting the ability of the guided filtering to remove blocking artifacts.

The guided filter has two parameters: ϵ and α. The parameter ϵ is a regularization parameter and α specifies the local window radius. Increasing ϵ and α generally results in more smoothing. Decreasing these parameters generally results in images which might still contain visible blocking artifacts. Following the same technique used in Ref. 40, we select the ϵ parameter of the guided filter based on the JPEG quality factor. To determine this relationship between ϵ and quality factor (Q), we tested our algorithm on images from the LIVE database43 (29 original images). We compressed images using JPEG quality factors from 1 to 100 and recorded the ϵ that offered best structure similarity (SSIM)44 as the optimal ϵ for each image at that setting. The measured optimal ϵ values were then plotted against the corresponding Q values, and these data were then fitted with the following power function:

ϵ=0.0067Q0.78910.0003.

Note that Q is available at the decoder because the quantization table used in JPEG is a Q-scaled version of a predefined quantization table.

We also choose α empirically (α=4), which generally yields good results across a wide variety of images. However, the selection of this value is not critical, the results are very close when α is chosen within a ±20% range.

Figure 5, provided later in the paper, will show results of the deblocking stage [see Figs. 5(a)5(c)].

2.2.

Edge Regeneration

Besides blocking artifacts, it is well known that JPEG causes aliasing and blurring artifacts that occur around the edges within the image. In this second and final stages, we use local area information in an attempt to regenerate the edges and thus remove these edge artifacts. The flow chart of this stage is shown in Fig. 3. Let Z denotes the output result of the deblocking stage.

Fig. 3

Flow chart of edge-regeneration stage.

JEI_23_1_013018_f003.png

To improve the distorted edges, we use the fact that the blurring and aliasing artifacts are located around the edges; thus, further away from the edges, the quality improves. Our edge-regeneration algorithm operates by regenerating edge pixels as well as pixels that are located slightly farther away from the edges.

In this stage, we first use Canny edge detection45 to detect the strong edges [see Fig. 5(d)]. In order to compute the strong edges, we use the adaptive threshold parameters γT and βT for low threshold and high threshold of the Canny edge detection, respectively, where T indicates the threshold of the image computed by applying Otsu’s method46 to the gradient magnitude image used during Canny edge detection. Because we focus on improving the strong edges, so we chose γ=1.5 and β=2.5, and the kernel size parameter is set to 4 for the Canny edge detector. As we will demonstrate, these parameters generally yield excellent results across a variety of images and bit rates. However, the selection of these values is not critical. The results are very close when the values are chosen within a ±20% range.

Let denote an edge pixel value which is detected by Canny and is located at the positions i and j. We determine which area around the edges is more similar to the edges pixel by applying the following equation:

Eq. (1)

Z(i,j)dleft=|Z(i,j)Z(i,j1)+Z(i,j2)2|,dright=|Z(i,j)Z(i,j+1)+Z(i,j+2)2|.

In Eq. (1), we chose to compare the edge pixel against the average of the two pixel values to the left and right. In this equation, dleft represents the difference between the edge pixel Z(i,j) and the average of the two pixels horizontally to the left. Also, dright represents the difference between the edge pixel Z(i,j) and the average of the two pixels horizontally to the right. The smaller of the two measurements, dleft and dright, signifies the side which is more similar to the edge pixel. By computing dleft and dright, we measure the similarity of two sides of the edge pixel to the pixel on the edge.

After computing dleft and dright, if dleft<dright, this signifies that the left side of the edge pixel is more similar to the edge pixel than the right side. We would then use the side which is similar to the edge pixel to remake the edge pixel and its neighbors. Next, we use the pixels on the side which is different from the edge to reduce the artifacts around the edge.

To illustrate this process, Fig. 4(a) shows the 9×9 matrix centered around the edge pixel Z(i,j). This matrix was designed around the fact that edge artifacts reduce in pixels further away from the edge.

Fig. 4

Illustration of the edge regeneration process for a particular edge pixel Z(i,j). (a) If dright>dleft or dabove>dbelow. (b) If dright<dleft or dabove<dbelow.

JEI_23_1_013018_f004.png

In the following equation, we take the benefit of the area which is similar to the edge pixel to improve the edge pixel:

For (i,j)=(i+1,j1),(i,j1),(i+1,j) [see the green blocks in Fig. 4(a)]:

Eq. (2)

Z(i,j)=Z(i+1,j)+Z(i,j1)+Z(i+1,j1)+4Z(i,j)7.

For the edge pixel itself:

Eq. (3)

Z(i,j)=Z(i+1,j1)+Z(i,j1)+Z(i+1,j)+Z(i,j)4.

For (i,j)=(i,j3),(i,j2),(i+1,j3),(i+1,j2),(i+2,j2),(i+2,j1),(i+2,j),(i+3,j1),(i+3,j) [see the red blocks in Fig. 4(a)]:

Eq. (4)

Z(i,j)=Z(i+1,j)+Z(i,j1)+Z(i+1,j1)+4Z(i,j)7.

In Eqs. (2)–(4), the algorithm begins by improving the pixels near the center pixel. Equation (2) describes the regeneration of the pixels neighboring the center block, as identified by the green blocks in Fig. 4(a). In this equation, we use the average of three neighboring pixels along with the pixel itself to regenerate the selected pixel. The algorithm then uses these improved pixel values to enhance the center block. Specifically, Eq. (3) uses the average of regenerated pixels as well as the previous value of the edge pixel to regenerate the edge pixel. Finally, Eq. (4) describes how to enhance the quality of other pixels on the similar side, but slightly further away [red blocks in Fig. 4(a)].

The next step is to reduce aliasing and blurring artifacts on the other side. By using the following equation, we regenerate pixels which are slightly further away from the edge pixel to reduce the artifacts around the edges:

For (i,j)=(i3,j),(i2,j),(i2,j+1),(i1,j),(i1,j+1),(i1,j+2),(i,j+1),(i,j+2),(i,j+3) [see the gray blocks in Fig. 4(a)]:

Eq. (5)

Z(i,j)=Z(i1,j)+Z(i,j+1)+Z(i1,j+1)3.

For (i,j)=(i3,j+1),(i3,j+2),(i3,j+3),(i2,j+3),(i2,j+2),(i1,j+3) [see the brown blocks in Fig. 4(a)]:

Eq. (6)

Z(i,j)=Z(i1,j)+Z(i,j+1)+Z(i1,j+1)+4Z(i,j)7.

In Eqs. (5) and (6), by using the average of the pixels on the side which is different from the edge pixel, we regenerate pixels around the edge and reduce the aliasing and blurring to make the edge appear sharper. First, by using Eq. (5), we regenerate neighboring pixels [gray blocks in Fig. 4(a)]. Next, by using Eq. (6), we regenerate selected pixels [brown blocks in Fig. 4(a)] to make the area around the edge more smooth.

Equations (2)–(6) are applied to all edge pixels identified by the Canny edge detection algorithm. However, the algorithm will not regenerate an edge pixel until it is the center pixel in Fig. 4(a). For example, if the pixel directly below Z(i,j) in Fig. 4(a) is identified as an edge pixel, it will be used only for calculating the result of Eq. (2); however, Eq. (2) will not be applied to that pixel. Only Eq. (3) is used in regenerating pixels identified as edge pixels by the Canny edge detector algorithm.

In Eq. (1), after calculating dleft and dright, if dright<dleft, this signifies that the right side of the edge pixel is more similar to the edge pixel than the left side. In this condition, the equations are similar to Eqs. (2)–(6), except that i is replaced with j and j is replaced with i, resulting in a 180 deg rotation in the image plane of the regeneration pattern about Z(i,j) [see Fig. 4(b)]. We chose the basic structure for Eqs. (2)–(6) due to its relative simplicity and its ability to operate effectively on all edge orientations.

If dleft=dright, it means that the edge is horizontal. As a result, we use the following equation to compute the side which is similar to the edge pixel:

Eq. (7)

dabove=|Z(i,j)Z(i1,j)+Z(i2,j)2|,dbelow=|Z(i,j)Z(i+1,j)+Z(i+2,j)2|.

In Eq. (7), dabove represents the difference between the edge pixel Z(i,j) and the average of the two pixels vertically above. Also, ddown represents the difference between the edge pixel Z(i,j) and the average of the two pixels vertically below. If dabove>dbelow, this signifies that the region below the edge pixel is more similar to the edge pixel than the region above. Therefore, we use Eqs. (2)–(6) for the pixels which are shown in Fig. 4(a). On the other hand, if dabove<dbelow, it means that the region above is more similar to the edge pixel than the region below. In this case, we use Eqs. (2)–(6) after changing i with j and j with i.

Equations (1)–(6) are applied to all the edge pixels which are detected by the Canny edge detector. The edge-regeneration stage operates on edges of all orientations. As long as a pixel is an edge pixel (as detected by the Canny edge detector), then the edge regenerator is applied at that pixel. The context structure shown in Fig. 4 is meant to illustrate only which surrounding pixels are used during the regeneration. So the algorithm will operate on edges of all orientation, but which surrounding pixels it uses to perform the regeneration is limited to the two choices shown in Fig. 4. Note that if the surrounding pixels are also part of the edge, those pixels are not used to regenerate the current edge.

Figure 5(e) shows the result of this stage. Additionally, in Fig. 5(f), we provide close-up results for the edges around the shoulder, hat, and cheek of Lena to demonstrate that this stage can significantly improve strong edges regardless of the edge orientations. Specifically, as shown in Fig. 5(f), our proposed method improves the quality of edges without considering the orientations of the edges. A further close-up for part of Lena’s shoulder is shown in Fig. 6. Further results are provided in Sec. 3.

Fig. 5

Deblocking and edge regeneration results. (a) Input image PSNR=30.40. (b) Result of guided filter without reducing discontinuities between the neighboring blocks. (c) Result after deblocking algorithm [applying Eqs. (1)–(4) and guided filter]. (d) Edge map of Lena by using Canny edge detection on (c). (e) Image after proposed edge regeneration, PSNR=30.56dB. (f) Shoulder, hat, and cheek of Lena before (left side) and after (right side) applying proposed algorithm.

JEI_23_1_013018_f005.png

Fig. 6

Close-up results for part of Lena’s shoulder. (a) Before edge regeneration. (b) The result of Canny edge detection on (b). (c) The result after edge regeneration.

JEI_23_1_013018_f006.png

3.

Results

In this section, we evaluate the performance of our proposed algorithm and compare it to competing algorithms, both in terms of qualitative results and in terms of quantitative results. For these evaluations, we have used the 11 standard images shown in Fig. 7.

Fig. 7

Eleven 512×512 test images. For the first row images, from left to right: Lena, baboon, Barbara, peppers, plane, and bridge. For the second row images, from left to right: Zelda, redwood, actor, fruit, and Native American.

JEI_23_1_013018_f007.png

3.1.

Qualitative Results

We compared our proposed algorithm with Refs. 12, 38, 40, and 41. As mentioned in Sec. 2, all of these are modern, state-of-the-art postprocessing-based methods designed specifically for the use with JPEG images. Reference 12, by Liew and Yan, is a method based on a noniterative wavelet-based deblocking algorithm. Reference 38, by Luo and Ward, is an adaptive approach in both the spatial-domain and DCT-domain to reduce the block-to-block discontinuities. Reference 40, by Zhai et al., is a deblocking method for JPEG images via postfiltering in shifted windows of image blocks. Reference 41, by Zhai et al., employs a QT decomposition with block-shift filtering to reduce JPEG artifacts.

The visual results of our proposed algorithm on four images (Lena, peppers, Barbara, and baboon) are provided in Figs. 8Fig. 910 and are compared against the aforementioned algorithms. We chose Lena, Barbara, peppers, and baboon because the results of Refs. 12, 38, 40, and 41 on these images have been published in the respective papers. Note that the results for Ref. 12 were obtained from the authors, the results for Refs. 38 and 40 were generated by reimplementing their algorithms, and the results of Ref. 41 were obtained from their paper.

Fig. 8

Subjective quality comparison of Lena compressed at 0.24 bpp. (a) The result of Ref. 12, PSNR=30.99dB. (b) The result of Ref. 41, PSNR=31.23. (c) The result of our proposed algorithm, PSNR=30.56dB. (d), (e), and (f) a close-up of Lena for the results shown in (a), (b), and (c), respectively. (g), (h), (i) a close-up of Lena for the results shown in (d), (e), and (f), respectively.

JEI_23_1_013018_f008.png

Fig. 9

Close-up of Lena compressed at 0.24 bpp. (a) Result of Ref. 38, (b) result of Ref. 12, (c) result of Ref. 40, (d) result of Ref. 41, and (e) result of our proposed algorithm.

JEI_23_1_013018_f009.png

Fig. 10

Subjective quality comparison of the images Barbara, baboon, and peppers compressed at quality factor Q=10. (a), (b), and (c) JPEG compressed images, PSNR=26.33, 23.43, and 30.72 dB. (d), (e), and (f) the result of Ref. 12, PSNR=26.02, 23.53, and 30.91 dB. (g), (h), and (i) the result of Ref. 40, PSNR=26.03, 23.67, and 30.90 dB. (j), (k), and (l) the results of our proposed algorithm, PSNR=26.44, 23.68, and 31.28 dB. (m), (n), and (o) the two left images are close-up of the results from Refs. 40, 12 and right image is a close-up of the result from our proposed algorithm.

JEI_23_1_013018_f010.png

Figures 8(a) and 8(b) depict the results of Refs. 12 and 41 on Lena. For ease of examination, close-ups of Figs. 8(a) and 8(b) are provided in Figs. 8(d) and 8(e). The result of our proposed algorithm is shown in Fig. 8(c), and its close-up is provided in Fig. 8(f). Notice that whereas the results of Refs. 12 and 41 still exhibit aliasing and blurring (e.g., along the hat of Lena), these artifacts are better reduced by the proposed algorithm. Overall, the proposed algorithm has noticeably reduced the appearance of blocking and for edges blurring and aliasing. Figures 8(g), 8(h), and 8(i) show further close-ups of the results of the algorithms on the band in Lena’s hat. Notice that the results from the proposed algorithm [Fig. 8(i)] exhibits the least ringing and the sharpest edge.

A portion of the shoulder in Lena is shown in Fig. 9. We chose this portion because of the smooth regions on the skin of the shoulder, as well as the strong edges of the shoulder rim. Our proposed algorithm yields superior distortion suppression performance in this region in comparison to others.

In Fig. 10, we compare our results for Barbara, baboon, and peppers with original JPEG compression and also the results of Refs. 40 and 12. Figures 10(a), 10(b), and 10(c) show the JPEG results of Barbara, baboon, and peppers. In Figs. 10(d) and 10(g), although Refs. 40 and 12 reduced many blocking artifacts, some noticeable blocking still remains. The result of our proposed algorithm in Fig. 10(j) shows that our algorithm removed blocking artifacts as well as removed aliasing and blurring around the edges. Figures 10(e) and 10(h) show the results of Refs. 40 and 12 for baboon. In Fig. 10(k), the edges along the nose of the baboon look sharper and better in comparison with Figs. 10(h) and 10(b). Also, Fig. 10(k) has less blocking artifacts compared to Fig. 10(e). In Fig. 10(l), the edges of peppers do not have ringing artifacts as compared to Figs. 10(c), 10(f), and 10(i). Figures 10(m), 10(n), and 10(o) provide close-ups of the aforementioned results from our proposed algorithm against Refs. 12 and 40. Figure 11 provides more visual comparison results of our algorithm to Refs. 12 and 40. Additional examples are available in the online supplement to this paper located at http://vision.okstate.edu/jpgregen/jpgregen.htm.

Fig. 11

Subjective visual quality comparison of the images Cactus and Aerial city compressed at quality factors Q=10, 20, and 30. (a) and (m) compressed at Q=10. (e) and (q) compressed at Q=20. (i) and (u) compressed at Q=30. (b), (f), (j), (n), (r), and (v) the result of Ref. 38. (c), (g), (k), (o), (s), and (w) the result of Ref. 40. (d), (h), (l), (p), (t), and (x) the results of our proposed algorithm.

JEI_23_1_013018_f011.png

3.2.

Quantitative Results

PSNR and SSIM44 are two different, but widely accepted, methods for numerically representing image quality. Table 1 shows PSNR results of our proposed algorithm, as well as the results from Refs. 12, 38, 40, and 41. Table 2 shows a similar comparison using SSIM. Both Tables 1 and 2 show that the proposed algorithm gives competitive scores for both PSNR and SSIM.

Table 1

Comparison of PSNR (dB) for various postprocessing techniques applied to different JPEG images. In this table, the bold numbers represent the best performing results and italicized numbers represent the second best performing results.

ImagesBitrate (bpp)JPEGRef. 12Ref. 38Ref. 40Ref. 41Proposed
Lena0.1727.3228.1327.5328.4628.3128.33
0.2430.4030.9930.3131.0731.2330.56
0.3131.9332.4231.5932.5532.5732.61
0.3632.9533.3432.4333.4033.3933.41
Baboon0.2621.5221.7921.5721.9621.9822.06
0.4623.4323.5323.3023.6723.6223.68
0.6224.5024.5624.2924.7324.5924.61
0.7625.2625.2924.9725.4325.3125.24
Bridge0.2323.0623.4223.0823.4523.4323.34
0.4125.1325.3424.9025.3225.4824.98
0.5626.2526.4025.8826.4026.5126.49
0.6927.0127.1126.5427.1427.1027.14
Barbara0.2324.5024.3823.9924.8524.8724.91
0.3526.3326.0225.6626.0326.0726.44
0.4527.6427.3326.9027.7527.2827.64
0.5428.7828.4727.9528.8028.4128.51
Peppers0.2128.1928.2727.5628.6128.2629.08
0.3130.7230.9130.1330.9030.9031.28
0.3831.8532.1431.3632.1832.1332.29
0.4632.7132.9132.1032.9832.8532.99
Plane0.1826.5427.1426.6827.2327.2927.02
0.2830.1630.6129.6430.1830.4329.71
0.3631.9432.1931.0631.8231.9532.51
0.4332.9733.2331.9933.0333.0133.40

Table 2

Comparison of SSIM for various postprocessing techniques applied to different JPEG images. In this table, the bold numbers represent the best performing results and italicized numbers represent the second best performing results.

ImagesBitrate (bpp)JPEGRef. 12Ref. 38Ref. 40Ref. 41Proposed
Lena0.170.7360.7840.7510.7900.7910.794
0.240.8170.8440.8200.8430.8520.846
0.310.8520.8710.8480.8680.8770.872
0.360.8730.8840.8360.8840.8910.886
Baboon0.260.5390.5310.5320.5260.5350.539
0.460.6810.6660.6660.6600.6820.673
0.620.7450.7310.7280.7310.7500.732
0.760.7830.7710.7650.7740.7890.776
Bridge0.230.5710.5720.5610.5630.5690.573
0.410.7110.7030.6930.6940.7100.702
0.560.7750.7650.7540.7680.7800.782
0.690.8090.7990.7880.8000.8200.803
Barbara0.230.6860.7090.6720.7110.7090.719
0.350.7870.7900.7690.7950.8010.795
0.450.8320.8340.8160.8390.8440.845
0.540.8630.8630.8470.8690.8720.873
Peppers0.210.7070.7730.7350.7800.7780.757
0.310.7710.8280.7940.8200.8260.792
0.380.7940.8450.8210.8420.8470.806
0.460.8130.8540.8360.8540.8590.819
Plane0.180.7750.8200.7780.8100.8150.821
0.280.8530.8760.8650.8650.8740.878
0.360.8890.8990.8730.8910.8980.907
0.430.9020.9120.8910.9070.9130.916

To compare the algorithms’ performances, we selected six images from Fig. 7 which represent a range of image content with different properties, such as textures and edges. In Table 1, we provide PSNR for these six images at four different ranges of bit rates, providing a total of 24 scores. Out of the 24 scores for PSNR, our algorithm had the best performance 13 times and the second best performance four times.

Table 2 shows SSIM results for the algorithms. In terms of SSIM, our proposed algorithm outperforms the results of Refs. 12, 38, and 40, and has comparable performance to Ref. 41. In this comparison, the proposed algorithm is a top performer as many times as any other existing state-of-the-art algorithm. However, it is important to note that neither PSNR nor SSIM are perfect measures of visual quality. For example, Refs. 12, 40, and 41 all have higher PSNR on Lena at 0.24 bpp. However, as shown previously in Figs. 8 and 9, the proposed algorithm yields better visual results.

3.3.

Analysis of Computation Time

To determine the effects of image size on our algorithm’s runtime, we used the original images in the categorical image quality (CSIQ)47 database (30 images) and generated different sizes of the images (256×256, 512×512, 768×768, and 1024×1024). To determine the effects of quality factor on our algorithm’s runtime, for each size, we used quality factors of 10, 30, and 50 to generate low-, medium-, and high-quality images, respectively. The test was performed on a modern desktop computer (AMD Phenom II ×4965 Processor at 3.39 GHz, 4 GB RAM, Windows 7 Pro 64 bit, MATLAB 7.8.0), and the processing time was obtained by using MATLAB’s time analysis tools.

Table 3 and Fig. 12 show the average runtime of our algorithm in seconds for each stage separately as well as the total runtime. Figure 13 shows the average execution time of our algorithm for images of different sizes of different qualities.

Table 3

Average runtime (in s) of our proposed algorithm on different image sizes for each stage and totally.

256×256512×512768×7681024×1024
Deblocking0.4091.7343.8556.512
Edge regeneration0.5661.9574.0547.085
Total runtime0.9753.6917.90913.597

Fig. 12

Average runtime (in s) of our proposed algorithm for each stage as well as the total runtime of the algorithm on different image sizes.

JEI_23_1_013018_f012.png

Fig. 13

Average runtime (in s) of our proposed algorithm on different qualities for each image size.

JEI_23_1_013018_f013.png

As shown in Table 3 and Fig. 12, the runtime of the algorithm increases approximately linearly with the number of pixels. As Fig. 13 shows, our proposed algorithm has consistent runtimes performance regardless of the image quality factor for each size.

3.4.

Limitations and Areas for Future Work

Although the proposed algorithm generally works quite effectively at improving quality, there are limitations of the approach. As with most algorithms for artifact removal, the proposed approach is not quite effective for images coded at extremely low bit rates (<0.15bpp). Generally, such low-rate images tend to have severe blocking artifacts and suffer from severe aliasing around the edges (particularly, diagonal edges), see Fig. 14.

Fig. 14

Some failure cases of our algorithm. (a) Lena compressed at 0.13 bpp. (b) Edge map of Lena via Canny edge detection on (a). (c) Result after applying our algorithm on (a), (d) peppers compressed at 0.14 bpp. (e) Edge map of peppers via Canny edge detection on (d). (f) Result after applying our algorithm on (d).

JEI_23_1_013018_f014.png

To handle these types of failures, we would need either a better edge detector to detect strong edges accurately and/or some prior information about the map of the image’s edges. Nonetheless, as shown in Fig. 14, the proposed algorithm does indeed make the images appear more natural, despite the fact that the overall quality is not significantly improved.

The proposed algorithm is also currently limited to processing only the grayscale information in an image; thus, artifacts induced via quantization of the DCT components of the chrominance channels are not addressed. Further improvements in performance could possibly be realized by designing artifact-removal techniques for such color artifacts.

4.

Conclusions

In this paper, we proposed a new method to enhance the visual quality of JPEG images. Our algorithm address the three types of artifacts which are prevalent in JPEG images: blocking, and for edges blurring, and aliasing. Our proposed algorithm first reduces blocking artifacts by smoothing boundaries of blocks and via guided filtering, and it then improves the edges of the image by performing local edge regeneration. We demonstrated the performance of the proposed algorithm against several well-known JPEG artifact-removal methods. The results show that our algorithm is competitive, and in many cases outperforms, competing methods. We have made our codes and images available to the research community. A MATLAB implementation of the proposed method and the images used in this paper are available at http://vision.okstate.edu/jpgregen/jpgregen.htm.

Acknowledgments

This material is based upon the work supported by the National Science Foundation Award 1054612.

References

1. 

N. AhmedT. NatarajanK. R. Rao, “Discrete cosine transform,” IEEE Trans. Comput., C-23 (1), 90 –93 (1974). http://dx.doi.org/10.1109/T-C.1974.223784 ITCOB4 0018-9340 Google Scholar

2. 

G. K. Wallace, “The JPEG still picture compression standard,” Commun. ACM, 34 (4), 30 –44 (1991). http://dx.doi.org/10.1145/103085.103089 CACMA2 0001-0782 Google Scholar

3. 

C. Fogget al., MPEG Video Compression Standard, Springer, Boston, Massachusetts (1996). Google Scholar

4. 

J. Ostermannet al., “Video coding with h. 264/avc: tools, performance, and complexity,” IEEE Circuits Syst. Mag., 4 (1), 7 –28 (2004). http://dx.doi.org/10.1109/MCAS.2004.1286980 ICSYDV 0163-6812 Google Scholar

5. 

T. ChenH. R. WuB. Qiu, “Adaptive postfiltering of transform coefficients for the reduction of blocking artifacts,” IEEE Trans. Circuits Syst. Video Technol., 11 (5), 594 –602 (2001). http://dx.doi.org/10.1109/76.920189 ITCTEM 1051-8215 Google Scholar

6. 

J. XuS. ZhengX. Yang, “Adaptive video-blocking artifact removal in discrete hadamard transform domain,” Opt. Eng., 45 (8), 080501 (2006). http://dx.doi.org/10.1117/1.2280609 OPEGAR 0091-3286 Google Scholar

7. 

H. S. MalvarD. H. Staelin, “The lot: transform coding without blocking effects,” IEEE Trans. Acoust., Speech, Signal Process., 37 (4), 553 –559 (1989). http://dx.doi.org/10.1109/29.17536 0096-3518 Google Scholar

8. 

D. E. PearsonM. W. Whybray, “Transform coding of images using interleaved blocks,” IEE Proc. Commun. Radar Signal Process., Part F, 131 (5), 466 –472 (1984). http://dx.doi.org/10.1049/ip-f-1.1984.0072 0143-7070 Google Scholar

9. 

A. Zakhor, “Iterative procedures for reduction of blocking effects in transform image coding,” IEEE Trans. Circuits Syst. Video Technol., 2 (1), 91 –95 (1992). http://dx.doi.org/10.1109/76.134377 ITCTEM 1051-8215 Google Scholar

10. 

H. S. Malvar, “Biorthogonal and nonuniform lapped transforms for transform coding with reduced blocking and ringing artifacts,” IEEE Trans. Signal Process., 46 (4), 1043 –1053 (1998). http://dx.doi.org/10.1109/78.668555 ITPRED 1053-587X Google Scholar

11. 

Y.-Q. ZhangR. L. PickholtzM. H. Loew, “A new approach to reduce the blocking effect of transform coding [image coding],” IEEE Trans. Commun., 41 (2), 299 –302 (1993). http://dx.doi.org/10.1109/26.216502 IECMBT 0090-6778 Google Scholar

12. 

A. W.-C. LiewH. Yan, “Blocking artifacts suppression in block-coded images using overcomplete wavelet representation,” IEEE Trans. Circuits Syst. Video Technol., 14 (4), 450 –461 (2004). http://dx.doi.org/10.1109/TCSVT.2004.825555 ITCTEM 1051-8215 Google Scholar

13. 

H. C. Reeve IIIJ. S. Lim, “Reduction of blocking effects in image coding,” Opt. Eng., 23 (1), 230134 (1984). http://dx.doi.org/10.1117/12.7973248 OPEGAR 0091-3286 Google Scholar

14. 

B. RamamurthiA. Gersho, “Nonlinear space-variant postprocessing of block coded images,” IEEE Trans. Acoust., Speech, Signal Process., 34 (5), 1258 –1268 (1986). http://dx.doi.org/10.1109/TASSP.1986.1164961 0096-3518 Google Scholar

15. 

J. D. McDonnellR. N. ShortenA. D. Fagan, “An edge classification based approach to the post-processing of transform encoded images,” in 1994 IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP-94), V/329 –V/332 (1994). Google Scholar

16. 

W. E. LynchA. R. ReibmanB. Liu, “Post processing transform coded images using edges,” in 1995 Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP-95), 2323 –2326 (1995). Google Scholar

17. 

J. Huet al., “Removal of blocking and ringing artifacts in transform coded images,” in 1997 IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP-97), 2565 –2568 (1997). Google Scholar

18. 

Y. L. LeeH. C. KimH. W. Park, “Blocking effect reduction of JPEG images by signal adaptive filtering,” IEEE Trans. Image Process., 7 (2), 229 –234 (1998). http://dx.doi.org/10.1109/83.661000 IIPRE4 1057-7149 Google Scholar

19. 

H. JiwuY. Q. ShiD. Xianhua, “Blocking artefact removal based on frequency analysis,” Electron. Lett., 34 (24), 2323 –2325 (1998). http://dx.doi.org/10.1049/el:19981609 ELLEAK 0013-5194 Google Scholar

20. 

J. G. ApostolopoulosN. S. Jayant, “Postprocessing for very low bit-rate video compression,” IEEE Trans. Image Process., 8 (8), 1125 –1129 (1999). http://dx.doi.org/10.1109/83.777093 IIPRE4 1057-7149 Google Scholar

21. 

T. Kasezawa, “Blocking artifacts reduction using discrete cosine transform,” IEEE Trans. Consumer Electron., 43 (1), 48 –55 (1997). http://dx.doi.org/10.1109/30.580384 ITCEDA 0098-3063 Google Scholar

22. 

S. S. O. ChoyY.-H. ChanW.-C. Siu, “Reduction of block-transform image coding artifacts by using local statistics of transform coefficients,” IEEE Signal Process. Lett., 4 (1), 5 –7 (1997). http://dx.doi.org/10.1109/97.551686 IESPEJ 1070-9908 Google Scholar

23. 

B. Zeng, “Reduction of blocking effect in DCT-coded images using zero-masking techniques,” Signal Process., 79 (2), 205 –211 (1999). http://dx.doi.org/10.1016/S0165-1684(99)00094-8 SPRODR 0165-1684 Google Scholar

24. 

B. JeonJ. Jeong, “Blocking artifacts reduction in image compression with block boundary discontinuity criterion,” IEEE Trans. Circuits Syst. Video Technol., 8 (3), 345 –357 (1998). http://dx.doi.org/10.1109/76.678634 ITCTEM 1051-8215 Google Scholar

25. 

Y.-Y. ChenY.-W. ChangW.-C. Yen, “Design a deblocking filter with three separate modes in DCT-based coding,” J. Visual Commun. Image Represent., 19 (4), 231 –244 (2008). http://dx.doi.org/10.1016/j.jvcir.2008.02.001 JVCRE7 1047-3203 Google Scholar

26. 

J. Singhet al., “A signal adaptive filter for blocking effect reduction of JPEG compressed images,” AEU Int. J. Electron. Commun., 65 (10), 827 –839 (2011). http://dx.doi.org/10.1016/j.aeue.2011.01.012 1434-8411 Google Scholar

27. 

Y. YangN. P. GalatsanosA. K. Katsaggelos, “Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images,” IEEE Trans. Circuits Syst. Video Technol., 3 (6), 421 –432 (1993). http://dx.doi.org/10.1109/76.260198 ITCTEM 1051-8215 Google Scholar

28. 

R. L. Stevenson, “Reduction of coding artifacts in transform image coding,” in 1993 IEEE Int. Conf. on Acoustics, Speech, and Signal Processing (ICASSP-93), 401 –404 (1993). Google Scholar

29. 

Y. YangN. P. GalatsanosA. K. Katsaggelos, “Projection-based spatially adaptive reconstruction of block-transform compressed images,” IEEE Trans. Image Process., 4 (7), 896 –908 (1995). http://dx.doi.org/10.1109/83.392332 IIPRE4 1057-7149 Google Scholar

30. 

J. ChouM. CrouseK. Ramchandran, “A simple algorithm for removing blocking artifacts in block-transform coded images,” IEEE Signal Process. Lett., 5 (2), 33 –35 (1998). http://dx.doi.org/10.1109/97.659544 IESPEJ 1070-9908 Google Scholar

31. 

H. PaekR.-C. KimS.-U. Lee, “On the pocs-based postprocessing technique to reduce the blocking artifacts in transform coded images,” IEEE Trans. Circuits Syst. Video Technol., 8 (3), 358 –367 (1998). http://dx.doi.org/10.1109/76.678636 ITCTEM 1051-8215 Google Scholar

32. 

G. J. SullivanR. L. Baker, “Efficient quadtree coding of images and video,” IEEE Trans. Image Process., 3 (3), 327 –331 (1994). http://dx.doi.org/10.1109/83.287030 IIPRE4 1057-7149 Google Scholar

33. 

R. SzeliskiH.-Y. Shum, “Motion estimation with quadtree splines,” IEEE Trans. Pattern Anal. Mach. Intell., 18 (12), 1199 –1210 (1996). http://dx.doi.org/10.1109/34.546257 ITPIDJ 0162-8828 Google Scholar

34. 

I. Rheeet al., “Quadtree-structured variable-size block-matching motion estimation with minimal error,” IEEE Trans. Circuits Syst. Video Technol., 10 (1), 42 –50 (2000). http://dx.doi.org/10.1109/76.825857 ITCTEM 1051-8215 Google Scholar

35. 

B. A. BanisterT. R. Fischer, “Quadtree classification and TCQ image coding,” IEEE Trans. Circuits Syst. Video Technol., 11 (1), 3 –8 (2001). http://dx.doi.org/10.1109/76.894278 ITCTEM 1051-8215 Google Scholar

36. 

A. S. Al-FohoumA. M. Reza, “Combined edge crispiness and statistical differencing for deblocking JPEG compressed images,” IEEE Trans. Image Process., 10 (9), 1288 –1298 (2001). http://dx.doi.org/10.1109/83.941853 IIPRE4 1057-7149 Google Scholar

37. 

S. LiuA. C. Bovik, “Efficient DCT-domain blind measurement and reduction of blocking artifacts,” IEEE Trans. Circuits Syst. Video Technol., 12 (12), 1139 –1149 (2002). http://dx.doi.org/10.1109/TCSVT.2002.806819 ITCTEM 1051-8215 Google Scholar

38. 

Y. LuoR. K. Ward, “Removing the blocking artifacts of block-based DCT compressed images,” IEEE Trans. Image Process., 12 (7), 838 –842 (2003). http://dx.doi.org/10.1109/TIP.2003.814252 IIPRE4 1057-7149 Google Scholar

39. 

A. FoiV. KatkovnikK. Egiazarian, “Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images,” IEEE Trans. Image Process., 16 (5), 1395 –1411 (2007). http://dx.doi.org/10.1109/TIP.2007.891788 IIPRE4 1057-7149 Google Scholar

40. 

G. Zhaiet al., “Efficient image deblocking based on postfiltering in shifted windows,” IEEE Trans. Circuits Syst. Video Technol., 18 (1), 122 –126 (2008). http://dx.doi.org/10.1109/TCSVT.2007.906942 ITCTEM 1051-8215 Google Scholar

41. 

G. Zhaiet al., “Efficient quadtree based block-shift filtering for deblocking and deringing,” J. Visual Commun. Image Represent., 20 (8), 595 –607 (2009). http://dx.doi.org/10.1016/j.jvcir.2009.06.004 JVCRE7 1047-3203 Google Scholar

42. 

K. HeJ. SunX. Tang, “Guided image filtering,” in Computer Vision–ECCV 2010, 1 –14 (2010). Google Scholar

43. 

H. R. SheikhM. F. SabirA. C. Bovik, “A statistical evaluation of recent full reference image quality assessment algorithms,” IEEE Trans. Image Process., 15 (11), 3440 –3451 (2006). http://dx.doi.org/10.1109/TIP.2006.881959 IIPRE4 1057-7149 Google Scholar

44. 

Z. Wanget al., “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process., 13 (4), 600 –612 (2004). http://dx.doi.org/10.1109/TIP.2003.819861 IIPRE4 1057-7149 Google Scholar

45. 

J. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Mach. Intell., PAMI-8 (6), 679 –698 (1986). http://dx.doi.org/10.1109/TPAMI.1986.4767851 ITPIDJ 0162-8828 Google Scholar

46. 

N. Otsu, “A threshold selection method from gray-level histograms,” Automatica, 11 (285–296), 23 –27 (1975). Google Scholar

47. 

E. C. LarsonD. M. Chandler, “Most apparent distortion: full-reference image quality assessment and the role of strategy,” J. Electron. Imaging, 19 (1), 011006 (2010). http://dx.doi.org/10.1117/1.3267105 JEIME5 1017-9909 Google Scholar

Biography

S. Alireza Golestaneh received his BSc degree in electrical engineering from the Shiraz University, Shiraz, Iran, in 2010. He is currently working toward the MSc degree in the School of Electrical and Computer Engineering, Oklahoma State University, Stillwater, Oklahoma. His research interests include image and video processing, multimedia compression, modeling, image and video quality assessment, computational vision, visual perception, computer vision, and machine learning.

Damon M. Chandler received his BS degree from Johns Hopkins University; and MS, and PhD degrees in electrical engineering from Cornell University. From 2005 to 2006, he was a postdoctoral researcher in Department of Psychology at Cornell. He is currently an associate professor in the School of Electrical and Computer Engineering at Oklahoma State University, where he heads the Laboratory of Computational Perception and Image Quality. His research interests include image processing, computational vision, and visual perception.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.
S. Alireza Golestaneh and Damon M. Chandler "Algorithm for JPEG artifact reduction via local edge regeneration," Journal of Electronic Imaging 23(1), 013018 (6 February 2014). https://doi.org/10.1117/1.JEI.23.1.013018
Published: 6 February 2014
Lens.org Logo
CITATIONS
Cited by 35 scholarly publications.
Advertisement
Advertisement
KEYWORDS
Image quality

Image compression

Image filtering

Detection and tracking algorithms

Visualization

Edge detection

Image enhancement


CHORUS Article. This article was made freely available starting 06 February 2015

Back to Top