Perceptually-inspired super-resolution of compressed videos

Di Ma; Mariana Afonso; Fan Zhang; David R. Bull

doi:10.1117/12.2530688

6 September 2019 Perceptually-inspired super-resolution of compressed videos

Di Ma, Mariana Afonso, Fan Zhang, David R. Bull

Proceedings Volume 11137, Applications of Digital Image Processing XLII; 1113717 (2019) https://doi.org/10.1117/12.2530688
Event: SPIE Optical Engineering + Applications, 2019, San Diego, California, United States

Abstract

Spatial resolution adaptation is a technique which has often been employed in video compression to enhance coding efficiency. This approach encodes a lower resolution version of the input video and reconstructs the original resolution during decoding. Instead of using conventional up-sampling filters, recent work has employed advanced super-resolution methods based on convolutional neural networks (CNNs) to further improve reconstruction quality. These approaches are usually trained to minimise pixel-based losses such as Mean-Squared Error (MSE), despite the fact that this type of loss metric does not correlate well with subjective opinions. In this paper, a perceptually-inspired super-resolution approach (M-SRGAN) is proposed for spatial up-sampling of compressed video using a modified CNN model, which has been trained using a generative adversarial network (GAN) on compressed content with perceptual loss functions. The proposed method was integrated with HEVC HM 16.20, and has been evaluated on the JVET Common Test Conditions (UHD test sequences) using the Random Access configuration. The results show evident perceptual quality improvement over the original HM 16.20, with an average bitrate saving of 35.6% (Bjøntegaard Delta measurement) based on a perceptual quality metric, VMAF.

Conference Presentation

Citation Download Citation

Di Ma, Mariana Afonso, Fan Zhang, and David R. Bull "Perceptually-inspired super-resolution of compressed videos", Proc. SPIE 11137, Applications of Digital Image Processing XLII, 1113717 (6 September 2019); https://doi.org/10.1117/12.2530688

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available