This PDF file contains the front matter associated with SPIE Proceedings Volume 9599, including the Title Page, Copyright information, Table of Contents, Introduction (if any), and Conference Committee listing.

The broadband spectrum contains more information than what the human eye can detect. Spectral information from different wavelengths can provide unique information about the intrinsic properties of an object. Recently compressed sensing imaging systems with low acquisition time have been introduced. To utilize compressed sensing strategies, strong reconstruction algorithms that can reconstruct a signal from sparse observations are required. This work proposes a cross-spectral multi-layered conditional random field (CS-MCRF) approach for sparse reconstruction of multi-spectral compressive sensing data in multi-spectral stereoscopic vision imaging systems. The CS-MCRF will use information between neighboring spectral bands to better utilize available information for reconstruction. This method was evaluated using simulated compressed sensing multi-spectral imaging data. Results show improvement over existing techniques in preserving spectral fidelity while effectively inferring missing information from sparsely available observations.

Nowadays most of digital information is obtained using mobile devices specially smartphones. In particular, it brings the opportunity for optical character recognition in camera-captured images. For this reason many recognition applications have been recently developed such as recognition of license plates, business cards, receipts and street signal; document classification, augmented reality, language translator and so on. Camera-captured images are usually affected by geometric distortions, nonuniform illumination, shadow, noise, which make difficult the recognition task with existing systems. It is well known that the Fourier phase contains a lot of important information regardless of the Fourier magnitude. So, in this work we propose a phase-based recognition system exploiting phase-congruency features for illumination/scale invariance. The performance of the proposed system is tested in terms of miss classifications and false alarms with the help of computer simulation.

A performance evaluation of several state-of-the-art correlation filters within the context of target tracking is presented. The filters are tested using an introduced algorithm that is adapted online using information of current and past scene frames of the scene. The algorithm achieves a high-rate operation by focusing signal processing on a small fragment of the scene in each frame. The correlation filters are tested using several video test sequences that contain geometric modifications of the target, partial occlusions and clutter. The performance of the tested filters is characterized in terms of detection efficiency, tracking accuracy, and computational complexity using objective metrics.

Shearlets are wavelet-like systems which are better suited for handling geometric features in multi-dimensional data than traditional wavelets. A novel method for edge and line detection which is in the spirit of phase congruency but is based on a complex shearlet transform will be presented. This approach to detection yields an approximate tangent direction of detected discontinuities as a byproduct of the computation, which then yields local curvature estimates.

Two applications of the edge detection method will be discussed. First, the tracking and classification of flame fronts is a critical component of research in technical thermodynamics. Quite often, the flame fronts are transient or weak and the images are noisy. The standard methods used in the field for the detection of flame fronts do not handle such data well. Fortunately, using the shearlet-based edge measure yields good results as well as an accurate approximation of local curvature. Furthermore, a modification of the method will yield line detection, which is important for certain imaging modalities.

Second, the Wadden tidal flats are a biodiverse region along the North Sea coast. One approach to surveying the delicate region and tracking the topographical changes is to use pre-existing Synthetic Aperture Radar (SAR) images. Unfortunately, SAR data suffers from multiplicative noise as well as sensitivity to environmental factors. The first large-scale mapping project of that type showed good results but only with a tremendous amount of manual interaction because there are many edges in the data which are not boundaries of the tidal flats but are edges of features like fields or islands. Preliminary results will be presented.

Deep learning-based algorithms have become increasingly efficient in recognition and detection tasks, especially when they are trained on large-scale datasets. Such recent success has led to a speculation that deep learning methods are comparable to or even outperform human visual system in its ability to detect and recognize objects and their features. In this paper, we focus on the specific task of gender recognition in images when they have been processed by privacy protection filters (e.g., blurring, masking, and pixelization) applied at different strengths. Assuming a privacy protection scenario, we compare the performance of state of the art deep learning algorithms with a subjective evaluation obtained via crowdsourcing to understand how privacy protection filters affect both machine and human vision.

A promising daytime smoke plume detection for a land-based early forest fire detection system is proposed. The visible video imagery from a land-based monitoring camera is processed to detect the smoke which likely rises in an early stage of a forest fire. Unlike the fire core and its surrounding heat which are detected via day/night infrared imaging, the relatively cold smoke plume can only be captured in the visible spectrum of light. The smoke plume is detected via exploitation of its temporal signature. This is accomplished via Principal Component Transformation (PCT) operations on consecutive sequences of visible video frames followed by spatial filtering of one of the resulting low-order Principal Component (PC) images. It is shown that the blue channel of the Red, Green, Blue (RGB) color camera is most effective in detecting the smoke plume. Smoke plume is clearly detected and isolated via simple blurring, thresholding, and median filtering of one of the resulting low-order principle component (PC) images. The robustness of this PCA-based method relative to simple temporal frame differencing and use of color, i.e., visible spectral signature of smoke, are discussed. Various parameters of the system including the required observation time and number of frames to retain for PCT, selection of which low-order PC to use, and types and sizes of the filters applied to the selected PC image to detect and isolate the smoke plume, are discussed.

The edge of an image is a set of points organized in a curved line, where in each of these points the brightness of the image changes abruptly, or has discontinuities, in order to find these edges there will be five different mathematical methods to be used and later on compared with its peers, this is with the aim of finding which of the methods is the one that can find the edges of any given image. In this paper these five methods will be used for medical purposes in order to find which one is capable of finding the edges of a scanned image more accurately than the others. The problem consists in analyzing the following two biomedicals images. One of them represents a brain tumor and the other one a liver tumor. These images will be analyzed with the help of the five methods described and the results will be compared in order to determine the best method to be used. It was decided to use different algorithms of edge detection in order to obtain the results shown below; Bessel algorithm, Morse algorithm, Hermite algorithm, Weibull algorithm and Sobel algorithm. After analyzing the appliance of each of the methods to both images it´s impossible to determine the most accurate method for tumor detection due to the fact that in each case the best method changed, i.e., for the brain tumor image it can be noticed that the Morse method was the best at finding the edges of the image but for the liver tumor image it was the Hermite method. Making further observations it is found that Hermite and Morse have for these two cases the lowest standard deviations, concluding that these two are the most accurate method to find the edges in analysis of biomedical images.

One inpainting procedure based on Airy diffusion is proposed, implemented via Maple and applied to some digital images. Airy diffusion is a partial differential equation with spatial derivatives of third order in contrast with the usual diffusion with spatial derivatives of second order. Airy diffusion generates the Airy semigroup in terms of the Airy functions which can be rewritten in terms of Bessel functions. The Airy diffusion can be used to smooth an image with the corresponding noise elimination via convolution. Also the Airy diffusion can be used to erase objects from an image. We build an algorithm using the Maple package ImageTools and such algorithm is tested using some images. Our results using Airy diffusion are compared with the similar results using standard diffusion. We observe that Airy diffusion generates powerful filters for image processing which could be incorporated in the usual packages for image processing such as ImageJ and Photoshop. Also is interesting to consider the possibility to incorporate the Airy filters as applications for smartphones and smart-glasses.

Wide color gamut such as BT.2020 allows pictures to be rendered with sharper details and more vivid colors. It is considered an essential video parameter for next generation content generation, and has recently drawn significant commercial interest. As the upgrade cycle of content production work flow and consumer displays take place, current generation and next generation video content are expected to co-exist. Thus, maintaining backward compatibility becomes an important consideration for efficient content delivery systems. The scalable extension of HEVC (SHVC) was recently finalized in the second version of the HEVC specifications. SHVC provides a color mapping tool to improve scalable coding efficiency when the base layer and the enhancement layer video signals are in a different color gamut. The SHVC color mapping tool uses a 3D Look-Up Table (3D LUT) based cross-color linear model to efficiently convert the video in the base layer color gamut into the enhancement layer color gamut. Due to complexity concerns, certain limitations, including limiting the maximum 3D LUT size to 8x2x2, were applied to the color mapping process in SHVC. In this paper, we investigate the complexity and performance trade-off of the 3D LUT based color mapping process. Specifically, we explore the performance benefits of enlarging the size of the 3D LUT with various linear models. In order to reduce computational complexity, a simplified linear model is used within each 3D LUT partition. Simulation results are provided to detail the various performance vs. complexity trade-offs achievable in the proposed design.

High Dynamic Range (HDR) and Wider Color Gamut (WCG) content represents a greater range of luminance levels and a more complete reproduction of colors found in real-world scenes. The current video distribution environments deliver Standard Dynamic Range (SDR) signal. Therefore, there might be some significant implication on today's end-to-end ecosystem from content creation to distribution and finally to consumption. For SDR content, the common practice is to apply compression on Y'CbCr 4:2:0 using gamma transfer function and non-constant luminance 4:2:0 chroma subsampling. For HDR and WCG content, it is desirable to examine if such signal format still works well for compression, and it is interesting to know if the overall system performance can be further improved by exploring different signal formats and processing workflows. In this paper, we will provide some of our insight into those problems.

Displays capable of showing a greater range of luminance values can render content containing high dynamic range information in a way such that the viewers have a more immersive experience. This paper introduces the design aspects of a high dynamic range (HDR) system, and examines the performance of the HDR processing chain in terms of compression efficiency. Specifically it examines the relation between recently introduced Society of Motion Picture and Television Engineers (SMPTE) ST 2084 transfer function and the High Efficiency Video Coding (HEVC) standard. SMPTE ST 2084 is designed to cover the full range of an HDR signal from 0 to 10,000 nits, however in many situations the valid signal range of actual video might be smaller than SMPTE ST 2084 supported range. The above restricted signal range results in restricted range of code values for input video data and adversely impacts compression efficiency. In this paper, we propose a code value remapping method that extends the restricted range code values into the full range code values so that the existing standards such as HEVC may better compress the video content. The paper also identifies related non-normative encoder-only changes that are required for remapping method for a fair comparison with anchor. Results are presented comparing the efficiency of the current approach versus the proposed remapping method for HM-16.2.

High Dynamic Range and Wide Color Gamut (HDR/WCG) Video Coding is an area of intense research interest in the engineering community, for potential near-term deployment in the marketplace. HDR greatly enhances the dynamic range of video content (up to 10,000 nits), as well as broadens the chroma representation (BT.2020). The resulting content offers new challenges in its coding and transmission. The Moving Picture Experts Group (MPEG) of the International Standards Organization (ISO) is currently exploring coding efficiency and/or the functionality enhancements of the recently developed HEVC video standard for HDR and WCG content. FastVDO has developed an advanced approach to coding HDR video, based on splitting the HDR signal into a smoothed luminance (SL) signal, and an associated base signal (B). Both signals are then chroma downsampled to YFbFr 4:2:0 signals, using advanced resampling filters, and coded using the Main10 High Efficiency Video Coding (HEVC) standard, which has been developed jointly by ISO/IEC MPEG and ITU-T WP3/16 (VCEG). Our proposal offers both efficient coding, and backwards compatibility with the existing HEVC Main10 Profile. That is, an existing Main10 decoder can produce a viewable standard dynamic range video, suitable for existing screens. Subjective tests show visible improvement over the anchors. Objective tests show a sizable gain of over 25% in PSNR (RGB domain) on average, for a key set of test clips selected by the ISO/MPEG committee.

In high dynamic range (HDR) video, it is possible to represent a wider range of intensities and contrasts compared to the current standard dynamic range (SDR) video. HDR video can simultaneously preserve details in very bright and very dark areas of a scene whereas these details become lost or washed out in SDR video. Because the perceived quality due to this increased fidelity may not fit the same model of perceived quality in the SDR video, it is not clear whether the objective metrics that have been widely used and studied for SDR visual experience are reasonably accurate for HDR cases, in terms of correlation with subjective measurement for HDR video quality. This paper investigates several objective metrics and their correlation to subjective quality for a variety of HDR video content. Results are given for the case of HDR content compressed at different bit rates. In addition to rating the relevance of each objective metric in terms of its correlation to the subjective measurements, comparisons are also presented to show how closely different objective metrics can predict the results obtained by subjective quality assessment in terms of coding efficiency provided by different coding processes.

This paper reports the details and results of the subjective evaluations conducted at EPFL to evaluate the responses to the Call for Evidence (CfE) for High Dynamic Range (HDR) and Wide Color Gamut (WCG) Video Coding issued by Moving Picture Experts Group (MPEG). The CfE on HDR/WCG Video Coding aims to explore whether the coding efficiency and/or the functionality of the current version of HEVC standard can be signi_cantly improved for HDR and WCG content. In total, nine submissions, five for Category 1 and four for Category 3a, were compared to the HEVC Main 10 Profile based Anchor. More particularly, five HDR video contents, compressed at four bit rates by each proponent responding to the CfE, were used in the subjective evaluations. Further, the side-by-side presentation methodology was used for the subjective experiment to discriminate small differences between the Anchor and proponents. Subjective results shows that the proposals provide evidence that the coding efficiency can be improved in a statistically noticeable way over MPEG CfE Anchors in terms of perceived quality within the investigated content. The paper further benchmarks the selected objective metrics based on their correlations with the subjective ratings. It is shown that PSNR-DE1000, HDRVDP- 2, and PSNR-Lx can reliably detect visible differences between the proposed encoding solutions and current HEVC standard.

Several techniques are introduced and reviewed in high-definition computer holography. The reconstructed object is usually CG-modeled virtual objects, but it is possible to digitally record light of physical objects and reconstruct it by the high-definition computer-generated hologram. The process can be considered as a sort of digital signal processing of light.

Holography has the potential to become the ultimate 3D experience. Nevertheless, in order to achieve practical working systems, major scientific and technological challenges have to be tackled. In particular, as digital holographic data represents a huge amount of information, the development of efficient compression techniques is a key component. This problem has gained significant attention by the research community during the last 10 years. Given that holograms have very different signal properties when compared to natural images and video sequences, existing compression techniques (e.g. JPEG or MPEG) remain suboptimal, calling for innovative compression solutions. In this paper, we will review and analyze past and on-going work for the compression of digital holographic data.

The hybrid point-source/wave-field method is a newly proposed approach for Computer-Generated Hologram (CGH) calculation, based on the slicing of the scene into several depth layers parallel to the hologram plane. The complex wave scattered by each depth layer is then computed using either a wave-field or a point-source approach according to a threshold criterion on the number of points within the layer. Finally, the complex waves scattered by all the depth layers are summed up in order to obtain the final CGH. Although outperforming both point-source and wave-field methods without producing any visible artifact, this approach has not yet been used for animated holograms, and the possible exploitation of temporal redundancies has not been studied. In this paper, we propose a fast computation of video holograms by taking into account those redundancies. Our algorithm consists of three steps. First, intensity and depth data of the current 3D video frame are extracted and compared with those of the previous frame in order to remove temporally redundant data. Then the CGH pattern for this compressed frame is generated using the hybrid point-source/wave-field approach. The resulting CGH pattern is finally transmitted to the video output and stored in the previous frame buffer. Experimental results reveal that our proposed method is able to produce video holograms at interactive rates without producing any visible artifact.

Recently several papers reported efficient techniques to compress digital holograms. Typically, the rate-distortion performance of these solutions was evaluated by means of objective metrics such as Peak Signal-to-Noise Ratio (PSNR) or the Structural Similarity Index Measure (SSIM) by either evaluating the quality of the decoded hologram or the reconstructed compressed hologram. Seen the specific nature of holograms, it is relevant to question to what extend these metrics provide information on the effective visual quality of the reconstructed hologram. Given that today no holographic display technology is available that would allow for a proper subjective evaluation experiment, we propose in this paper a methodology that is based on assessing the quality of a reconstructed compressed hologram on a regular 2D display. In parallel, we also evaluate several coding engines, namely JPEG configured with the default perceptual quantization tables and with uniform quantization tables, JPEG 2000, JPEG 2000 extended with arbitrary packet decompositions and direction-adaptive filters and H.265/HEVC configured in intra-frame mode. The experimental results indicate that the perceived visual quality and the objective measures are well correlated. Moreover, also the superiority of the HEVC and the extended JPEG 2000 coding engines was confirmed, particularly at lower bitrates.

The image fusion is combination of two or more images into one image. The fusion of multi-band spectral images has been in many applications, such as thermal system, remote sensing, medical treatment, etc. Images are taken with the different imaging sensors. If the sensors take images through the different optical paths in the same time, it will be in the different positions. The task of the image registration will be more difficult. Because the images are in the different field of views (F.O.V.), the different resolutions and the different view angles. It is important to build the relationship of the viewpoints in one image to the other image. In this paper, we focus on the problem of image registration for two non-pinhole sensors. The affine transformation between the 2-D image and the 3-D real world can be derived from the geometrical optics of the sensors. In the other word, the geometrical affine transformation function of two images are derived from the intrinsic and extrinsic parameters of two sensors. According to the affine transformation function, the overlap of the F.O.V. in two images can be calculated and resample two images in the same resolution. Finally, we construct the image registration model by the mapping function. It merges images for different imaging sensors. And, imaging sensors absorb different wavebands of electromagnetic spectrum at the different position in the same time.

Matching images is a fundamental problem in image processing. The most common technique used to compare binary images is to calculate the correlation between two images or simply to subtract them. Both of these methods –as well as other matching methods– require some type of similarity operation to be applied to the whole image, and hence they are image size dependent. This implies that as image size increases, more processing time is required. However, with image sizes already exceeding 20 mega-pixels and standard image sizes doubling approximately every five years, the need to find a size invariant image matching method is becoming crucial. In this paper, we present a quick way to compare and match binary images based on the Probabilistic Matching Model (PMM). We present two simple image size invariant methods based on PMM: one for fast detection of dissimilar binary images and another for matching binary images. For detecting dissimilar binary images we introduce the Dissimilar Detection via Mapping method (DDM). We compare DDM to other popular matching methods used in the image processing arena and show that DDM is magnitudes faster than any other method. For binary image matching, we use DDM as a preprocessor for other popular methods to speed up their matching speed. In particular, we use DDM with cross correlation to speed it up. Test results are presented for real images varying in size from 16 kilo-pixel images to 10 mega-pixel images to show the method’s size invariance.

The Kaup-Kupershmidt operator is applied to the two-dimensional solution of the Airy-diffusion equation and the resulting filter is applied via convolution to image processing. The full procedure is implemented using Maple code with the package ImageTools. Some experiments were performed using a wide category of images including biomedical images generated by magnetic resonance, computarized axial tomography, positron emission tomography, infrared and photon diffusion. The Airy-Kaup–Kupershmidt filter can be used as a powerful edge detector and as powerful enhancement tool in image processing. It is expected that the Airy-Kaup–Kupershmidt could be incorporated in standard programs for image processing such as ImageJ.

Magnetic resonance (MR) data reconstruction can be computationally a challenging task. The signal-to-noise ratio might also present complications, especially with high-resolution images. In this sense, data compression can be useful not only for reducing the complexity and memory requirements, but also to reduce noise, even to allow eliminate spurious components.This article proposes the use of a system based on singular value decomposition of low order for noise reconstruction and reduction in MR imaging system. The proposed method is evaluated using in vivo MRI data. Rebuilt images with less than 20 of the original data and with similar quality in terms of visual inspection are presented. Also a quantitative evaluation of the method is presented.

High dynamic range (HDR) displays use local backlight modulation to produce both high brightness levels and large contrast ratios. Thus, the display rendering algorithm and its parameters may greatly affect HDR visual experience. In this paper, we analyze the impact of display rendering on perceived quality for a specific display (SIM2 HDR47) and for a popular application scenario, i.e., HDR image compression. To this end, we assess whether significant differences exist between subjective quality of compressed images, when these are displayed using either the built-in rendering of the display, or a rendering algorithm developed by ourselves. As a second contribution of this paper, we investigate whether the possibility to estimate the true pixel-wise luminance emitted by the display, offered by our rendering approach, can improve the performance of HDR objective quality metrics that require true pixel-wise luminance as input.

Color transfer methods alter the look of a source image with regards to a reference image. So far, the proposed color transfer methods have been limited to low-dynamic-range (LDR) images. Unlike LDR images, which are display-dependent, high-dynamic-range (HDR) images contain real physical values of the world luminance and are able to capture high luminance variations and finest details of real world scenes. Therefore, there exists a strong discrepancy between the two types of images. In this paper, we bridge the gap between the color transfer domain and the HDR imagery by introducing HDR extensions to LDR color transfer methods. We tackle the main issues of applying a color transfer between two HDR images. First, to address the nature of light and color distributions in the context of HDR imagery, we carry out modifications of traditional color spaces. Furthermore, we ensure high precision in the quantization of the dynamic range for histogram computations. As image clustering (based on light and colors) proved to be an important aspect of color transfer, we analyze it and adapt it to the HDR domain. Our framework has been applied to several state-of-the-art color transfer methods. Qualitative experiments have shown that results obtained with the proposed adaptation approach exhibit less artifacts and are visually more pleasing than results obtained when straightforwardly applying existing color transfer methods to HDR images.

The JPEG standard has known an enormous market adoption. Daily, billions of pictures are created, stored and exchanged in this format. The JPEG committee acknowledges this success and spends continued efforts in maintaining and expanding the standard specifications. JPEG XT is a standardization effort targeting the extension of the JPEG features by enabling support for high dynamic range imaging, lossless and near-lossless coding, and alpha channel coding, while also guaranteeing backward and forward compatibility with the JPEG legacy format. This paper gives an overview of the current status of the JPEG XT standards suite. It discusses the JPEG legacy specification, and details how higher dynamic range support is facilitated both for integer and floating-point color representations. The paper shows how JPEG XT's support for lossless and near-lossless coding of low and high dynamic range images is achieved in combination with backward compatibility to JPEG legacy. In addition, the extensible boxed-based JPEG XT file format on which all following and future extensions of JPEG will be based is introduced. This paper also details how the lossy and lossless representations of alpha channels are supported to allow coding transparency information and arbitrarily shaped images. Finally, we conclude by giving prospects on upcoming JPEG standardization initiative JPEG Privacy & Security, and a number of other possible extensions in JPEG XT.

High Dynamic Range (HDR) video provides a step change in viewing experience, for example the ability to clearly see the soccer ball when it is kicked from the shadow of the stadium into sunshine. To achieve the full potential of HDR video, so-called true HDR, it is crucial that all the dynamic range that was captured is delivered to the display device and tone mapping is confined only to the display. Furthermore, to ensure widespread uptake of HDR imaging, it should be low cost and available on commodity hardware. This paper describes an end-to-end HDR pipeline for capturing, encoding and streaming high-definition HDR video in real-time using off-the-shelf components. All the lighting that is captured by HDR-enabled consumer cameras is delivered via the pipeline to any display, including HDR displays and even mobile devices with minimum latency. The system thus provides an integrated HDR video pipeline that includes everything from capture to post-production, archival and storage, compression, transmission, and display.

In recent years, High Dynamic Range (HDR) imaging has attracted significant attention from industry and academia. As a result, there are currently several on-going efforts towards standardization and benchmarking of existing tools for HDR image and video, and one of the key aspects is that of video quality measurement (both subjective and objective approaches). Therefore, this paper aims to identify few key challenges in the said area and then discuss existing solutions. Specifically, we first discuss a few important practical aspects that make HDR video quality measurement potentially challenging. Second, we report our recent efforts towards developing HDR video datasets that have been subjectively annotated for visual quality. Finally, we analyze and compare the effectiveness of existing solutions for objective quality prediction.

Dynamic range compression (or tone mapping) of HDR content is an essential step towards rendering it on traditional LDR displays in a meaningful way. This is however non-trivial and one of the reasons is that tone mapping operators (TMOs) usually need content-specific parameters to achieve the said goal. While subjective TMO parameter adjustment is the most accurate, it may not be easily deployable in many practical applications. Its subjective nature can also influence the comparison of different operators. Thus, there is a need for objective TMO parameter selection to automate the rendering process. To that end, we investigate into a new objective method for TMO parameters optimization. Our method is based on quantification of contrast reversal and naturalness. As an important advantage, it does not require any prior knowledge about the input HDR image and works independently on the used TMO. Experimental results using a variety of HDR images and several popular TMOs demonstrate the value of our method in comparison to default TMO parameter settings.

How to deliver videos to consumers over the network for optimal quality-of-experience (QoE) has been the central goal of modern video delivery services. Surprisingly, regardless of the large volume of videos being delivered everyday through various systems attempting to improve visual QoE, the actual QoE of end consumers is not properly assessed, not to say using QoE as the key factor in making critical decisions at the video hosting, network and receiving sites. Real-world video streaming systems typically use bitrate as the main video presentation quality indicator, but using the same bitrate to encode different video content could result in drastically different visual QoE, which is further affected by the display device and viewing condition of each individual consumer who receives the video. To correct this, we have to put QoE back to the driver's seat and redesign the video delivery systems. To achieve this goal, a major challenge is to find an objective video presentation QoE predictor that is accurate, fast, easy-to-use, display device adaptive, and provides meaningful QoE predictions across resolution and content. We propose to use the newly developed SSIMplus index (https://ece.uwaterloo.ca/~z70wang/research/ssimplus/) for this role. We demonstrate that based on SSIMplus, one can develop a smart adaptive video streaming strategy that leads to much smoother visual QoE impossible to achieve using existing adaptive bitrate video streaming approaches. Furthermore, SSIMplus finds many more applications, in live and file-based quality monitoring, in benchmarking video encoders and transcoders, and in guiding network resource allocations.

The visual Just-Noticeable-Difference (JND) metric is characterized by the detectable minimum amount of two visual stimuli. Conducting the subjective JND test is a labor-intensive task. In this work, we present a novel interactive method in performing the visual JND test on compressed image/video. JND has been used to enhance perceptual visual quality in the context of image/video compression. Given a set of coding parameters, a JND test is designed to determine the distinguishable quality level against a reference image/video, which is called the anchor. The JND metric can be used to save coding bitrates by exploiting the special characteristics of the human visual system. The proposed JND test is conducted using a binary-forced choice, which is often adopted to discriminate the difference in perception in a psychophysical experiment. The assessors are asked to compare coded image/video pairs and determine whether they are of the same quality or not. A bisection procedure is designed to find the JND locations so as to reduce the required number of comparisons over a wide range of bitrates. We will demonstrate the efficiency of the proposed JND test, report experimental results on the image and video JND tests.

Compression has enabled years of exponential growth in global video consumption, providing video everywhere, with few perceptible artifacts. Automated Video Quality Assessment (VQA) is an enabler of compression. We present data showing video contrast affects on artifact visibility. Based on our data, we propose a contrast-gain-control VQA algorithm, with target spatiotemporal property weighting, and using our data to tune existing VQA algorithms for improved artifact threshold predictions. This paper provides much needed data on natural video mask contrast and artifact visibility, and provides important insights for how VQA algorithms can be improved to better predict video quality in the high-quality regime.

Over-the-top mobile video streaming is invariably influenced by volatile network conditions which cause playback interruptions (stalling events), thereby impairing users' quality of experience (QoE). Developing models that can accurately predict users' QoE could enable the more efficient design of quality-control protocols for video streaming networks that reduce network operational costs while still delivering high-quality video content to the customers. Existing objective models that predict QoE are based on global video features, such as the number of stall events and their lengths, and are trained and validated on a small pool of ad hoc video datasets, most of which are not publicly available. The model we propose in this work goes beyond previous models as it also accounts for the fundamental effect that a viewer's recent level of satisfaction or dissatisfaction has on their overall viewing experience. In other words, the proposed model accounts for and adapts to the recency, or hysteresis effect caused by a stall event in addition to accounting for the lengths, frequency of occurrence, and the positions of stall events - factors that interact in a complex way to affect a user's QoE. On the recently introduced LIVE-Avvasi Mobile Video Database, which consists of 180 distorted videos of varied content that are afflicted solely with over 25 unique realistic stalling events, we trained and validated our model to accurately predict the QoE, attaining standout QoE prediction performance.

Natural and artificial textures occur frequently in images and in video sequences. Image/video coding systems based on texture synthesis can make use of a reliable texture synthesis quality assessment method in order to improve the compression performance in terms of perceived quality and bit-rate. Existing objective visual quality assessment methods do not perform satisfactorily when predicting the synthesized texture quality. In our previous work, we showed that texture regularity can be used as an attribute for estimating the quality of synthesized textures. In this paper, we study the effect of another texture attribute, namely texture granularity, on the quality of synthesized textures. For this purpose, subjective studies are conducted to assess the quality of synthesized textures with different levels (low, medium, high) of perceived texture granularity using different types of texture synthesis methods.

An approach to the neural measurement of perceived image quality using electroencephalography (EEG) is presented. 6 different images were tested on 6 different distortion levels. The distortions were introduced by a hybrid video encoder. The presented study consists of two parts: In a first part, subjects were asked to evaluate the quality of the test stimuli behaviorally during a conventional psychophysical test using a degradation category rating procedure. In a second part, subjects were presented undistorted and distorted texture images in a periodically alternating fashion at a fixed frequency. This alternating presentation elicits so called steady-state visual evoked potentials (SSVEP) as a brain response that can be measured on the scalp. The amplitude of modulations in the brain signals is significantly and strongly negatively correlated with the magnitude of visual impairment reported by the subjects. This neurophysiological approach to image quality assessment may potentially lead to a more objective evaluation, as behavioral approaches suffer from drawbacks such as biases, inter-subject variances and limitations to test duration.

This paper reports an empirical investigation into increasing the efficiency of subjective data collection by reducing the length of test sequences below the recommended 10 seconds. Twenty-four observers viewed four 10 second reference sequences, in addition to four truncated versions of each: 7 seconds; 5 seconds; 3 seconds and 1.5 seconds. Results indicated that, compared to the 10 second sequences, the ability of observers to identify compression artefacts was significantly reduced only when viewing the 1.5 second sequences. These results indicate that, when using the DSCQS methodology, a significant benefit in the efficiency of subjective data collection can be gained by reducing the length of test sequences to potentially as low as 3 seconds, without a significant impact upon reliability.

This paper proposes an efficient Scalable High efficiency Video Coding (SHVC) to High Efficiency Video Coding (HEVC) transcoder, which can reduce the transcoding complexity significantly, and provide a desired trade-off between the transcoding complexity and the transcoded video quality. To reduce the transcoding complexity, some of coding information, such as coding unit (CU) depth, prediction mode, merge mode, motion vector information, intra direction information and transform unit (TU) depth information, in the SHVC bitstream are mapped and transcoded to single layer HEVC bitstream. One major difficulty in transcoding arises when trying to reuse the motion information from SHVC bitstream since motion vectors referring to inter-layer reference (ILR) pictures cannot be reused directly in transcoding. Reusing motion information obtained from ILR pictures for those prediction units (PUs) will reduce the complexity of the SHVC transcoder greatly but a significant reduction in the quality of the picture is observed. Pictures corresponding to the intra refresh pictures in the base layer (BL) will be coded as P pictures in enhancement layer (EL) in the SHVC bitstream; and directly reusing the intra information from the BL for transcoding will not get a good coding efficiency. To solve these problems, various transcoding technologies are proposed. The proposed technologies offer different trade-offs between transcoding speed and transcoding quality. They are implemented on the basis of reference software SHM-6.0 and HM-14.0 for the two layer spatial scalability configuration. Simulations show that the proposed SHVC software transcoder reduces the transcoding complexity by up to 98-99% using low complexity transcoding mode when compared with cascaded re-encoding method. The transcoder performance at various bitrates with different transcoding modes are compared in terms of transcoding speed and transcoded video quality.

Fast prediction models of local distortion visibility and local quality can potentially make modern spatiotemporally adaptive coding schemes feasible for real-time applications. In this paper, a fast convolutional-neural- network based quantization strategy for HEVC is proposed. Local artifact visibility is predicted via a network trained on data derived from our improved contrast gain control model. The contrast gain control model was trained on our recent database of local distortion visibility in natural scenes [Alam et al. JOV 2014]. Further- more, a structural facilitation model was proposed to capture effects of recognizable structures on distortion visibility via the contrast gain control model. Our results provide on average 11% improvements in compression efficiency for spatial luma channel of HEVC while requiring almost one hundredth of the computational time of an equivalent gain control model. Our work opens the doors for similar techniques which may work for different forthcoming compression standards.

This paper presents a comparison-based test of the objective compression performance of the High Efficiency Video Coding (HEVC) standard, its format range extensions (RExt), and its draft screen content coding extensions (SCC). The current dominant standard, H.264/MPEG-4 AVC, is used as an anchor reference in the comparison. The conditions used for the comparison tests were designed to reflect relevant application scenarios and to enable a fair comparison to the maximum extent feasible – i.e., using comparable quantization settings, reference frame buffering, intra refresh periods, rate-distortion optimization decision processing, etc. It is noted that such PSNR-based objective comparisons generally provide more conservative estimates of HEVC benefit than are found in subjective studies. The experimental results show that, when compared with H.264/MPEG-4 AVC, HEVC version 1 provides a bit rate savings for equal PSNR of about 23% for all-intra coding, 34% for random access coding, and 38% for low-delay coding. This is consistent with prior studies and the general characterization that HEVC can provide about a bit rate savings of about 50% for equal subjective quality for most applications. The HEVC format range extensions provide a similar bit rate savings of about 13–25% for all-intra coding, 28–33% for random access coding, and 32–38% for low-delay coding at different bit rate ranges. For lossy coding of screen content, the HEVC screen content coding extensions achieve a bit rate savings of about 66%, 63%, and 61% for all-intra coding, random access coding, and low-delay coding, respectively. For lossless coding, the corresponding bit rate savings are about 40%, 33%, and 32%, respectively.

The H.265/MPEG-H High Efficiency Video Coding (HEVC) standard provides a significant increase in coding efficiency compared to its predecessor, the H.264/MPEG-4 Advanced Video Coding (AVC) standard, which however comes at the cost of a high computational burden for a compliant encoder. Motion estimation (ME), which is a part of the inter-picture prediction process, typically consumes a high amount of computational resources, while significantly increasing the coding efficiency. In spite of the fact that both H.265/MPEG-H HEVC and H.264/MPEG-4 AVC standards allow processing motion information on a fractional sample level, the motion search algorithms based on the integer sample level remain to be an integral part of ME. In this paper, a flexible integer sample ME framework is proposed, thereby allowing to trade off significant reduction of ME computation time versus coding efficiency penalty in terms of bit rate overhead. As a result, through extensive experimentation, an integer sample ME algorithm that provides a good trade-off is derived, incorporating a combination and optimization of known predictive, pattern-based and early termination techniques. The proposed ME framework is implemented on a basis of the HEVC Test Model (HM) reference software, further being compared to the state-of-the-art fast search algorithm, which is a native part of HM. It is observed that for high resolution sequences, the integer sample ME process can be speed-up by factors varying from 3.2 to 7.6, resulting in the bit-rate overhead of 1.5% and 0.6% for Random Access (RA) and Low Delay P (LDP) configurations, respectively. In addition, the similar speed-up is observed for sequences with mainly Computer-Generated Imagery (CGI) content while trading off the bit rate overhead of up to 5.2%.

The new state-of-the-art video coding standard, H.265/HEVC, has been finalized in 2013 and it achieves roughly 50% bit rate saving compared to its predecessor, H.264/MPEG-4 AVC. This paper provides the evidence that there is still potential for further coding efficiency improvements. A brief overview of HEVC is firstly given in the paper. Then, our improvements on each main module of HEVC are presented. For instance, the recursive quadtree block structure is extended to support larger coding unit and transform unit. The motion information prediction scheme is improved by advanced temporal motion vector prediction, which inherits the motion information of each small block within a large block from a temporal reference picture. Cross component prediction with linear prediction model improves intra prediction and overlapped block motion compensation improves the efficiency of inter prediction. Furthermore, coding of both intra and inter prediction residual is improved by adaptive multiple transform technique. Finally, in addition to deblocking filter and SAO, adaptive loop filter is applied to further enhance the reconstructed picture quality. This paper describes above-mentioned techniques in detail and evaluates their coding performance benefits based on the common test condition during HEVC development. The simulation results show that significant performance improvement over HEVC standard can be achieved, especially for the high resolution video materials.

In this paper, several coding tools are evaluated on top of the HEVC version 1. Among them there are straightforward extension of HEVC coding tools (such as Coding Unit size enlarging, fine granularity of Intra prediction angles) and algorithms that have been studied during HEVC development (such as secondary transform, multi-hypothesis CABAC, multi-parameter Intra prediction, bidirectional optical flow). Most of them improve performance of Intra coding. Minor adjustment to the final version of HEVC standard was done for efficient harmonization of the proposed coding tools with HEVC. Performance improvement observed from investigated tools is up to 7,1%, 9,9%, 4,5% and 5,7% in all-intra, random access, low-delay B and low-delay P test scenario (using HEVC common test conditions).

Screen content video coding extension of HEVC (SCC) is being developed by Joint Collaborative Team on Video Coding (JCT-VC) of ISO/IEC MPEG and ITU-T VCEG. Screen content usually features a mix of camera captured content and a significant proportion of rendered graphics, text, or animation. These two types of content exhibit distinct characteristics requiring different compression scheme to achieve better coding efficiency. This paper presents an efficient block matching schemes for coding screen content to better capture the spatial and temporal characteristics. The proposed schemes are mainly categorized as a) hash based global region block matching for intra block copy b) selective search based local region block matching for inter frame prediction c) hash based global region block matching for inter frame prediction. In the first part, a hash-based full frame block matching algorithm is designed for intra block copy to handle the repeating patterns and large motions when the reference picture constituted already decoded samples of the current picture. In the second part, a selective local area block matching algorithm is designed for inter motion estimation to handle sharp edges, high spatial frequencies and non-monotonic error surface. In the third part, a hash based full frame block matching algorithm is designed for inter motion estimation to handle repeating patterns and large motions across the temporal reference picture. The proposed schemes are compared against HM-13.0+RExt-6.0, which is the state-of-art screen content coding. The first part provides a luma BD-rate gains of -26.6%, -15.6%, -11.4% for AI, RA and LD TGM configurations. The second part provides a luma BD-rate gains of -10.1%, -12.3% for RA and LD TGM configurations. The third part provides a luma BD-rate gains of -12.2%, -11.5% for RA and LD TGM configurations.

Google started an opensource project, entitled the WebM Project, in 2010 to develop royaltyfree video codecs for the web. The present generation codec developed in the WebM project called VP9 was finalized in mid2013 and is currently being served extensively by YouTube, resulting in billions of views per day. Even though adoption of VP9 outside Google is still in its infancy, the WebM project has already embarked on an ambitious project to develop a next edition codec VP10 that achieves at least a generational bitrate reduction over the current generation codec VP9. Although the project is still in early stages, a set of new experimental coding tools have already been added to baseline VP9 to achieve modest coding gains over a large enough test set. This paper provides a technical overview of these coding tools.

With the increasing popularity of video streaming services and multimedia sharing via social networks, there is a need to protect the multimedia from malicious use. An attacker may use steganography and watermarking techniques to embed malicious content, in order to attack the end user. Most of the attack algorithms are robust to basic image processing techniques such as filtering, compression, noise addition, etc. Hence, in this article two novel, real-time, defense techniques are proposed: Smart threshold and anomaly correction. Both techniques operate at the DCT domain, and are applicable for JPEG images and H.264 I-Frames. The defense performance was evaluated against a highly robust attack, and the perceptual quality degradation was measured by the well-known PSNR and SSIM quality assessment metrics. A set of defense techniques is suggested for improving the defense efficiency. For the most aggressive attack configuration, the combination of all the defense techniques results in 80% protection against cyber-attacks with PSNR of 25.74 db.

MPEG has produced standards that have provided the industry with the best video compression technologies. In order to address the diversified needs of the Internet, MPEG issued the Call for Proposals (CfP) for internet video coding in July, 2011. It is anticipated that any patent declaration associated with the Baseline Profile of this standard will indicate that the patent owner is prepared to grant a free of charge license to an unrestricted number of applicants on a worldwide, non-discriminatory basis and under other reasonable terms and conditions to make, use, and sell implementations of the Baseline Profile of this standard in accordance with the ITU-T/ITU-R/ISO/IEC Common Patent Policy. Three different codecs had responded to the CfP, which are WVC, VCB and IVC. WVC was proposed jointly by Apple, Cisco, Fraunhofer HHI, Magnum Semiconductor, Polycom and RIM etc. it’s in fact AVC baseline. VCB was proposed by Google, and it’s in fact VP8. IVC was proposed by several Universities (Peking University, Tsinghua University, Zhejiang University, Hanyang University and Korea Aerospace University etc.) and its coding tools was developed from Zero. In this paper, we give an overview of the coding tools in IVC, and evaluate its performance by comparing it with WVC, VCB and AVC High Profile.

A significant number of pictures are posted to social media sites or exchanged through instant messaging and cloud-based sharing services. Most social media services offer a range of access control mechanisms to protect users privacy. As it is not in the best interest of many such services if their users restrict access to their shared pictures, most services keep users' photos unprotected which makes them available to all insiders. This paper presents an architecture for a privacy-preserving photo sharing based on an image scrambling scheme and a public key infrastructure. A secure JPEG scrambling is applied to protect regional visual information in photos. Protected images are still compatible with JPEG coding and therefore can be viewed by any one on any device. However, only those who are granted secret keys will be able to descramble the photos and view their original versions. The proposed architecture applies an attribute-based encryption along with conventional public key cryptography, to achieve secure transmission of secret keys and a fine-grained control over who may view shared photos. In addition, we demonstrate the practical feasibility of the proposed photo sharing architecture with a prototype mobile application, ProShare, which is built based on iOS platform.

The difficulty of parallelizing entropy coding is increasingly limiting the data throughputs achievable in media compression. In this work we analyze what are the fundamental limitations, using finite-state-machine models for identifying the best manner of separating tasks that can be processed independently, while minimizing compression losses. This analysis confirms previous works showing that effective parallelization is feasible only if the compressed data is organized in a proper way, which is quite different from conventional formats. The proposed new formats exploit the fact that optimal compression is not affected by the arrangement of coded bits, but it goes further in exploiting the decreasing cost of data processing and memory. Additional advantages include the ability to use, within this framework, increasingly more complex data modeling techniques, and the freedom to mix different types of coding. We confirm the parallelization effectiveness using coding simulations that run on multi-core processors, and show how throughput scales with the number of cores, and analyze the additional bit-rate overhead.

Compression of 3D object based video is relevant for 3D Immersive applications. Nevertheless, the perceptual aspects of the degradation introduced by codecs for meshes and point clouds are not well understood. In this paper we evaluate the subjective and objective degradations introduced by such codecs in a state of art 3D immersive virtual room. In the 3D immersive virtual room, users are captured with multiple cameras, and their surfaces are reconstructed as photorealistic colored/textured 3D meshes or point clouds. To test the perceptual effect of compression and transmission, we render degraded versions with different frame rates in different contexts (near/far) in the scene. A quantitative subjective study with 16 users shows that negligible distortion of decoded surfaces compared to the original reconstructions can be achieved in the 3D virtual room. In addition, a qualitative task based analysis in a full prototype field trial shows increased presence, emotion, user and state recognition of the reconstructed 3D Human representation compared to animated computer avatars.

As a part of the HDD manufacturing process, HDD stamped base, an exterior container, is one of the most essential components in which various parts become assembled to compose a hard disk drive (HDD). Height errors that are caused by pressing, breaking or cracking can occur on the base, because it is designed by a stamping method. In order to detect the height errors, the inspection process is essential in the production fields. In the current industry, CMM (Coordinate Measurement Machine) is one of the representative machines that inspect certain regions on the product. The machine probes a designated point by an operator and judges the defect by comparing the height of the point to the originally designed height. However, the method takes much time to inspect each designated point resulting in a total of 17 minutes. In order to reduce the total inspection time, we propose an inspection method using 3D point cloud data acquired from a holographic sensor. To compare the height from acquired 3D point cloud data with the one from the originally designed CAD data, the exact point cloud registration is important. There are differences between 2D image registration and 3D point cloud registration, such as translation on each plane, rotation, tilt, and nonlinear transformations. The relationship between the acquired 3D point cloud data and the originally designed CAD data can be obtained by projective transformation. If the projective transformation matrix between the two is obtained, 3D point cloud data registration can be performed. In order to calculate 3D projective transformation matrix, corresponding points between 3D point cloud data and CAD data are required. To find the corresponding points, we use the height map which is projected from 3D point cloud data onto XY plane. The height map has pixel intensity from the height value of each point. If the height maps from 3D point cloud data and CAD data are matched, corresponding points can be estimated. As one of the features of the HDD stamped base, there are multiple circles on the base. In this paper, we find the corresponding points between 3D point cloud data and CAD data using circle fitting, and obtain 2D Affine transformation matrix from the corresponding points. By applying 2D Affine transformation matrix to height map, the corresponding points on the 3D coordinate can be obtained. Using such points, we propose the method designed to achieve 3D projective transformation matrix. To find the proper 3D projective transformation matrix, we formulate a cost function which uses the relationship of the corresponding points. Also the proper 3D projective transformation matrix can be calculated by minimizing the cost function. Then the 3D point cloud data can be matched to CAD data and the height values of each point of 3D point cloud can be compared to the CAD data.

Common tracking systems are usually affected by environmental and technical interferences such as non-uniform illumination, sensors' noise, geometrical scene distortion, etc. Among these issues, the former is particularly interesting because it destroys important spatial characteristics of objects in observed scenes. We propose a two-step tracking algorithm: first, preprocessing locally normalizes the illumination difference between the target object and observed frames; second, the normalized object is tracked by means of a designed template structure based on histograms of oriented gradients and kinematic prediction model. The algorithm performance is tested in terms of recognition and localization errors in real scenarios. In order to achieve high rate of the processing, we use GPU parallel processing technologies. Finally, our algorithm is compared with other state-of-the-art trackers.

The multiple-input multiple-output (MIMO) system with the use of transmit and receive antenna arrays achieves diversity and array gains via transmit beamforming. Due to the absence of full channel state information (CSI) at the transmitter, the transmit beamforming vector can be quantized at the receiver and sent back to the transmitter by a low-rate feedback channel, called limited feedback beamforming. One of the key roles of Vector Quantization (VQ) is how to generate a good codebook such that the distortion between the original image and the reconstructed image is the minimized. In this paper, a novel adaptive codebook selection scheme for image classification is proposed with taking both spatial and temporal correlation inherent in the channel into consideration. The new codebook selection algorithm is developed to select two codebooks from the discrete Fourier transform (DFT) codebook, the generalized Lloyd algorithm (GLA) codebook and the Grassmannian codebook to be combined and used as candidates of the original image and the reconstructed image for image transmission. The channel is estimated and divided into four regions based on the spatial and temporal correlation of the channel and an appropriate codebook is assigned to each region. The proposed method can efficiently reduce the required information of feedback under the spatially and temporally correlated channels, where each region is adaptively. Simulation results show that in the case of temporally and spatially correlated channels, the bit-error-rate (BER) performance can be improved substantially by the proposed algorithm compared to the one with only single codebook.

Infrared resistor arrays perform a vital role in the hardware in the loop testing of infrared seekers. Infrared resistor arrays composed of large numbers of suspended resistor elements are commonly used to produce dynamic two-dimensional images of infrared radiation. Due to inconsistencies in the fabrication process of the resistor arrays, the temperature each resistor elements reaches for a given input voltage is variable and this leads to more significant radiance differences, these differences result in spatially-distributed radiance non-uniformity. Therefore, in order to obtain an available infrared image, non-uniformity correction (NUC) is necessary. In this paper, the non-uniformity characters of the infrared resistor arrays are analyzed base on measured data and then an improved sparse grid method for engineering are discussed and analyzed. First of all, the NUC camera has a strong influence on the effectiveness of the infrared resistor arrays NUC procedure. According to the actual fact and the laboratory condition, we presented an alternative method for collecting resistor arrays intended to reduce the influence causing by the NUC camera. Secondly, based on the measured non-uniformity data, we obtain the response characteristics of the infrared resistor arrays. In each gray level, we take two points or several points correction algorithm to calculate the gain data and the offset data, and then the linear look-up table is established. Finally, through MATLAB we develop the correction software, and we can obtain the driving output conveniently. The result shows that the image quality has a remarkable improvement after non-uniformity correction, the non-uniformity correction flow and algorithm preferably satisfies the requirement of the high confidence infrared imaging simulation.

The pupil size of the human eye has a large effect in the image quality due to inherent aberrations. Several studies have been performed to calculate its size relative to the luminance as well as considering other factors, i.e., age, size of the adapting field and mono and binocular vision. Moreover, ideal lighting conditions are known, but software suited to our specific requirements, low cost and low computational consumption, in order to simulate radiation adaptation and image formation in the retina with ideal lighting conditions has not yet been developed. In this work, a database is created consisting of 70 photographs corresponding to the same scene with a fixed target at different times of the day. By using this database, characteristics of the photographs are obtained by measuring the luminance average initial threshold value of each photograph by means of an image histogram. Also, we present the implementation of a digital filter for both, image processing on the threshold values of our database and generating output images with the threshold values reported for the human eye in ideal cases. Some potential applications for this kind of filters may be used in artificial vision systems.