Adaptive media playout techniques are used to avoid buffer underflow in a dynamic streaming environment where the
available bandwidth may be fluctuating. In this paper we report human perceptions from audio quality studies that we
performed on speech and music samples for adaptive audio playout. Test methods based on ITU-R BS. 1534-1
recommendation were used. Studies were conducted for both slow playout and fast playout. Two scales - a coarse scale
and a finer scale was used for the slow and fast audio playout factors. Results from our study can be used to determine
acceptable slow and fast playout factors for speech and music content. An adaptive media playout algorithm could use
knowledge of these upper and lower bounds on playback speeds to decide its adaptive playback schedule.
Hypothetical Reference Decoder is a hypothetical decoder model that specifies constraints on the variability of
conforming network abstraction layer unit streams or conforming byte streams that an encoding process may produce.
High Efficiency Video Coding (HEVC) builds upon and improves the design of the generalized hypothetical reference
decoder of H.264/ AVC. This paper describes some of the main improvements of hypothetical reference decoder of
Perceived video quality studies were performed on a number of key classes of noise removal algorithms to determine
viewer preference. The noise removal algorithm classes represent increase in complexity from linear filter to nonlinear
filter to adaptive filter to spatio-temporal filter. The subjective results quantify the perceived quality improvements that
can be obtained with increasing complexity. The specific algorithm classes tested include: linear spatial one channel
filter, nonlinear spatial two-channel filter, adaptive nonlinear spatial filter, multi-frame spatio-temporal adaptive filter.
All algorithms were applied on full HD (1080P) content. Our subjective results show that spatio-temporal (multi-frame)
noise removal algorithm performs best amongst the various algorithm classes. The spatio-temporal algorithm
improvement compared to original video sequences is statistically significant. On the average, noise-removed video
sequences are preferred over original (noisy) video sequences. The Adaptive bilateral and non-adaptive bilateral two
channel noise removal algorithms perform similarly on the average thus suggesting that a non-adaptive parameter tuned
algorithm may be adequate.
We designed a series of experiments to measure user preference for the noise-detail tradeoff, including tests of the
assumption that all true image detail is preferred. We generated samples with noise-detail tradeoff by designing a
sequence of coring filters with increasing strength. A user study method is developed using magnitude estimation
approach. In the first experiment the coring filter sequence is applied to original video samples without any additional
noise. It is observed that the subjective quality score increases as coring strength is increased, reaches a peak and then
decreases. Thus users prefer slightly cored images compared to original images. In the second experiment the coring
filter sequence is applied to video samples with additive noise of different strength. It is observed that the most preferred
coring strength increases as the amount of noise in the image increases. The results from our experiments can be used to
design parameters for various image/ video post-processing and noise removal algorithms.
We define experiments to measure vernier acuity caused by synchronization mismatch for moving images.
The experiments are used to obtain synchronization mismatch acuity threshold as a function of object
velocity and as a function of occlusion or gap width. Our main motivation for measuring the
synchronization mismatch vernier acuity is its relevance in the application of tiled display systems which
create a single contiguous image using individual discrete panels arranged in a matrix with each panel
utilizing a distributed synchronization algorithm to display parts of the overall image. We also propose a
subjective assessment method for perception evaluation of synchronization mismatch for large ultra high
resolution tiled displays. For this we design a synchronization mismatch measurement test video set for
various tile configurations for various inter-panel synchronization mismatch values. The proposed method
for synchronization mismatch perception can evaluate tiled displays with or without tile bezels. The results
from this work can help during design of low cost tiled display systems which utilize distributed
synchronization mechanisms for a contiguous or bezeled image display.
We propose an adaptive timeline aware client controlled HTTP streaming method to
improve performance in a situation where the client has buffer constraints and is
connected to the network with a constrained bandwidth link. The proposed approach
uses the HTTP/1.1 byte ranges feature or URL parameters to achieve a better HTTP
streaming performance. The proposed method does not require any change to the HTTP
server side. It can support pausing the HTTP stream without any network data transfer
occurring during the paused state. It does not rely on the TCP flow control and so it can
work with any TCP/IP stack. The proposed approach allows client to intelligently employ
single or multiple concurrent HTTP connections to receive streaming media. The client
adaptively switches between using single and multiple concurrent HTTP connections
based on the streaming media reception status with respect to wall-clock timeline and
the media playout timeline.
We propose a method for selective frame dropping based on hypothetical reference
decoder buffer model for initial buffering delay reduction. The client side buffering
consists of two logical buffers: a de-jitter buffer and a pre-decoder buffer. To playback an
encoded bit-stream without underflow the client must do a minimum initial buffering. This
minimum initial buffering is a property of the bit-stream. The minimum initial buffering
relates to the pre-decoder buffer. In addition the client can do additional initial buffering
to handle network jitter and other bandwidth variations. Our proposed approach relates
to reducing the minimum initial buffering delay for an already encoded bit-stream. We
propose a method for selectively dropping frames to reduce the amount of initial
buffering the client needs to do to avoid underflow during the streaming. Our proposed
method is especially applicable to pre-stored content. The method is also particularly
useful for variable bit-rate (VBR) encoded media. The method can be used by a
streaming server. Alternatively the method can be implemented by a trans-rater/ transcoder.
In a preferred embodiment our method can be applied in advance on a pre-stored
bit-stream to decide which frames to drop to reduce the required minimum initial
We propose a method to prevent receiver buffer underflow for AV streaming media under varying channel
conditions using a variable scale factor adaptive media playout algorithm. Our proposed algorithm
dynamically calculates a slow or a fast playout factor based on the current buffer state, target buffer level,
past history of media data reception, estimate of future data arrival, content characteristics and the estimated
current network conditions. As a result our algorithm results in prevention of underflow in situations where
prior art approach can still result in underflow. Our algorithm can also avoid oscillations between slow and
fast playout. Also variants of our algorithm can result in more smooth transition to normal playout from
adaptive playout stage thus improving the perceptual user experience. The proposed algorithm can also be
used to reduce initial buffering latency at the start of a media stream playback while achieving same
robustness by reaching the desired target buffer level. We present a number of network simulation results for
our proposed approach.
In this paper we present an automatic enhanced video display and navigation capability for networked streaming video and networked video playlists. Our proposed method uses Synchronized Multimedia Integration Language (SMIL) as presentation language and Real Time Streaming Protocol (RTSP) as network remote control protocol to automatically generate a "enhanced video strip" display for easy navigation. We propose and describe two approaches - a smart client approach and a smart server approach. We also describe a prototype system implementation of our proposed approach.
JPEG2000 is a new image coding standard and system. A client side cache model is defined in JPEG2000 Part 9 - Interactivity tools, APIs and protocols (JPIP). The JPIP standard does not define how the client side and the server side internally keeps track of the client side cache. In this paper we propose XML based JPIP client side cache model management. We propose to use XML for JPIP client side cache model management because XML is standardized and easily exchangeable approach. Since the JPIP cache model descriptors follow hierarchical structure, this can be very well represented using XML. Using our proposed approach, the client and/or server side can use XML DOM API to easily add, delete, and update its cache model view of the client cache. We define a XML DTD and Schema for the proposed XML based client side cache model management.
We address the problem of robust streaming of high-quality video over wireless local area networks in a home environment. By robust streaming, we mean maintaining the highest possible video quality and preventing interruptions to the video under varying bandwidth conditions, which may be due to distance, interference, obstructions, and existence of multiple streams. We propose an application-layer approach where we provide algorithms for dynamic on-line network bandwidth estimation and dynamic on-line adaptation of video rate according to the available network bandwidth. The proposed system employs a packet scheduler, and a video rate control and adaptation mechanism at the sender, and bandwidth measurement and feedback mechanisms at the receiver. Our bandwidth estimation approach uses the actual video data in real time by transmitting it in packet bursts; hence, separate test traffic is not required. Since the proposed method operates at the application layer, it is flexible and applicable to different local area network types and implementations. We propose an extension to multiple streams by providing an algorithm for joint rate allocation to multiple video streams over a network enabling network-adaptive simultaneous streaming of high-quality video.
SC876: Unified Modeling Language (UML) for Researchers and Engineers
The Unified Modeling Language (UML) is a language for specifying, modeling, constructing, and documenting systems. UML specification is officially defined by the Object Management Group (OMG). UML is the industry standard language and the preferred choice for modeling software and systems.
This course provides researchers and engineers with knowledge of UML. Practical examples corresponding to modeling and developing code for an IETF RFC will be used to help researchers learn UML 2.1 language. All different types of UML diagrams including structure diagrams, behavior diagrams, interaction diagrams will be covered. Forward and reverse code engineering with UML, along with practical exercises, will be covered in this course.