Tele-immersive systems can improve productivity and aid communication by allowing distributed parties to exchange information via a shared immersive experience. The TEEVE research project at the University of Illinois at Urbana-Champaign and the University of California at Berkeley seeks to foster the development and use of tele-immersive environments by a holistic integration of existing components that capture, transmit, and render three-dimensional (3D) scenes in real time to convey a sense of immersive space. However, the transmission of 3D video poses significant challenges. First, it is bandwidth-intensive, as it requires the transmission of multiple large-volume 3D video streams. Second, existing schemes for 2D color video compression such as MPEG, JPEG, and H.263 cannot be applied directly because the 3D video data contains depth as well as color information. Our goal is to explore from a different angle of the 3D compression space with factors including complexity, compression ratio, quality, and real-time performance. To investigate these trade-offs, we present and evaluate two simple 3D compression schemes. For the first scheme, we use color reduction to compress the color information, which we then compress along with the depth information using zlib. For the second scheme, we use motion JPEG to compress the color information and run-length encoding followed by Huffman coding to compress the depth information. We apply both schemes to 3D videos captured from a real tele-immersive environment. Our experimental results show that: (1) the compressed data preserves enough information to communicate the 3D images effectively (min. PSNR > 40) and (2) even without inter-frame motion estimation, very high compression ratios (avg. > 15) are achievable at speeds sufficient to allow real-time communication (avg. ≈ 13 ms per 3D video frame).
Live peer to peer (P2P) media streaming faces many challenges such as peer unreliability and bandwidth heterogeneity. To effectively address these challenges, general "mesh" based P2P streaming architectures have recently been adopted. Mesh-based systems allow peers to aggregate bandwidth from multiple neighbors, and dynamically adapt to changing network conditions and neighbor failures. However, a drawback of mesh-based overlays is that it is difficult to guarantee network connectivity in a distributed fashion, especially when network locality needs to be optimized. This paper introduces a new P2P streaming framework called DagStream, which (1) organizes peers into a directed acyclic graph (DAG) where each node maintains at least k parents, thus has provable network connectivity (and hence failure resilience), and (2) enables peers to quickly achieve locality awareness in a distributed fashion, thus ensures efficient network resource usage. Our experiment results in both simulation and wide area environment show that with our DagStream protocol, peers can quickly self-organize into a locality aware DAG. Further, by selecting additional parents as needed, peers can achieve good streaming quality commensurate with their downlink bandwidth.
This paper presents a framework for preventing both selfishness and denial-of-service attacks in peer-to-peer media streaming systems. Our framework, called Oversight, achieves prevention of these undesirable activities by running a separate peer-to-peer download rate enforcement protocol along with the underlying peer-to-peer media streaming protocol. This separate Oversight protocol enforces download rate limitations on each participating peer. These limitations prevent selfish or malicious nodes from downloading an overwhelming amount of media stream data that could potentially exhaust the entire system. Since Oversight is based on a peer-to-peer architecture, it can accomplish this enforcement functionality in a scalable, efficient, and decentralized way that fits better with peer-to-peer media streaming systems compared to other solutions based on central server architectures. As peer-to-peer media streaming systems continue to grow in popularity, the threat of selfish and malicious peers participating in such large peer-to-peer networks will continue to grow as well. For example, since peer-to-peer media streaming systems allow users to send small request messages that result in the streaming of large media objects, these systems provide an opportunity for malicious users to exhaust resources in the system with little effort expended on their part. However, Oversight addresses these threats associated with selfish or malicious peers who cause such disruptions with excessive download requests. We evaluated our Oversight solution through simulations and our results show that applying Oversight to peer-to-peer media streaming systems can prevent both selfishness and denial-of-service attacks by effectively limiting the download rates of all nodes in the system.
By composing distributed, autonomous services dynamically to provide new functionalities, service composition provides an attractive way for customized multimedia content production and delivery. Previous
research work has addressed various aspects of service composition such as composibility, QoS-awareness, and load balancing. However, most of the work has focused on applications where data flow from a single source is processed by intermediate services and
then delivered to a single destination. In this paper, we address the service composition problem for advanced multimedia applications where data flows from multiple content sources are processed and aggregated into a composite flow, which is then delivered to one or more destinations, possibly after being customized for each receiver. We formally define the problem and prove its NP hardness. We also design a heuristic algorithm to solve the problem. Our algorithm has the following attractive features: (1) it is effective at finding low cost composition solutions; (2) it has the ability to trade off computation overhead for better results; (3) it is
efficient and can scale to relatively large number of network nodes and component services.
Nowadays, multiple video cameras are employed for live broadcast and recording of almost all major social events in the TV industry, and all these camera streams have to be aggregated and rendered into one video program for audiences. While this content composition process aims at presenting the most interesting perspective of an event, it leads to the problem of how to fully customize the finally composed video program to different audience interests without requiring too much input from the audience. The goal of this work is to solve this problem by proposing the Automatic Video Production with User Customization (AVPUC) system that separates the video stream interestingness comparison from video program rendering to provide space for maximized customization. The unique feature of the AVPUC systems is that Human-controlled video selection and automatic video evaluation are combined to support video content customization and reduce redundant audience inputs. Preliminary evaluation results confirm that AVPUC's capturing-evaluation-render model for video production improves audiences' satisfaction for customized multi-perspective viewing of social events.
Our previous MMCN03 paper reported a cross-layer adaptation framework, GRACE-1, that coordinates the adaptation of CPU frequency/voltage, CPU scheduling, and application quality. GRACE-1 assumes that all application processes (or threads) are independent from each other and adapt individually. This assumption, however, is invalid for multi-threaded applications that include dependent and cooperative processes. To support the joint performance requirements
of such dependent processes, this paper extends GRACE-1 with a process group management mechanism. The enhanced framework, called <i>GRACE-grp</i>, introduces a new OS abstraction, <i>group control block</i>, to provide the OS-level recognition and support of process groups. Through a group control block, dependent processes can explicitly set up a group and specify their dependency in the OS kernel. Consequently, GRACE-grp schedules and adapts them in a synchronized and consistent manner, thereby delivering joint performance guarantees. We have implemented and evaluated the GRACE-grp framework. Our experimental results show that compared to GRACE-1, GRACE-grp provides better support for the joint quality of dependent processes and reduces CPU energy consumption by 6.2% to 8.7% for each process group.
To achieve scalable and efficient on-demand media distribution, existing solutions mainly make use of multicast as underlying data delivery support. However, due to the intrinsic conflict between the synchronous multicast transmission and the asynchronous nature of on-demand media delivery, these solutions either suffer from large playback delay or require clients to be capable of receiving multiple streams simultaneously and buffering large amount of data. Moreover, the limited and slow deployment of IP multicast hinders their application on the Internet.
To address these problems, we propose asynchronous multicast, which is able to directly support on-demand data delivery. Asynchronous multicast is an application level solution. When it is deployed on a proxy network, stable and scalable media distribution can be achieved. In this paper, we focus on the problem of efficient media distribution. We first propose a temporal dependency model to formalize the temporal relations among asynchronous media requests. Based on this model, we propose the concept of Media Distribution Graph (MDG), which represents the dependencies among all asynchronous requests in the proxy network. Then we formulate the problem of efficient media distribution as finding Media Distribution Tree (MDT), which is the minimal spanning tree on MDG. Finally, we present our algorithm for MDT construction/maintenance. Through theoretical analysis and experimental study, we claim that our solution can meet the goals of scalability, efficiency and low access latency at the same time.
Mobile multimedia systems must provide application quality of service (QoS) in the presence of dynamically varying and multiple resource constraints (e.g., variations in available CPU time, energy, and bandwidth). Researchers have therefore proposed adaptive systems that can respond to changing resource availability and application demands. All system layers can benefit from adaptation, but fully exploiting these benefits requires a new cross-layer adaptation framework to coordinate the adaptations in the different layers. This paper presents such a framework and its first prototype, called GRACE-1. The framework supports application QoS under CPU and energy constraints via coordinated adaptation in the hardware, OS, and application layers. Specifically, GRACE-1 uses <i>global</i> adaptation to handle large and long-term variations, setting application QoS, CPU allocation, and CPU frequency/voltage to qualitatively new levels. In response to small and temporary variations, it uses <i>local</i> adaptation within each layer. We have implemented the GRACE-1 prototype on an HP laptop with an adaptive processor. Our experimental results show that, compared to previous approaches that exploit adaptation in only some of the layers or in an uncoordinated way, GRACE-1 can provide higher overall system utility in several cases.
With the deployment of multimedia service proxies at different locations in the networks, it is possible to create an application-level media service proxy network. Multimedia sources and clients will then be able to connect to this network, and create customized, value-added, and composite media service delivered by one or more proxies in the media service proxy network. In this paper, we focus on the problem of finding multimedia service path in the media service proxy network. Our goal is to find the best path with respect to end-to-end resource availability for each service path request. Our solution includes (1) a mechanism to monitor and propagate resource availability information in the media service proxy network and (2) an algorithm to find the best service path based on the resource monitoring results. Its main features include: (1) the resource monitoring mechanism provides reasonable accuracy and stability, while incurring controlled overhead; (2) the service path finding algorithm finds the best path for each service path request, and achieves high overall success rate among all requests; and (3) it is an application-level solution, and does not require changes to the lower-level network infrastructure.
Accessing remote data is a challenging task in mobile ad hoc networks. Two problems have to be solved: (1) how to learn about available data in the network; and (2) how to access desired data even when the original copy of the data is unreachable. In this paper, we develop an integrated data lookup and replication scheme to solve these problems. In our scheme, a group of mobile nodes collectively host a set of data to improve data accessibility for all members of the group. They exchange data availability information by broadcasting advertising (ad) messages to the group using an adaptive sending rate policy. The ad messages are used by other nodes to derive a local data lookup table, and to reduce data redundancy within a connected group. Our data replication scheme predicts group partitioning based on each node's current location and movement patterns, and replicates data to other partitions before partitioning occurs. Our simulations show that data availability information can quickly propagate throughout the network, and that the successful data access ratio of each node is significantly improved.
In different areas of applications such as education, entertainment, medical surgery, or space shuttle launching, distributed visual tracking systems are of increasing importance. In this paper we describe the design, implementation and evaluation of OmniTrack, a distributed omni-directional visual tracking system, developed at the University of Illinois at Urbana-Champaign, with an Adaptive Middleware Architecture as the core of the system. With respect to both operating systems and network connections, adaptation is of fundamental importance to the tracking system, since it runs in an environment with large performance variations and without support of Quality of Service guarantees.
Nowadays the multimedia technology in distributed environments becomes realistic and the multimedia copyright protection issue becomes more and more important. Various digital watermarking techniques have been proposed in recent years as the methods to protect the copyright of multimedia data. Although, conceptually, these techniques can be easily extended for protecting digital audio data, it is challenging to apply these techniques to MPEG Audio streams because we need to design the watermarking schemes working directly in the compressed data domain. In this paper, we present watermarking methods which will embed the watermark directly into the MPEG audio bit streams rather than going through expensive decoding/encoding process in order to apply watermarking schemes in uncompressed data domain. Among the two presented schemes, one embeds the watermark into the Scale Factors of the MPEG audio streams and another one embeds the watermark into the MPEG encoded samples. Our experimental results show that both methods perform well and the distortion could be controlled at the minimal level. While we use MPEG Audio Layer II streams in our experimental tests, the proposed schemes can be applied to MPEG Audio Layer I and III. Furthermore, by enforcing creation of the watermark through a standard encryption function such as DES, the proposed schemes will be successful in resolving rightful ownership of watermarked MPEG audio.
We present a simple, efficient, and secure multicast protocol with copyright protection in an open and insecure network environment. There is a wide variety of multimedia applications that can benefit from using our secure multicast protocol, e.g., the commercial pay-per-view multicast, or highly secure military intelligence video conference. Our secure multicast protocol is designed to achieve the following goals. (1) It can run in any open network environment. It does not rely on any security mechanism on intermediate network switches or routers. (2) It can be built on top of any existing multicast architecture. (3) Our key distribution protocol is both secure and robust in the presence of long delay or membership message. (4) It can support dynamic group membership, e.g., JOIN/LEAVE/EXPEL operations, in a network bandwidth efficient manner. (5) It can provide copyright protection for the information provider. (6) It can help to identify insiders in the multicast session who are leaking information to the outside world.
To support real-time multimedia applications in wireless packet networks, it is an essential challenge to provide seamless quality of service (QoS) to mobile users. In this paper, we address the problem of real-time multimedia multicast in cellular networks, and present our solution to avoid large QoS fluctuations during handoffs. Specifically, during a multicast session, a mobile host may experience varying packet delay, delay jitter, and channel error when it moves from one cell to another. It is thus desirable that these location-dependent QoS parameters appear as seamless as possible to mobile hosts. We present protocols to achieve a degree of transmission synchronization among multiple cells, so that the delays and delay jitters of each packet to all subscribing mobile hosts do not vary substantially. In addition, we apply Forward Error Correction technique to recover the QoS-mandatory packets from wireless channel errors. We show through analysis and simulation that the mobile hosts will experience brief, smooth, and low packet loss rate handoffs.
In current end systems, multiple flexible, complex and distributed applications concurrently share and compete both end system resource and transmission bandwidth of heterogeneous multi-protocol networks, especially the Internet. Our objective is to enable adaptation awareness in these applications to fully cope with the dynamics in resource availability over the heterogeneous Internet, as well as fluctuations in QoS requirements of the applications themselves. In this paper, we present the theoretical and practical aspects of a Task Control Model implemented in the middleware layer, which applies control theoretical approaches to utilize measurement-based samples monitored in the network traffic, as well as resource and QoS demand dynamics observed in the end systems.
Many current distributed multimedia applications such as video on demand (VOD) are designed, implemented and used on top of general purpose OS and network platforms. Within this 'best-effort' environment, applications must use adaptive synchronization protocols and balance non-deterministic behavior of the underlying OS/network subsystem to achieve user acceptance for lip synchronization of the audio-visual information. We have designed, implemented and validated an adaptive synchronization scheme integrating adaptive services and synchronization protocols to provide lip synchronization within a VOD system. Our results indicate that most of the synchronization skew values are in the user desirable range of ms and that the remaining skew values are in the user acceptable range of ms.