Forms of surveillance are very quickly becoming an integral part of crime control policy, crisis management, social control theory and community consciousness. In turn, it has been used as a simple and effective solution to many of these problems. However, privacy-related concerns have been expressed over the development and deployment of this technology. Used properly, video cameras help expose wrongdoing but typically come at the cost of privacy to those not involved in any maleficent activity.
This work describes the design and implementation of a real-time, privacy-protecting data collection infrastructure that fuses additional sensor information (e.g. Radio-frequency) with video streams and an access control framework in order to make decisions about how and when to display the individuals under surveillance. This video surveillance system is a particular instance of our data collection framework, and here we describe in detail the real-time video processing techniques used in order to achieve tracking of users in pervasive spaces while utilizing the additional sensor data provided by the various instrumented sensors. In particular, we discuss background modeling techniques, object tracking and implementation techniques that pertain to the overall development of this system.