Team activity analysis and recognition based on Kinect depth map and optical imagery techniques

Vinayak Elangovan; Vinod K. Bandaru; Amir Shirkhodaie

doi:10.1117/12.919946

17 May 2012 Team activity analysis and recognition based on Kinect depth map and optical imagery techniques

Vinayak Elangovan, Vinod K. Bandaru, Amir Shirkhodaie

Proceedings Volume 8392, Signal Processing, Sensor Fusion, and Target Recognition XXI; 83920W (2012) https://doi.org/10.1117/12.919946
Event: SPIE Defense, Security, and Sensing, 2012, Baltimore, Maryland, United States

Abstract

Kinect cameras produce low-cost depth map video streams applicable for conventional surveillance systems. However, commonly applied image processing techniques are not directly applicable for depth map video processing. Kinect depth map images contain range measurement of objects at expense of having spatial features of objects suppressed. For example, typical objects' attributes such as textures, color tones, intensity, and other characteristic attributes cannot be fully realized by processing depth map imagery. In this paper, we demonstrate application of Kinect depth map and optical imagery for characterization of indoor and outdoor group activities. A Casual-Events State Inference (CESI) technique is proposed for spatiotemporal recognition and reasoning of group activities. CESI uses an ontological scheme for representation of casual distinctiveness of a priori known group activities. By tracking and serializing distinctive atomic group activities, CESI allows discovery of more complex group activities. A Modified Sequential Hidden Markov Model (MS-HMM) is implemented for trail analysis of atomic events representing correlated group activities. CESI reasons about five levels of group activities including: Merging, Planning, Cooperation, Coordination, and Dispersion. In this paper, we present results of capability of CESI approach for characterization of group activities taking place both in indoor and outdoor. Based on spatiotemporal pattern matching of atomic activities representing a known group activities, the CESI is able to discriminate suspicious group activity from normal activities. This paper also presents technical details of imagery techniques implemented for detection, tracking, and characterization of atomic events based on Kinect depth map and optical imagery data sets. Various experimental scenarios in indoors and outdoors (e.g. loading and unloading of objects, human-vehicle interactions etc.,) are carried to demonstrate effectiveness and efficiency of the proposed model for characterization of distinctive group activities.

Citation Download Citation

Vinayak Elangovan, Vinod K. Bandaru, and Amir Shirkhodaie "Team activity analysis and recognition based on Kinect depth map and optical imagery techniques", Proc. SPIE 8392, Signal Processing, Sensor Fusion, and Target Recognition XXI, 83920W (17 May 2012); https://doi.org/10.1117/12.919946

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available