In this paper we describe an Advanced Digital Video Surveillance system based on TASC-developed object behavior based video analysis and indexing prototype. The advantages of using video analysis in surveillance and physical security applications are twofold. Firstly, the ability to automatically analyze the surveillance video contents facilitates timely detection of events that require immediate attention. Secondly, the amount of video to be archived can be reduced considerably by recording only the portions of video that include behaviors and events of interest from vast amount of surveillance data being collected everyday. Our object-behavior and event based indexing paradigm for video data treats an identifiable object behavior, action or event as the basic indexing unit facilitating efficient querying and report generation as well as derivation of statistical information about the behavior patterns over periods of time. We describe our methodology and present preliminary results in near real time behavior and event detection.