Wide Area Motion Imagery (WAMI) enables image based surveillance of areas that can cover multiple square kilometers. Interpreting and analyzing information from such sources, becomes increasingly time consuming as more data is added from newly developed methods for information extraction. Captured from a moving Unmanned Aerial Vehicle (UAV), the high-resolution images allow detection and tracking of moving vehicles, but this is a highly challenging task. By using a chain of computer vision detectors and machine learning techniques, we are capable of producing high quality track information of more than 40 thousand vehicles per five minutes. When faced with such a vast number of vehicular tracks, it is useful for analysts to be able to quickly query information based on region of interest, color, maneuvers or other high-level types of information, to gain insight and find relevant activities in the flood of information.
In this paper we propose a set of tools, combined in a graphical user interface, which allows data analysts to survey vehicles in a large observed area. In order to retrieve (parts of) images from the high-resolution data, we developed a multi-scale tile-based video file format that allows to quickly obtain only a part, or a sub-sampling of the original high resolution image. By storing tiles of a still image according to a predefined order, we can quickly retrieve a particular region of the image at any relevant scale, by skipping to the correct frames and reconstructing the image. Location based queries allow a user to select tracks around a particular region of interest such as landmark, building or street. By using an integrated search engine, users can quickly select tracks that are in the vicinity of locations of interest. Another time-reducing method when searching for a particular vehicle, is to filter on color or color intensity. Automatic maneuver detection adds information to the tracks that can be used to find vehicles based on their behavior.
Ground surveillance is normally performed by human assets, since it requires visual intelligence. However, especially for military operations, this can be dangerous and is very resource intensive. Therefore, unmanned autonomous visualintelligence systems are desired. In this paper, we present an improved system that can recognize actions of a human and interactions between multiple humans. Central to the new system is our agent-based architecture. The system is trained on thousands of videos and evaluated on realistic persistent surveillance data in the DARPA Mind’s Eye program, with hours of videos of challenging scenes. The results show that our system is able to track the people, detect and localize events, and discriminate between different behaviors, and it performs 3.4 times better than our previous system.
Detection of anomalies (outliers or abnormal instances) is an important element in a range of applications such as
fault, fraud, suspicious behavior detection and knowledge discovery. In this article we propose a new method for
anomaly detection and performed tested its ability to detect anomalous behavior in videos from DARPA's Mind's
Eye program, containing a variety of human activities. In this semi-unsupervised task a set of normal instances
is provided for training, after which unknown abnormal behavior has to be detected in a test set. The features
extracted from the video data have high dimensionality, are sparse and inhomogeneously distributed in the
feature space making it a challenging task. Given these characteristics a distance-based method is preferred, but
choosing a threshold to classify instances as (ab)normal is non-trivial. Our novel aproach, the Adaptive Outlier
Distance (AOD) is able to detect outliers in these conditions based on local distance ratios. The underlying
assumption is that the local maximum distance between labeled examples is a good indicator of the variation in
that neighborhood, and therefore a local threshold will result in more robust outlier detection. We compare our
method to existing state-of-art methods such as the Local Outlier Factor (LOF) and the Local Distance-based
Outlier Factor (LDOF). The results of the experiments show that our novel approach improves the quality of
the anomaly detection.