Our Multi-INT Data Association Tool (MIDAT) learns patterns of life (POL) of a geographical area from video analyst observations called out in textual reporting. Typical approaches to learning POLs from video make use of computer vision algorithms to extract locations in space and time of various activities. Such approaches are subject to the detection and tracking performance of the video processing algorithms. Numerous examples of human analysts monitoring live video streams annotating or “calling out” relevant entities and activities exist, such as security analysis, crime-scene forensics, news reports, and sports commentary. This user description typically corresponds with textual capture, such as chat. Although the purpose of these text products is primarily to describe events as they happen, organizations typically archive the reports for extended periods. This archive provides a basis to build POLs. Such POLs are useful for diagnosis to assess activities in an area based on historical context, and for consumers of products, who gain an understanding of historical patterns. MIDAT combines natural language processing, multi-hypothesis tracking, and Multi-INT Activity Pattern Learning and Exploitation (MAPLE) technologies in an end-to-end lab prototype that processes textual products produced by video analysts, infers POLs, and highlights anomalies relative to those POLs with links to “tracks" of related activities performed by the same entity. MIDAT technologies perform well, achieving, for example, a 90% F1-value on extracting activities from the textual reports.
The concept of trackability is intimately related to the establishment of optimal trade-offs between the nosiness of
the environment, due to poor sensing, and the randomness of the kinematics of the phenomena being examined,
due to poor knowledge of their behaviors.
Classically, a sensor system receives low level data in the form of numerical or analog signals and then through
signal processing produces a high level observation suitable for a higher level state estimation process. These two
phases may be further refined into a hierarchical chain of "tiers", where observations at each level are obtained
through the computation of a set of properties of the system's estimated state at the lower level.
An important factor that seems to have an impact on the overall ability to track high level phenomena in real
time is the computational complexity of deciding those properties when generating observations between the
tiers. And this complexity characterizes the accuracy of what can be computed within a bounded time frame.
In this paper we intend to investigate the "real time" trackability of phenomena through the analysis of the
complexity of individual models in relation to the computational complexity of computing observations in any
multi-tiered tracking system.
We have developed a general framework, called a Process Query System (PQS), that serves as a foundation for formulating tracking problems, implementing software solutions to tracking problems and understanding theoretical issues related to tracking in specific scenarios. The PQS framework posits that an environment consists of multiple dynamical processes. Processes have states, state transitions (deterministic, nondeterministic or probabilistic) and
observables related to state occupancy. Examples of such dynamical processes are nondeterministic automata, Hidden Markov Models and classical state space models. We define a tracking problem as the inverse problem of determining the processes and process states that explain a stream of observations. This paper describes a quantitative concept of trackability by considering the rate of growth of state sequences of a process model given a temporal sequence of observations. Recent formal results concerning this notion of trackability are summarized without proof. Complete proofs of the various results are contained in a technical report by the authors and cited in the bibliography.
In this paper we describe metrics related to the quantification of
situational awareness of surveillance systems based on sensor
networks. Our work emphasizes the necessity for the sensor system to
be able to track processes that evolve in general stochastically and
may even be driven by intelligence. The result is a hierarchical
model for surveillance systems in which different levels of
description of the system's state and its kinematics correspond to
different levels of situational awareness and require the activation
of different sensor modalities.
This paper presents an automated decentralized surveillance system
for the problem of tracking multiple mobile ground targets with no
signature in a bounded area. The system consists of unmanned aerial
vehicles (UAVs) and unattended fixed ground sensors (UGSs) with limited
communication and detection range that are deployed in the area of
interest. Each component of the system (UAV and/or Sensor) is
completely autonomous and programmed to scan the area searching for
targets and share its knowledge with other components within
communication range. UAV scheduling of the areas to search is
stochastic and the characterizing probability distributions are
determined through hypotheses of consistent tracks of target
observations. Such hypotheses are formulated by a client subsystem
called Process Query System, which is queried with streams of
incoming observations of targets and stochastic models of their
kinematics. The purpose of this work is also to provide a quantitative
measure of the situational awareness of the monitoring system in
relation to the accuracy of the target models and the degree of
decentralization of the control.
This paper presents a fully automated and decentralized surveillance system for the problem of detecting and possibly tracking mobile unknown ground vehicles in a bounded area. The system consists ideally of unmanned aerial vehicles (UAVs) and unattended fixed sensors with limited communication and detection range that are deployed in the area of interest. Each component of the system (UAV and/or sensor) is completely autonomous and programmed to scan the area searching for targets and share its knowledge with other components within communication range. We assume that both UAVs and sensors have similar computing and sensing capabilities and differ only in their mobility (sensors are stationary while UAVs are mobile). Gathered information is reported to a base station (monitor) that computes an estimate of the global state of the system and quantifies the quality of the surveillance based on a measure of the uncertainty on the number and position of the targets overtime. The present solution has been achieved through a robotic implementation of a software simulation that was in turn realized under the principles of a novel top-down methodology for the design of provably performant agent-based control systems. In this paper we provide a description of our solution including the distributed algorithms that have been employed in the control of the UAV navigation and monitoring. Finally we show the results of an novel experimental performance analysis of our surveillance system.
In this paper, we investigate the link between the rate at which
events are observed by a monitoring system and the ability of the
system to effectively perform its tracking and surveillance tasks. In
general, higher sampling rates provide better performance, but they
also require more resources, both computationally and from the sensing
We have used Hidden Markov Models to describe the dynamic processes to
be monitored and (alpha,beta)-currency as a performance measure for the monitoring system. Our ultimate goal is to be able to determine the minimum sampling rate at which we can still fulfill the performance requirements of our system.
Along with the theoretical work, we have performed simulation-based
tests to examine the validity of our approach; we compare performance
results obtained by simulation with the theoretical value obtained
a priori from the scenario parameters and illustrate with a
simple example a technique for estimating the required sampling rate
to achieve a given level of performance.
Process Query Systems (PQS) are a new kind of information retrieval technology in which user queries are expressed as process descriptions. The goal of a PQS is to detect the processes using a datastream or database of events that are correlated with the processes' states. This is in contrast with most traditional database query processing, information retrieval systems and web search engines in which user queries are typically formulated as Boolean expressions. In this paper, we outline the main features of Process Query Systems and the technical challenges that process detection entails. Furthermore, we describe several importance application areas that can benefit from PQS technology. Our working prototype of a PQS, called TRAFEN (for TRAcking and Fusion ENgine) is described as well.