Advanced text and video analytics for proactive decision making

Elizabeth K. Bowman; Matt Turek; Paul Tunison; Reed Porter; Steve Thomas; Vadas Gintautas; Peter Shargo; Jessica Lin; Qingzhe Li; Yifeng Gao; Xiaosheng Li; Ranjeev Mittu; Carolyn Penstein Rosé; Keith Maki; Chris Bogart; Samrihdi Shree Choudhari

doi:10.1117/12.2276369

Translator Disclaimer

3 May 2017 Advanced text and video analytics for proactive decision making

Elizabeth K. Bowman, Matt Turek, Paul Tunison, Reed Porter, Steve Thomas, Vadas Gintautas, Peter Shargo, Jessica Lin, Qingzhe Li, Yifeng Gao, Xiaosheng Li, Ranjeev Mittu, Carolyn Penstein Rosé, Keith Maki, Chris Bogart, Samrihdi Shree Choudhari

Author Affiliations +

Proceedings Volume 10207, Next-Generation Analyst V; 102070K (2017) https://doi.org/10.1117/12.2276369
Event: SPIE Defense + Security, 2017, Anaheim, California, United States

Abstract

Today’s warfighters operate in a highly dynamic and uncertain world, and face many competing demands. Asymmetric warfare and the new focus on small, agile forces has altered the framework by which time critical information is digested and acted upon by decision makers. Finding and integrating decision-relevant information is increasingly difficult in data-dense environments. In this new information environment, agile data algorithms, machine learning software, and threat alert mechanisms must be developed to automatically create alerts and drive quick response. Yet these advanced technologies must be balanced with awareness of the underlying context to accurately interpret machine-processed indicators and warnings and recommendations. One promising approach to this challenge brings together information retrieval strategies from text, video, and imagery. In this paper, we describe a technology demonstration that represents two years of tri-service research seeking to meld text and video for enhanced content awareness. The demonstration used multisource data to find an intelligence solution to a problem using a common dataset. Three technology highlights from this effort include 1) Incorporation of external sources of context into imagery normalcy modeling and anomaly detection capabilities, 2) Automated discovery and monitoring of targeted users from social media text, regardless of language, and 3) The concurrent use of text and imagery to characterize behaviour using the concept of kinematic and text motifs to detect novel and anomalous patterns. Our demonstration provided a technology baseline for exploiting heterogeneous data sources to deliver timely and accurate synopses of data that contribute to a dynamic and comprehensive worldview.

Citation Download Citation

Elizabeth K. Bowman, Matt Turek, Paul Tunison, Reed Porter, Steve Thomas, Vadas Gintautas, Peter Shargo, Jessica Lin, Qingzhe Li, Yifeng Gao, Xiaosheng Li, Ranjeev Mittu, Carolyn Penstein Rosé, Keith Maki, Chris Bogart, and Samrihdi Shree Choudhari "Advanced text and video analytics for proactive decision making", Proc. SPIE 10207, Next-Generation Analyst V, 102070K (3 May 2017); https://doi.org/10.1117/12.2276369

ACCESS THE FULL ARTICLE