For maximizing the benefit of today’s ISR (Intelligence, Surveillance and Reconnaissance) systems, an improved collection planning is essential. In our paper we present an approach to apply artificial intelligence and machine learning in support of collection planning tasks. One subtask in collection planning requires matchmaking between ISR-resources (further referred as assets, combining sensors and their corresponding carriers) and collection requirements, taking additional operational constraints (for example mission risk) into account. This subtask requires high competence in assessment of asset capabilities in relation to collection requirements taking actual and future operational constraints into account and is mostly conducted in a time sensitive environment. We derive a general model of our matchmaking problem. This model serves, in combination with existing requirements derived from the operational domain, as input for the analysis of artificial intelligence and machine learning methods to work out their fundamental suitability and adaptability for our model. This subset will be further analyzed for its pros and cons, if only few operational data is available and the evolving knowledge of the use of resources during mission operation has to be taken into account.
This publication addresses a structured approach to support information management and intelligence creation in defense coalitions under consideration of the corresponding operational processes. From the methodical point of view, key aspects are the application of a semantic world modeling system and the dedicated combination of data-driven as well as knowledge-based Artificial Intelligence (AI) methods. In the context of this publication, during system operations, in particular Joint Intelligence, Surveillance and Reconnaissance (ISR) results in form of textual ISR reports being in accordance with NATO reporting standards and agreements serve as input to the world modeling system. To obtain maximum benefit from the respective information, relevant information elements have to be extracted from both, structured and unstructured parts of the reports and to be combined with information being already available in the semantic world modeling system. For structured parts of a report, a predefined mapping of the respective parts of the data model of the report to the target model of the semantic world modeling system can be applied. To extract the relevant information elements from unstructured parts of the report, Natural Language Processing (NLP) techniques are needed additionally. In this context, specific challenges with regard to the application of data-driven AI methods in the domain of defense are addressed through a two-step approach for information extraction from unstructured text based on an intermediate semantic representation.
In complex operational scenarios where multiple nations and forces cooperate, flexible System of Systems (SoS) architectures being customizable to specific operations are needed. Relevant operational processes as defined within Joint ISR (Intelligence, Surveillance and Reconnaissance) and the Intelligence Cycle need to be supported. To maximize efficiency and effectiveness of Joint ISR capabilities, each Joint ISR result needs to answer the corresponding information requirement accurately. Commanders must receive the relevant information in a condensed, well-prepared manner instead of being overflowed with large amounts of (raw) data. Ensuring a common understanding of each exchanged piece of information within the defence coalition is also of utmost importance. Architectures supporting these requirements need to make use of relevant standards and agreements for data/ information management. As reports may be provided by all Joint ISR capabilities, the topic of reporting is of high importance, here. Within the described context, our publication deals with formal reporting which can be defined as organizational process at which relevant information is provided as formal reports, i.e., as documents being structured according to pre-defined (agreed) rules. We present means for ensuring allied interoperability and further (semi-)automatic processability of the information being contained in formal reports by technical means and under consideration of the relevant doctrines and standards. We also address specific means needed to ensure the creation of formal reports of high quality. Finally, we discuss current issues and new requirements on formal reporting which have to be still addressed in the field of Joint ISR.
Globalization has created complex economic and sociological dependencies. The nature of conflicts has changed and nations are confronted with a vast number of new threat scenarios. Information superiority is a question of being able to get the right information at the right time. Technology allows to disseminate information in near real-time and enables both aggressors and defenders to act remotely and network over time and space. Technologies in the areas of sensors and platforms as well as network technology and storage capability have evolved to a level where mass data can be easily shared and disseminated. To benefit fully from these new capabilities, there is a need for systems and services that can interact with each other in a well-defined interoperable way. On an organizational level it is necessary to define common processes to coordinate actors, their activities, the assets available and the data and information created. Security restrictions, (intellectual) property rights as well as data privacy regulations need to be fulfilled. The Coalition Shared Data (CSD) concept supports operational processes as defined by NATO within Joint ISR (Intelligence, Surveillance and Reconnaissance) and the Intelligence Cycle by defining standardized interfaces, data models, services and workflows. To support information provision additionally, techniques of data and information extraction, fusion and visual analysis can be added at the system level. Other available sources can be connected through the usage of semantic world models. To ensure data integrity multilevel security measures need to be combined with the existing concept.
The publication introduces the operational processes defined within NATO doctrines and process descriptions and maps the CSD concept to it. It describes the new Edition of STANAG (NATO Standardization Agreement) 4559 Edition 4 that implements the CSD concept and connects it to operational processes. Based on this it introduces a system architecture for ISR Analytics.
Today, drone technology has been made available around the world. Anyone can purchase a drone from an online retailer. Government agencies and military are seeing a rise in drones used for terrorism, destruction and espionage. The emergence of threats caused by unfriendly or hostile drones requires proactive drone detection in order to decide on appropriate defensive actions. In this contribution, a high-level data fusion component for drone classification is presented. The high-level data fusion component is part of our counter UAV system MODEAS including decision support. The component provides well-defined interfaces which allow it to be integrated also into other counter UAV systems. The aim of the high-level data fusion component is to support an operator in his decision making by providing detailed information about detected drones together with assigned threat levels. To identify a detected and tracked drone with sufficient detail, a knowledge-based classification is performed, based on background knowledge like drone model specifications. By fusing the knowledge-based classification results with prior results of a sensor-based classification, the overall classification is improved. The fusion results, in addition to kinematic data, also contain specific capabilities of the respective drone like its maximum payload, endurance, and speed as well as recorded incidents with similar drones or their typical (commercial) usage, if known. Based on these fusion results, a threat analysis is performed. The component’s output then is a ranked list of dossiers for the most probable types of drones with regard to the observation data and their assigned threat levels.
Bayesian statistics offers a well-founded and powerful fusion methodology also for the fusion of heterogeneous
information sources. However, except in special cases, the needed posterior distribution is not analytically
derivable. As consequence, Bayesian fusion may cause unacceptably high computational and storage costs in
practice. Local Bayesian fusion approaches aim at reducing the complexity of the Bayesian fusion methodology
significantly. This is done by concentrating the actual Bayesian fusion on the potentially most task relevant
parts of the domain of the Properties of Interest. Our research on these approaches is motivated by an analogy
to criminal investigations where criminalists pursue clues also only locally. This publication follows previous
publications on a special local Bayesian fusion technique called focussed Bayesian fusion. Here, the actual
calculation of the posterior distribution gets completely restricted to a suitably chosen local context. By this,
the global posterior distribution is not completely determined. Strategies for using the results of a focussed
Bayesian analysis appropriately are needed. In this publication, we primarily contrast different ways of embedding
the results of focussed Bayesian fusion explicitly into a global context. To obtain a unique global posterior
distribution, we analyze the application of the Maximum Entropy Principle that has been shown to be successfully
applicable in metrology and in different other areas. To address the special need for making further decisions
subsequently to the actual fusion task, we further analyze criteria for decision making under partial information.
Information fusion is essential for the retrieval of desired information in a sufficiently precise, complete, and robust
manner. The Bayesian approach provides a powerful and mathematically funded framework for information
fusion. By local Bayesian fusion approaches, the computational complexity of Bayesian fusion gets drastically
reduced. This is done by a concentration of the actual fusion task on its probably most task relevant aspects. In
this contribution, further research results on a special local Bayesian fusion technique called focussed Bayesian
fusion are reported. At focussed Bayesian fusion, the actual Bayesian fusion task gets completely restricted to
the probably most relevant parts of the range of values of the Properties of Interest. The practical usefulness
of focussed Bayesian fusion is shown by the use of an example from the field of reconnaissance. Within this
example, final decisions are based on local significance considerations and consistency arguments. As shown in
previous publications, the absolute values of focussed probability statements represent upper bounds for their
global values. Now, lower bounds which are obtained from the knowledge about the construction of the focussed
Bayesian model are proven additionally. The usefulness of the resulting probability interval scheme is discussed.
Local Bayesian fusion approaches aim to reduce high storage and computational costs of Bayesian fusion which
is separated from fixed modeling assumptions. Using the small world formalism, we argue why this proceeding is
conform with Bayesian theory. Then, we concentrate on the realization of local Bayesian fusion by focussing the
fusion process solely on local regions that are task relevant with a high probability. The resulting local models
correspond then to restricted versions of the original one. In a previous publication, we used bounds for the
probability of misleading evidence to show the validity of the pre-evaluation of task specific knowledge and prior
information which we perform to build local models. In this paper, we prove the validity of this proceeding using
information theoretic arguments. For additional efficiency, local Bayesian fusion can be realized in a distributed
manner. Here, several local Bayesian fusion tasks are evaluated and unified after the actual fusion process.
For the practical realization of distributed local Bayesian fusion, software agents are predestinated. There is
a natural analogy between the resulting agent based architecture and criminal investigations in real life. We
show how this analogy can be used to improve the efficiency of distributed local Bayesian fusion additionally.
Using a landscape model, we present an experimental study of distributed local Bayesian fusion in the field of
reconnaissance, which highlights its high potential.
Proc. SPIE. 6242, Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications 2006
KEYWORDS: Information fusion, Criminalistics, Reconnaissance, Computer architecture, Signals intelligence, Information architecture, Space reconnaissance, Data fusion, Data analysis, Bayesian inference
A new architecture for fusing information and data from heterogeneous sources is proposed. The approach takes criminalistics as a model. In analogy to the work of detectives, who attempt to investigate crimes, software agents are initiated that pursue clues and try to consolidate or to dismiss hypotheses. Like their human pendants, they can, if questions beyond their competences arise, consult expert agents. Within the context of a certain task, region, and time interval, specialized operations are applied to each relevant information source, e.g. IMINT, SIGINT, ACINT,..., HUMINT, data bases etc. in order to establish hit lists of first clues. Each clue is described by its pertaining facts, uncertainties, and dependencies in form of a local degree-of-belief (DoB) distribution in a Bayesian sense. For each clue an agent is initiated which cooperates with other agents and experts. Expert agents support to make use of different information sources. Consultations of experts, capable to access certain information sources, result in changes of the DoB of the pertaining clue. According to the significance of concentration of their DoB distribution clues are abandoned or pursued further to formulate task specific hypotheses. Communications between the agents serve to find out whether different clues belong to the same cause and thus can be put together. At the end of the investigation process, the different hypotheses are evaluated by a jury and a final report is created that constitutes the fusion result.
The approach proposed avoids calculating global DoB distributions by adopting a local Bayesian approximation and thus reduces the complexity of the exact problem essentially.
Different information sources are transformed into DoB distributions using the maximum entropy paradigm and considering known facts as constraints. Nominal, ordinal and cardinal quantities can be treated within this framework equally. The architecture is scalable by tailoring the number of agents according to the available computer resources, to the priority of tasks, and to the maximum duration of the fusion process. Furthermore, the architecture allows cooperative work of human and automated agents and experts, as long as not all subtasks can be accomplished automatically.