This PDF file contains the front matter associated with SPIE Proceedings Volume 11006 including the Title Page, Copyright information, Table of Contents, Introduction, and Conference Committee listing.

Artificial Intelligence / Machine Learning (AI/ML) is a foundational requirement for Multi-Domain Operations (MDO). To solve some of MDO’s most critical problems, for example, penetrating and dis-integrating an adversary’s antiaccess/area denial (A2/AD) systems, the future force requires the ability to converge capabilities from across multiple domains at speeds and scales beyond human cognitive abilities. This requires robust, interoperable AI/ML that operates across multiple layers: from optimizing technologies and platforms, to fusing data from multiple sources, to transferring knowledge across joint functions to accomplish critical MDO tactical tasks. This paper provides an overview of ongoing work from the Unified Quest Future Study Plan and other events with the Army’s Futures and Concepts Center to operationalize AI/ML to address MDO problems with this layered approach. It includes insights and required AI/ML capabilities determined with subject matter experts from various organizations at these learning events over the past two years, as well as vignettes that illustrate how AI/ML can be operationalized to enable successful Multi-Domain Operations against a near peer adversary.

The paper presents a research agenda on supporting machine intelligence at the tactical network edge, and overviews early results in that space developed under the Internet of Battlefield Things Collaborative Research Alliance (IoBT CRA); a collaboration between the US Army Research Labs and a consortium of academia and industry led by the University of Illinois. It is becoming evident today that the use of artificial intelligence and machine learning components in future military operations will be inevitable. Yet, at present, the dependability limitations and failure modes of these components in a complex multi-domain battle environment are poorly understood. Most civilian research investigates solutions that exceed the SWaP (Size, Weight, and Power) limitations of tactical edge devices, and/or require communication with a central back-end. Resilience to adversarial inputs is not well developed. The need for significant labeling to train the machine slows down agility and adaptation. Cooperation between resource-limited devices to attain reliable intelligent functions is not a central theme. These gaps are filled by recent research emerging from the IoBT CRA. The paper reviews the field and presents the most interesting early accomplishments of the Alliance aiming to bridge the aforementioned capability gaps for future military operations.

Recent trends in artificial intelligence and machine learning (AI/ML), dynamic data driven application systems (DDDAS), and cloud computing provide opportunities for enhancing multidomain systems performance. The DDDAS framework utilizes models, measurements, and computation to enhance real-time sensing, performance, and analysis. One example the represents a multi-domain scenario is “fly-by-feel” avionics systems that can support autonomous operations. A "fly-by-feel" system measures the aerodynamic forces (wind, pressure, temperature) for physics-based adaptive flight control to increase maneuverability, safety and fuel efficiency. This paper presents a multidomain approach that identifies safe flight operation platform position needs from which models, data, and information are invoked for effective multidomain control. Concepts are presented to demonstrate the DDDAS approach for enhanced multi-domain coordination bringing together modeling (data at rest), control (data in motion) and command (data in use).

We are developing an agent-based model that allows us to explore control and distributed machine learning concepts in a systems-of-systems framework. The model captures the complex interactions between vehicles with very different operational tempos (such as satellites and unmanned aerial vehicles (UAVs)) and a variety of environmental elements (communication towers, objects of interest, etc.). Treating the model as a complex adaptive system, we can explore issues of controllability and observability, such as the constraints needed to maintain multi-vehicle formations under diverse conditions, and scaling questions, such as the data rates among vehicles and control centers under diverse system parameters.

The Army is evolving its warfighting concepts to militarily compete, penetrate, dis-integrate, and exploit adversaries as part of a Multi-Domain Operations (MDO) Joint Force. Artificial Intelligence/Machine Learning (AI/ML) is critical to the Armys vision for AI-enabled capabilities to achieve MDO but has significant challenges and risks. The Army faces rapidly changing, never-before-seen situations, where pre-existing training data will quickly become ineffective; tactical training data is noisy, incomplete, and erroneous; and adversaries will employ deception. This is especially challenging at the Tactical Edge that operates in complex urban settings that are dynamic, distributed, resource-constrained, fast-paced, contested, and often physically and virtually isolated. Federated machine learning is collaborative training where training data is not exchanged in order to overcome constraints on training data sharing (policy, security, coalition constraints) and/or insufficient network capacity that are prevalent at the Tactical Edge. We describe the applicability of federated machine learning to MDO using a motivating scenario and identify when it is advantageous to be used. The attributes and design inputs for the deployment of AI/ML (learn-infer-act process), the factors that impact learning-inference processes, and the operational factors impacting the deployment of machine learning are identified. We propose strategies for six AI/ML deployment regimes that are at the intersection of total uncertainty (model and environmental) and the operational timeliness that is required, and map AI/ML techniques to address these challenges and requirements. Scientific research questions that must be answered to fill critical knowledge gaps are identified, and ongoing research approaches to answer them are highlighted.

In order to make sensible decisions during a multi domain battle, autonomous systems, just like humans, need to understand the current military context. They need to ‘know’ important mission context information such as, what is the commander’s intent and where are, and in what state, are friendly and adversary actors. They also need an understanding of the operating environment; the state of the physical systems ‘hosting’ the AI; and just as importantly, the state of the communication networks that allows each AI ‘node’ to receive and share critical information. The problem is: capturing, representing, and reasoning over this contextual information is especially challenging in distributed, dynamic, congested and contested multi domain battlespaces. This is not only due to rapidly changing contexts and noisy, incomplete and potentially erroneous data, but also because, at the tactical edge, we have limited computing, storage and battery resources. The US Army Research Laboratory, Australia’s Defence Science Technology Group and associated University partners are collaborating to develop an autonomous system called SMARTNet that can transform, prioritize and control the flow of information across distributed, intermittent and limited tactical networks. In order to do this however, SMARTNet requires a good understanding of the current military context. This paper describes how we are developing this contextual understanding using new AI and ML approaches. It then describes how we are integrating these approaches into an exemplar tactical network application that improves the distribution of information in complex operating environments. It concludes by summarizing our results to-date and by setting a way forward for future research.

The paper describes a vision for dependable application of machine learning-based inferencing on resource-constrained edge devices. The high computational overhead of sophisticated deep learning learning techniques imposes a prohibitive overhead, both in terms of energy consumption and sustainable processing throughput, on such resource-constrained edge devices (e.g., audio or video sensors). To overcome these limitations, we propose a “cognitive edge” paradigm, whereby (a) an edge device first autonomously uses statistical analysis to identify potential collaborative IoT nodes, and (b) the IoT nodes then perform real-time sharing of various intermediate state to improve their individual execution of machine intelligence tasks. We provide an example of such collaborative inferencing for an exemplar network of video sensors, showing how such collaboration can significantly improve accuracy, reduce latency and decrease communication bandwidth compared to non-collaborative baselines. We also identify various challenges in realizing such a cognitive edge, including the need to ensure that the inferencing tasks do not suffer catastrophically in the presence of malfunctioning peer devices. We then introduce the soon-to-be deployed Cognitive IoT testbed at SMU, explaining the various features that enable empirical testing of various novel edge-based ML algorithms.

Real-life AI based solutions usually consist of a complex chain of processing elements, which may include a mixture of machine learning based approaches and traditional programmed knowledge. The solution uses this chain of processing elements to convert input information into an output decision. When information is provided for a specific solution, the impact of the information on the decision can be measured quantitatively as a Value of Information (VoI) metric. In prior work, we have considered how the VoI metric can be defined for a single AI-based processing element. To be useful in real-life solution instances, the VoI metric needs to be enhanced to handle a complex chain of processors, and be extended to AI-based solutions, as well as supporting elements that may not necessarily use AI. In this paper, we propose a definition of VoI that can be used across AIbased processing, as well as non AI based processing, and show how the construct can be used to analyze and understand the impact of a piece of information on a chain of processing elements.

It is readily evident that the U.S. Army must establish an artificial intelligence (AI) and machine learning (ML) science and technology (S and T) strategy to rapidly capture, develop, and field the steady stream of discoveries and innovations derived from the global proliferation of AI. In order to focus on Army-specific problem sets, the U.S. Army Combat Capabilities Development Command (CCDC) Army Research Laboratory (ARL) intends to stand up the Army AI Innovation Institute (A2I2) in 2019. This paper discusses how the A2I2 will coordinate, conduct, and accelerate basic research to address Army-specific challenges, with a focus on advancing AI capabilities for autonomous maneuver in multi-domain operations (MDO). CCDC ARL will leverage its existing distributed high-performance computing (HPC) and network infrastructure, along with its regional laboratory extensions, to enable basic AI research with top-tier universities, small and large commercial businesses, established Department of Defense (DOD) industrial partners, and other DOD and non-DOD government organizations. This paper also discusses how the A2I2 will establish an accessible database of heterogeneous data, a repository of AI and ML algorithms and software tools, and military-relevant challenge problems.

OPENING QUOTE MOTIVATING THIS PROPOSED PRESENTATION: In addition to innovations in Artificial Intelligence & Machine Learning (AI/ML), there is a need to be able to innovate with speed and agility. “… DoD does not have an innovation problem; it has an innovation adoption problem … Moving in days and weeks rather than months and years is a necessity…” - Statement by Dr. Eric Schmidt, Defense Innovation Board member to House Armed Services Committee, April 17, 2018 ACCELERATING CROWD-SOURCED AI/ML INNOVATIONS: This paper presents a novel crowd-sourcing distance research strategy for not only accelerating innovations in Artificial Intelligence / Machine Learning (AI/ML) by moving in days and weeks rather than months and years but to do it with agility while also growing tomorrow’s AI/ML innovators for both commercial and defense priorities. We call this new paradigm for distance research, MOORE for Massive Open Online Research and Engineering (MOORE). The MOORE distance innovation and collaboration concept was developed and refined utilizing 3D virtual environment technology over 10 years under a former multi-million dollar Air Force research internship program called Discovery Lab. The underlying 3D virtual distance research environment utilizes avatar technologies as a unique adaptation of the Massive Open Online Course (MOOC) concept represented by distance education courses like Udacity, edX, Khan Academy, Coursera, and others. 3D VIRTUAL AI/ML RESEARCH CAMPUS (ARC): In this paper, we will detail the development of the distance research MOORE concept, its associated 3D virtual AI/ML Research Campus (ARC) and the crowd-sourced AI/ML student-team projects being formed in the ARC under a grand challenge umbrella that we call Project Beast Master. We will also demonstrate how this MOORE concept with the 3D virtual ARC environment could facilitate collaboration with interested government, industry, and universities working with these crowds of student teams recruited from across the country. COOPERATIVE AUTONOMY PROJECT BEAST MASTER: These AI/ML projects will explore innovations designed to explicitly support future military Multi Domain Battle (MDB) operations centered around teams of highly-dispersed warfighters and agents (robotic and software) operating in distributed, dynamic, complex, cluttered environments. The AI/ML innovations will initially be pursued at the intersection of two Army Research Laboratory (ARL) Essential Research Areas (ERA) – Artificial Intelligence/Machine Learning and Human-Agent Teaming. To motivate and to give focus to the crowds of multidisciplinary student teams recruited to collaborate as avatars inside the 3D virtual ARC, we have organized the initial AI/ML ARC projects in these two ERAs under a Cooperative Autonomy grand challenge umbrella that we call Project Beast Master. DISCOVER LAB – GLOBAL (DLG) AND DEEP HORIXONS (DHX): Although a novel research paradigm, it is based on a highly successful model developed as part of a multi-million dollar Air Force research internship program that operated for 10 years before it ended upon the founding director’s retirement to continue it philosophically as Discovery Lab – Global (DLG), an separate, independently operated 501(c)(3) entrepreneurial Science, Technology, Engineering, and Math (STEM) lab. The original Air Force program, Discovery Lab, recruited 100-plus engineering and computer science students year round from 40+ universities and colleges across the country. These students collaborated on a broad portfolio of technology areas of interest to the Air Force to include data analytics, robotics, 3D printing, mobile apps, information visualization, wearable technologies, internet of things, cybersecurity and of course, artificial intelligence. To reach students who could not participate on-site, the Air Force program developed 3D virtual campus environments utilizing non-proprietary, open-source virtual environment technology such as Open Simulator (Open Sim). The resulting campus environment which was called Deep Horizons allowed students to participate from across the country …and even globally… as avatar from their university or their residence. All that was required was a computer with internet connection and free software that was provided password-protected access to the research campus environment. This concept of creating a password-protected 3D virtual campus environment to make collaborative distance research opportunities accessible via avatars continues today as Deep HoriXons (DHX). Part of this paper will detailed the construction and operation of DHX which is central to the 3D virtual AI/ML Research Campus (ARC) which will host the crowds of Beast Master human-agent teaming projects. Two example projects being developed for Human-Agent Teaming and Human-Information Interfaces utilizing home-grown 3D printed "drones" and AI-assisted neural interface for human-agent communication will be summarized to illustrate the potential of a 3D virtual AI/ML research campus (ARC) for not only accelerating AI/ML innovations in Cooperative Autonomy but also growing future innovators. STRATEGIC PARTNERSHIPS: Our goal is 100+ students recruited from across 20+ universities and colleges across the country …and globally … working together inside the 3D virtual ARC on up to 20 Human-Agent Teaming and Human-Information Interfacing student-team projects to demonstrate prototype Cooperative Autonomy capabilities. We will show how the virtual world programming language will allow us to create Cooperative Autonomy research experiment to design, develop, and demonstrate Beast Master proto-type innovations in Human-Agent Teaming and Human-Information Interfaces within a 3D virtual test range. This approach creates tremendous opportunities for collaboration with partners from government, industry, and academia to not only rapid prototype AI/ML innovations around the Beast Master concept but at the same time mentor our nation’s future AI/ML innovators.

In preparations for Multi-Domain Operations and Battles, All-Source and OSINT intelligence analysts gather, assess, and extract relevant information from operational databases as well as publicly available information. This data, often unstructured text documents, is noisy with relevant snippets buried within the document corpus. The costs of exploratory search and exploitive document analysis required to find these hidden snippets of information often drive searches toward a small subset of documents. Additionally, modern search tools may reinforce the confirmation bias of analysts by providing only those documents that closely match their search query. Due to the potentially high tempo of multi-domain battle, the end result is a decision or hypothesis that is ill-considered and substantiated by potentially biased information. An automated information foraging framework can mitigate these challenges by automatically identifying a wide breadth of topics for the user, extracted directly from a document corpus. A semantic network formed from the constituent entities within a document corpus contains inherently valuable topological structures that can be used to generate topics and also guide the analyst?s information exploration. Leveraging a suite of information retrieval and graph analysis algorithms that analyze the semantic network, a framework is defined for assisting analysts in both exploring and exploiting relevant information from a corpus to support the sensemaking process.

The lack of tools to rapidly identify and align data from different sources is a critical, needed capability for the Department of Defense especially when it comes to automated ingestion. In the current open source Karma Mapping Tool, the Steiner tree optimization algorithm suggests semantic types during data alignment. We hypothesize that Machine Learning (ML) may perform better than the Steiner approach on a subset of column types, or “labels”, where 1.) the data is extremely similar in structure and content and 2.) inferring column type correctly is highly dependent on the interrelated components of the dataset. In this session we discuss the experimental design, our initial results, and a path toward future work in broader applications beginning with intelligence analysis in the maritime domain. The initial results from this experiment show there is promise in using ML to do column prediction in analysis environments where there are many similar or overlapping data.

Intelligent military systems require perception capabilities that are flexible, dynamic, and robust to unstructured environments and new situations. However, current state-of-the-art algorithms are based on deep learning, require large amounts of data, and require a proportionally large human effort in collection and annotation. To help improve this situation, we define a method of comparing 3D environment reconstructions without ground truth based on the exploitation of available reflexive information, and use the method to evaluate existing RGBD mapping algorithms in an effort to generate a large, fully-annotated data set for visual learning tasks. In addition, we describe algorithms and software that support rapid manual annotation of these reconstructed 3D environments for a variety of vision tasks. Our results show that we can use existing data sets as well as synthetic data to bootstrap tools that allow us to quickly and efficiently label larger data sets without ground truth, maximizing human effort without requiring crowd sourcing techniques.

Machine Learning systems rely on data for training, input and ongoing feedback and validation. Data in the field can come from varied sources, often anonymous or unknown to the ultimate users of the data. Whenever data is sourced and used, its consumers need assurance that the data accuracy is as described, that the data has been obtained legitimately, and they need to understand the terms under which the data is made available so that they can honour them. Similarly, suppliers of data require assurances that their data is being used legitimately by authorised parties, in accordance with their terms, and that usage is appropriately recompensed. Furthermore, both parties may want to agree on a specific set of quality of service (QoS) metrics, which can be used to negotiate service quality based on cost, and then receive affirmation that data is being supplied within those agreed QoS levels. Here we present a conceptual architecture which enables data sharing agreements to be encoded and computationally enforced, remuneration to be made when required, and a trusted audit trail to be produced for later analysis or reproduction of the environment. Our architecture uses blockchainbased distributed ledger technology, which can facilitate transactions in situations where parties do not have an established trust relationship or centralised command and control structures. We explore techniques to promote faith in the accuracy of the supplied data, and to let data users determine trade-offs between data quality and cost. Our system is exemplified through consideration of a case study using multiple data sources from different parties to monitor traffic levels in urban locations.

Commercial Artificial Intelligence (AI), e.g., the self driving car industry, is often used in predictable settings, with structured surroundings. Significant AI and Machine Learning (ML) progress, particularly in visual perception, has been made in these settings with the use of large publicly available datasets. However, there still exists a prevalent domain mismatch between this data and military relevant environments. In this work we begin to analyze the importance of mobile robot platform design and heterogeneity to effectively collect data more representative of the military domain. The framework of our research is rooted in the importance of expressing constantly changing, yet repeated conditions, with disadvantageous lighting and perspectives in highly unstructured environments.

How to build a machine that can continuously learn from observations in its life and make accurate inference/prediction? This is one of the central questions in Artificial Intelligence. Many challenges are present, such as the difficulty of learning from infinitely many observations (data), the dynamic nature of the environments, noisy and sparse data, the intractability of posterior inference, etc. This tutorial will discuss how the Bayesian approach provides a natural and efficient answer. We will start from the basic of Bayesian models, and then the variational Bayes method for inference. Next, we will discuss how to learn a Bayesian model from an infinite sequence of data. Some challenges such as catastrophic forgetting phenomenon, concept drifts, and overfitting will be discussed.

Autonomous systems are expected to have a major impact in future coalition operations. These systems are enabled by a variety of Artificial Intelligence (AI) learning algorithms that contextualize and adapt in varying, possibly unforeseen situations to assist humans in achieving complex tasks. Moreover, these systems will be required to operate in dynamic and challenging environments that impose certain constraints such as task formation and collaboration, ad-hoc resource availability, rapidly changing environmental conditions and the requirement to abide by mission objectives. Therefore, such systems require the capability to adapt and evolve so that they can behave autonomously at the edge of the network in new situations. Crucially, autonomous systems have to understand the bounds in which they can operate based on their own capability and the constraints of the environment. Policies are typically used by systems to define their behavior and constraints and often these policies are manually configured and managed by humans. AI-enabled systems will require novel approaches to rapidly learn, create, augment, and model emerging policies to support real-time decision making. Recent work has shown that such policy model generations are possible through symbolic learning to shallow and deep learning approaches for different classes of problems. Motivated by this observation, in this paper, we propose to apply recent advances in explainable-AI to develop an approach which is agnostic to the learning algorithm, thus enabling seamless policy generation in the coalition environment.

Nowadays, intelligent unmanned vehicles, such as unmanned aircraft and tanks, are involved in many complex tasks in the modern battlefield. They compose the networked intelligent systems with varying degrees of operational autonomy, which will continue to be used increasingly on the future battlefield. To deal with such a highly unstable environment, intelligent agents need to collaborate to explore the information and achieve the entire goal. In this paper, we will establish a novel comprehensive cooperative deep deterministic policy gradients (C₂DDPG) algorithm by designing a special reward function for each agent to help collaboration and exploration. The agents will receive states information from their neighboring teammates to achieve better teamwork. The method is demonstrated in a real-time strategy game, StarCraft micromanagement, which is similar to a battlefield with two groups of units.

Situational understanding is impossible without causal reasoning and reasoning under and about uncertainty, i.e. probabilistic reasoning and reasoning about the confidence in the uncertainty assessment. We therefore consider the case of subjective (uncertain) Bayesian networks. In previous work we notice that when observations are out of the ordinary, confidence decreases because the relevant training data, effective instantiations, to determine the probabilities for unobserved variables, on the basis of the observed variables, is significantly smaller than the size of the training data, the total number of instantiations. It is therefore of primary importance for the ultimate goal of situational understanding to be able to efficiently determine the reasoning paths that lead to low confidence whenever and wherever it occurs: this can guide specific data collection exercises to reduce such an uncertainty. We propose three methods to this end, and we evaluate them on the basis of a case-study developed in collaboration with professional intelligence analysts.

As humans and agents or machines are utilized to accomplish missions in multi domain battles there is an increase in artificial intelligence (AI) and machine learning (ML) to support the interaction. However, the techniques and algorithms within AI/ML are not without challenges. One key challenge is how uncertainty of the results influences decision making. Added to this challenge is where uncertainty is introduced and how it impacts the decision making tasks. Uncertainty can come from limitations in the data that is used to develop or train the AI/ML model to lack of confidence in the behavior or suggestions that the agents generate. Uncertainty can come from underlying motivations or objectives tied to the mission to interpretations of the operational area. In this paper we will define and scope uncertainty, linking it to selected components of decision making. We will discuss the generation of a measure of uncertainty that we are including in simulations along with the supporting information that will impact the decisions. Following this, the selected parameters values for several simulations for multi domain battle scenarios are presented. Analysis and evaluation of the results from these simulations will be shown. Supporting data will be mentioned to frame the results and the plans for future investigations.

Army Intelligence operates in a data rich environment with limited ability to operationalize exponentially increasing volumes of disparate structured and unstructured data to deliver timely, accurate, relevant, and tailored intelligence in support of mission command at echelon. The volume, velocity, variety, and veracity (the 4 Vs) of data challenge existing Army intelligence systems and processes, degrading the efficacy of the Intelligence Warfighting Function (IWfF). At the same time, industry has exploited the recent growth in data science technology to address the challenge of the 4 Vs and bring relevant data-driven insights to business leaders. To bring together the lessons from industry and the data science community, the US Army Research Laboratory (ARL) has collaborated with the US Army Intelligence Center of Excellence (USAICoE) to research these Military Intelligence (MI) challenges in an Army AR 5-5 Study entitled, “Application of Data Science within the Army Intelligence Warfighting Function.” This paper summarizes the problem statement, research performed, key findings, and way forward for MI to effectively employ data science and data scientists to reduce the burden on Army Intelligence Analysts and increase the effectiveness of data exploitation to maintain a competitive edge over our adversaries.

Disruptive change in the conduct of military operations occur not with the emergence of new technologies, but with the integration of the new technologies into processes of the military organization [1]. The emergence of new computational tools over the past several years represent new technologies which must be integrated into military processes in order to appropriately exploit their capabilities, with the potential for disruptive change in the conduct of military operations. To integrate these tools it is important to first to understand the computational ecosystem and how this ecosystem supports constant development of Artificial Intelligence (AI) and other computational tools. Understanding this dynamic process and how AI is really a suite of constantly improving tools, each with unique strengths and weaknesses provides the perspective necessary to properly integrate them. Understanding the dynamics and categories of computational tools allows one to overlay each tool on the appropriate places of the Joint Intelligence Process (JIP) and Joint Planning Process (JPP). Critical to this integration is not just the emergence of these new technologies but also ensuring those individuals who are applying them have the capability of basic coding to leverage these tools for the unique situations they are trying to influence. The emergence of new computational tools, as epitomized by AI, represents a new and powerful capability which must be properly integrated into military processes if they are going to be fully exploited to support analysis and decision-making.

When training data for machine learning is obtained from many different sources, not all of which may be trusted, it is difficult to determine which training data to accept and which to reject. A policy-based approach for data curation, where the policies are generated after examining the properties of the offered data, can provide a way to only accept selected data for creating a machine learning model. In this paper, we discuss the challenges associated with generating policies that can manage training data from different sources. An efficient policy generation scheme needs to determine the order in which information is received, must have an approach to determine the trustworthiness of each partner, must have an approach to decide how to quickly assess which data subset can add value to a complex model, and must address several other issues. After providing an overview of the challenges, we propose approaches to solve them and study the properties of those approaches.

To make progress in applying technological advancements towards multi-domain command and control (MDC2), including user-facing data collection and evaluation is critical but made more difficult by the lack of existing MDC2 systems. To address these challenges, we used a two-pronged approach that blended HMT Systems Engineering (SE) insights on designing user interfaces for artificial intelligence (AI) with MDC2 common operational picture (COP) research. We interviewed C2 operators using modified cognitive task analysis methods and analyzed data via thematic coding. User stories and future concept ideas were created from the interview data. The authors designed and delivered annotated mock-ups to reflect those concepts; they are in process of being incorporated into a software prototype. The designs were iteratively refined based on feedback from operators. The HMT SE guide, future concepts, annotated mockups, and prototypes are being shared with sponsors and used as foundational input for MITRE’s MDC2 exploration environment for FY19 and beyond. Products from this work include insights for how technology should support MDC2 planning and decision making, user stories capturing those insights and mock-up graphical concepts, and a set of design seeds for a future MDC2 Common Operational Picture, including layers, content, and other capabilities towards decision support.

While deep reinforcement learning techniques have led to agents that are successfully able to learn to perform a number of tasks that had been previously unlearnable, these techniques are still susceptible to the longstanding problem of reward sparsity. This is especially true for tasks such as training an agent to play StarCraft II, a real-time strategy game where reward is only given at the end of a game which is usually very long. While this problem can be addressed through reward shaping, such approaches typically require a human expert with specialized knowledge. Inspired by the vision of enabling reward shaping through the more-accessible paradigm of natural-language narration, we investigate to what extent we can contextualize these narrations by grounding them to the goal-specific states. We present a mutual-embedding model using a multi-input deep-neural network that projects a sequence of natural language commands into the same high-dimensional representation space as corresponding goal states. We show that using this model we can learn an embedding space with separable and distinct clusters that accurately maps natural-language commands to corresponding game states . We also discuss how this model can allow for the use of narrations as a robust form of reward shaping to improve RL performance and efficiency.

Head mounted displays (HMD) may prove useful for synthetic training and augmentation of military C5ISR decisionmaking. Motion sickness caused by such HMD use is detrimental, resulting in decreased task performance or total user dropout. The genesis of sickness symptoms is often measured using paper surveys, which are difficult to deploy in live scenarios. Here, we demonstrate a new way to track sickness severity using machine learning on data collected from heterogeneous, non-invasive sensors worn by users who navigated a virtual environment while remaining stationary in reality. We discovered that two models, one trained on heterogeneous sensor data and another trained only on electroencephalography (EEG) data, were able to classify sickness severity with over 95% accuracy and were statistically comparable in performance. Greedy feature optimization was used to maximize accuracy while minimizing the feature subspace. We found that across models, the features with the most weight were previously reported in the literature as being related to motion sickness severity. Finally, we discuss how models constructed on heterogeneous vs homogeneous sensor data may be useful in different real-world scenarios.

Collective intelligence is generally defined as the emergence and evolution of intelligence derived from the collective and collaborative efforts of several entities; to include humans and (dis)embodied intelligent agents. Recent advances in immersive technology have led to cost-effective tools that allow us to study and replicate interactions in a controlled environment. Combined together, immersive collective intelligence holds the promise of a symbiotic intelligence that could be greater than the sum of the individual parts. For the military, where the decision making process is typically characterized by high-stress and high-consequence, the concept of a distributive, immersive collective intelligence capability is game changing. Commanders and staff will now be able to remotely immerse themselves in their operational environment with subject matter expertise and advanced analytics. This paper presents the initial steps to understanding immersive collective intelligence with a demonstration designed to discern how military intelligence analysts benefit from an immersive data visualization.

The Army plans to integrate artificial intelligence (AI)/machine learning (ML) and other intelligent decision-making aids into future dismounted Warfighter systems to augment situational awareness and target acquisition capabilities. However, due to the unique constraints of dismounted operations, successful implementation of intelligent decisionmaking aids in dismounted systems necessitates a human-in-the-loop approach, which includes the ability for the Warfighter to provide feedback to the autonomous system. Human-in-the-loop feedback can augment current machine learning techniques by reducing the size of datasets needed to train algorithms and allow algorithms to be flexible and adaptive to changing battlespace conditions. As such, research is required to define the bidirectional interactions between man and machine in this context, to optimize human-intelligent agent teaming for the dismounted Warfighter. In this paper, we focus on a specific application of dismounted Human-AI interaction to weapon mounted target acquisition (small arms fire control systems) and discuss issues pertaining to an important component of this optimization: how intelligent information is communicated to the end user. We consider how the intelligent information is presented to the Warfighter, and what underlying cognitive and perceptual processes can be leveraged to optimize teamed decision making. Such factors are critical to the successful implementation of human-in-the-loop AI in dismounted applications and ultimately the effectiveness of intelligent decision-making aids.

With the exponential growth of technology, future military operations will be comprised of not just ground operations but a multi-domain battlespace. Paramount to mission success will be the reliance on intelligent adaptive computational agents and effective human-agent teaming. An agent teammate can assist the Soldier with tasks that may be seen as physically difficult, cognitively fatiguing, or high risk. However, successful teaming is compromised when an agent lacks the attributes that contribute to effective human-human collaboration, such as knowledge about team-members’ work preferences or capabilities. One way to provide agents with a sense of team-member preferences or capabilities is to quantitatively characterize such preferences as a function of the job the human intends to perform. To address this, we analyzed a modified survey from the Army Research Institute that is commonly used to identify work-abilities variables in military personnel based on the service member’s Military Occupational Specialty (MOS). Using machine learning techniques, statistical comparisons are made in order to quantitatively assess populationaveraged responses that Soldiers from various MOS codes provided on an Army Abilities questionnaire. Similarities and differences across groupings of MOS codes can provide a set of observations that might be parametrized into a computational agent’s framework. The goal of this work is to identify MOS code related parameters that might be incorporated into a computational agent’s framework in the future development of flexibly adaptive agents for Soldieragent teams.

Based on current capabilities, many Machine Learning techniques are often inscrutable and they can be hard for users to trust because they lack effective means of generating explanations for their outputs. There is much research and development investigating this area, with a wide variety of proposed explanation techniques for AI/ML across a variety of data modalities. In this paper we investigate which modality of explanation to choose for a particular user and task, taking into account relevant contextual information such as the time available to them, their level of skill, what level of access they have to the data and sensors in question, and the device that they are using. Additional environmental factors such as available bandwidth, currently usable sensors and services are also able to be accounted for. The explanation techniques that we are investigating range across transparent and post-hoc mechanisms and form part of a conversation with the user in which the explanation (and therefore human understanding of the AI decision) can be ascertained through dialogue with the system. Our research is exploring generic techniques that can be used to underpin useful explanations in a range of modalities in the context of AI/ML services that operate on multisensor data in a distributed, dynamic, contested and adversarial setting. We define a meta-model for representing this information and through a series of examples show how this approach can be used to support conversational explanation across a range of situations, datasets and modalities.

Future military operations will require teams of Soldiers and intelligent systems to plan and execute collective action in a dynamic and adversarial environment. In human teams, teamwork processes such as effective communication and shared understanding underlie effective team performance. Recent work proposes a vision for generalizing this theory to human-agent teams and facilitating teamwork via individualized, adaptive technologies. We propose a dynamical system model to understand how individualized, adaptive technology can facilitate teamwork in human-agent teams. The model reveals three scientific challenges: describing the dynamics of team state, understanding how technological interventions will manifest in team states, and observing latent teamwork states. Using this model, we motivate a problem in which we predict team outcomes from non-obtrusive observation of a military staff during a training exercise. Representing pairwise interactions between team members as a weighted adjacency matrix, we use low-rank matrix recovery techniques to identify communication patterns that predict external evaluations of three team processes during task completion: effective communication, shared understanding, and positive affect.

In recent work, we utilized convergent cross mapping (CCM) to quantify coordination in a multi-agent reinforcement learning (MARL) paradigm by measuring causal influence between pairs of agents in a joint task. CCM was originally developed to detect causal influences within ecological systems, and as we previously demonstrated, it can be used to measure causal dependencies between pairs of time-series data. While this work has provided important insight into the coordination between 2 teammates, it is not clear how such coordination scales with the number of agents working together with a shared goal. Within a predator-prey pursuit environment, the current study investigates the influence that an incremental increase in number of predator agents has on the inherently causal relationship between predators working together to pursue a single prey. We hypothesize that averaged CCM values will decrease with increasing number of predators due to a redistribution of coordination across all predator agents. This work provides a quantitative assessment for the fundamental influence that number of cooperative agents has on the causal relationship between agents working together on a joint task, and insight into coordinated group behaviors.

Machine learning systems can provide outstanding results, but their black-box nature means that it’s hard to understand how the conclusion has been reached. Understanding how the results are determined is especially important in military and security contexts due to the importance of the decisions that may be made as a result. In this work, the reliability of LIME (Local Interpretable Model Agnostic Explanations), a method of interpretability, was analyzed and developed. A simple Convolutional Neural Network (CNN) model was trained using two classes of images of “gun-wielder” and “non-wielder". The sensitivity of LIME improved when multiple output weights for individual images were averaged and visualized. The resultant averaged images were compared to the individual images to analyze the variability and reliability of the two LIME methods. Without techniques such as those explored in this paper, LIME appears to be unstable because of the simple binary coloring and the ease with which colored regions flip when comparing different analyses. A closer inspection reveals that the significantly weighted regions are consistent, and the lower weighted regions flip states due to inherent randomness of the method. This suggests that improving the weighting methods for explanation techniques, which can then be used in the visualization of the results, is important to improve perceived stability and therefore better enable human interpretation and trust.

The density, diversity, connectedness and scale of urban environments make military operations challenging. This paper shows that different artificial intelligence techniques can be combined to provide the commander with various form of intelligence augmentation and to support the decision making process. A warfare model has been developed where an AI system, representing a red unit, learns how to select the position for a target and for several improvised explosive devices (IEDs) in order to prevent the blue unit to locate the target. The blue unit is trained to reach the target by using deep reinforcement learning, while an evolutionary algorithm is used to train the red unit. These techniques do not rely on large amounts of historical data. Different approaches have been used and discussed to optimise the co-learning of the two agents, showing that optimal behaviour can be learned in an urban environment. Information about the most likely positions of the target and the IEDs can be extracted from the policy learned by the system, and used by the commander to provide intelligence augmentation while planning an operation and evaluating different possible courses of action. The reliability of this information depends on the realism of the AI system simulating the red unit, that is strictly dependent on the model used for the blue unit during the training.

In this paper, we describe a phenomenon, which we named “super-convergence”, where neural networks can be trained an order of magnitude faster than with standard training methods. The existence of super-convergence is relevant to understanding why deep networks generalize well. One of the key elements of super-convergence is training with one learning rate cycle and a large maximum learning rate. A insight that allows super-convergence training is that large learning rates regularize the training, hence requiring a reduction of all other forms of regularization in order to preserve an optimal regularization balance. We also derive a simplification of the Hessian Free optimization method to compute an estimate of the optimal learning rate. Experiments demonstrate super-convergence for Cifar-10/100, MNIST and Imagenet datasets, and resnet, wide-resnet, densenet, and inception architectures. In addition, we show that super-convergence provides a greater boost in performance relative to standard training when the amount of labeled training data is limited. The architectures and code to replicate the figures in this paper are available at github.com/lnsmith54/super-convergence.

Barrier coverage problems in wireless camera sensor networks (WCSNs) have drawn the attention by academic community because of their huge potential applications. Various versions of barrier coverage under WCSNs have been studied such as minimal exposure path, strong/weak barrier, 1/k barrier, full view barrier problems. In this paper, based on new (k−ω) coverage model, we study how to achieve (k −ω) barrier coverage problem under uniform random deployment scheme (hereinafter A(k − ω)BC problem). This problem aims to juggle whether any given camera sensor networks is (k − ω) barrier coverage. A camera sensor network is called (k − ω) barrier coverage if any crossing path is (k − ω) coverage. The A(k − ω)BC problem is useful because it can make balance of the number of camera sensors used and the information retrieved by the camera sensors. Furthermore, this problem is vital for design and applications for camera sensor networks when camera sensor nodes were deployed randomly. Thus, we formulate the A(k − ω)BC problem and then proposed an efficient method named Dynamic Partition for solving this problem . An extensive experiments were conducted on random instances, and the results indicated that the proposed algorithm can achieve high quality and stable solutions in real-time execution.

Artificial intelligence (AI) has enormous potential for military applications. Fully realizing the conceived benefits of AI requires effective interactions among Soldiers and computational agents in highly uncertain and unconstrained operational environments. Because AI can be complex and unpredictable, computational agents should support their human teammates by adapting their behavior to the human’s elected strategy for a given task, facilitating mutuallyadaptive behavior within the team. While some situations entail explicit and easy-to-understand human top-down strategies, more often than not, human strategies tend to be implicit, ad hoc, exploratory, and difficult to describe. In order to facilitate mutually-adaptive human-agent team behavior, computational teammates must identify, adapt, and modify their behaviors to support human strategies with little or no a priori experience. This challenge may be achieved by training learning agents with examples of successful group strategies. Therefore, this paper focuses on an algorithmic approach to extract group strategies from multi-agent teaming behaviors in a game-theoretic environment: predator-prey pursuit. Group strategies are illuminated with a new method inspired from Graph Theory. This method treats agents as vertices to generate a timeseries of group dynamics and analytically compares timeseries segments to identify group coordinated behaviors. Ultimately, this approach may lead to the design of agents that can recognize and fall in line with strategies implicitly adopted by human teammates. This work can provide a substantial advance to the field of humanagent teaming by facilitating natural interactions within heterogeneous teams.

This paper proposes a novel solution for the Traveling Salesman Problem, a NP (non-deterministic polynomial-time) hardness problem. The algorithm presented in this paper offers an innovative solution by combining the qualities of a Nearest Neighbor (NN) greedy algorithm and the Genetic Algorithm (GA), by overcoming their weaknesses. The paper analyses the algorithm features/improvements and presents this implementation on a FPGA based target board. The experimental results of the algorithm, tested in software (Matlab) and implemented on a portable hardware (FPGA for GA, Raspberry Pi 3 for NN) shows a significant improvement: a shorter route, compared to NN , a shorter running time (less generations) compared to traditional GA , and reaching the optimal minimum (validated by Matlab). In real time, the algorithm runs on a handheld console, which can also act as a server, through a custom Android client application.

Varying resolution quality of operational data, size of targets, view occlusions, and large variation in sensors due to nature of overhead systems as compared to consumer devices contribute to degradation of the maritime vessel identification. We exploit the maritime domain characteristics to optimize and refine the deep learning Mask-RCNN framework for training generic maritime vessel classes. Maritime domain, compared to consumer domain, lack alternative targets that would be incorrectly associated as maritime vehicles: this allows us to relax the parameter constraints learned on urban natural scenes in consumer photos, adjust parameters of the model inference, and achieve robust performance and high AP measure for transfer learning scenarios. In this paper, we build upon this robust localization work, and extend our transfer learning work to new domains and datasets. We propose new approach for identifying specific category of maritime vessels and build a refined multi-label classifier that is based on deep Mask-RCNN features. The classifier is designed to be robust to domain transfer (e.g. different overhead maritime video feed), and to the noise in the data annotation (e.g. vessel is not correctly marked or label is ambiguous). We demonstrate superior category classification results of this low shot learning approach on publicly available MarDCT dataset.

Vehicle detection in aerial imagery has become tremendously a challenging task due to the low resolution characteristics of the aerial images. Super-Resolution; a technique which recovers high-resolution image from a single low-resolution image can be an effective approach to resolve this shortcoming. Hence, our prime focus is to design a framework for detecting vehicles in super resolved aerial images. Our proposed system can be represented as a combination of two deep sub-networks. The first sub-network aims to use a Generative Adversarial Network (GAN) for getting super resolved images. A GAN consists of two networks: a generator network and a discriminator network. It ensures recovery of photo-realistic images from down-sampled images. The second sub-network consists of a deep neural network (DNN)-based object detector for detecting vehicles in super resolved images. In our architecture, the Single Shot Multi Box Detector (SSD) is used for vehicle detection. The SSD generates fixed-size bounding boxes with predicting scores for different object class instances in those boxes. It also employs a non-maximum suppression step to produce final detections. In our algorithm, our deep SSD detector is trained with the predicted super resolved images and its performance is then compared with an SSD detector that is trained only on the low-resolution images. Finally, we compare the performance of our proposed pre-trained SSD detector on super-resolved images with an SSD that is trained only on the original high resolution images.

Gaining insight from unlabeled data is a widespread, challenging problem with many immediate applications. Clustering, dimensionality reduction for visualization, and anomaly detection are unsupervised learning solutions to this problem. Typically, the efficacy of these methods relies on obtaining sufficient amounts of data, presenting many challenges to cases where only limited data exists. In this work, we demonstrate that deep convolutional autoencoders can comfortably perform these tasks either directly or through manipulations of the latent space in a limited data setting. Clustering on common benchmark datasets produces comparable results to the current state-of-the-art in unsupervised classification. Clustering the activation maps of the encoding layers results in a form of unsupervised image segmentation with limited two-dimensional data. Visualizing the activation maps through dimension reduction demonstrates the possibilities of anomaly detection and semi-supervised learning. We are currently utilizing our versatile autoencoder to explore the ambitious task of finding anomalous and/or inconspicuous objects from single images.

This paper describes the coupling of Bayesian learning methods with realistic statistical models for randomly scattered signals. Such a formulation enables efficient learning of signal properties observed at sensors in urban and other complex environments. It also provides a realistic assessment of the uncertainties in the sensed signal characteristics, which is useful for calculating target class probabilities in automated target recognition. In the Bayesian formulation, the physics-based model for the random signal corresponds to the likelihood function, whereas the distribution for the uncertain signal parameters corresponds to the prior. Single and multivariate distributions for randomly scattered signals (as appropriate to single- and multiple-receiver problems, respectively) are reviewed, and it is suggested that the log-normal and gamma distributions are the most useful due to their physical applicability and the availability of Bayesian conjugate priors, which enable efficient refinement of the signal hyperparameters. Realistic simulations for sound propagation are employed to illustrate the Bayesian processing. The processing is found to be robust to mismatches between the simulated signal distributions and the assumed forms of the likelihood functions.

Object detection from images captured by Unmanned Aerial Vehicles (UAVs) are widely used for surveillance, precision agricultural, package delivery, aerial photography, among others. Very recently, a benchmark on object detection using UAVs collected images called VisDrone2018 has been released. However, large performance drop is observed when current state-of-the-art object detection approaches developed primarily for ground-to-ground images are directly applied on the VisDrone2018 dataset. For example, the best detection model on the VisDrone2018 has only achieved detection accuracy of 0.31 mAP, significantly lower than that of ground-based object detection. This performance drop is mainly caused by several challenges, such as 1) varying flying altitudes from 1000 feet to 10 feet, 2) different weather conditions like foggy, rainy and low-light 3) a wide range of camera viewing angles. To overcome these challenges, in this paper we propose to leverage a novel approach of adversarial training that aims to learn domain invariant features with respect to varying altitudes, viewing angles, weather conditions, and object scales. The adversarial training draws on “free” meta-data that comes with the UAV datasets providing information about the data themselves, such as heights, scene visibility, viewing angles, etc. We demonstrate the effectiveness of our proposed algorithm on the recently proposed UAVDT dataset, and also show it to generalize well when applied to a different VisDrone2018 dataset. We will also show robustness of the proposed approach to variations in altitude, viewing angle, weather, and object scale.

We focus on defending against adversarial attacks in deep neural networks using signal analysis technology. The method employs a novel signal processing theory as a defense to adversarial perturbations. The method neither modifies the protected network nor requires knowledge of the process for generating adversarial examples. Extensive evaluation experiments demonstrate the efficiency and effectiveness of the proposed adversarial defending method.

Future military coalition operations will increasingly rely on machine learning (ML) methods to improve situational awareness. The coalition context presents unique challenges for ML: the tactical environment creates significant computing and communications limitations while also having to deal with an adversarial presence. Further, coalition operations must operate in a distributed manner, while coping with the constraints posed by the operational environment. Envisioned ML deployments in military assets must be resilient to these challenges. Here, we focus on the susceptibility of ML models to be poisoned (during training) or fooled (after training) by adversarial inputs. We review recent work on distributed adversarial ML, and present new results from our own investigations into model poisoning attacks on distributed learning systems without a central parameter aggregation node.

Cyber-attackers are likely to exploit advances in artificial intelligence to achieve faster, stealthier, and more effective operational effects. Defenders need to keep pace by developing their own advances, which may preclude human-in-theloop decision making. Consequently, future systems will have to rely on automated reasoning and automated responses to ensure mission success and continuously adapt to an evolving adversary. Automated reasoning about defensive cyber responses is essentially sequential decision making based on the projection of possible futures from a current situation. This problem is especially complicated in cyberspace, however, because the current situation and future projections are highly uncertain. Our research tackles these challenges using the formal framework of partially observable Markov decision problems (POMDPs). We show how to break the “curse of dimensionality” that makes these problems intractable by computing approximate solutions using a Monte Carlo online planner that incorporates a computationally feasible simulation of the cyber security problem. Our simulation is an extension of MITRE’s Cyber Security Game simulator, which explores the mission-impact-focused strategies of an adaptive, intelligent attacker. Preliminary results on small problems, where the optimal solution can be calculated precisely, show that our approach consistently finds the optimal answer, not just a good approximation. We are in the process of increasing the fidelity of the simulator and POMDP representation to model more realistic cyber environments by increasing attacker and defender actions, increasing the variety of sensor types (including sensing of both actions and states), accounting for multiple incident effects, and improving the scaling properties.

Ongoing research with The International Technology Alliance in Distributed Analytics and Information Sciences (DAIS-ITA) aims to enable secure, dynamic, semantically-aware, distributed analytics for deriving situational understanding in future coalitions. This paper sets out an example military scenario and operations in a future time frame, to capture the expected battlespace context and key future challenges. Key considerations involve complex multi-actor situations, high complexity information, high tempo processing, all within human-machine hybrid-teams. The coalition composition of these teams is critical and all resources will be constrained. A phased operation is proposed across rural and urban operation involving a range of ISR sensors and autonomous devices. All these are subject to enemy action and perturbation and must be used across a highly contested and congested electromagnetic spectrum. Agile command and control is required across the coalition with information arriving from multiple sources and partners that may also be utilised for learning.

In cyber and threat intelligence areas, Indicators of Compromise (IOC) can be used as inputs to security controls to guide defense and mitigation activities. We propose a collaboration model in certain attributes in IOC model related to the (1) seriousness of the threat that the IOC triggers and (2) the confidence in the IOC detection or prediction are built based on a community or collaborative model. In this model, users can subscribe or introduce new IOCs based on their own/systems’ exposures or analysis. They can also assess IOCs created by others and vote to continuously change IOC seriousness and confidence values.

Current challenges in spectrum monitoring include radar emitter state identification and the ability to detect changes in radar activity. Recently, large labeled datasets and better compute power have led to improvements in the classification performance of deep neural network (DNN) models on structured data like time series and images. The reliance on large labeled dataset in order to achieve state of the art performance is a hindrance for machine learning applications especially in the area of radar, which tends to have a wealth of noisy and unlabeled data. Due to the abundance of unlabeled data, the problem of radar emitter and activity identification is commonly setup as a clustering problem, which requires no labels. The deep clustering approach uses an underlying deep feature extractor such as an autoencoder to learn a low dimensional feature representation in the service of facilitating a clustering task. In this paper, we will evaluate different clustering loss functions such as K-means for training DNNs, and we use radar emitter state and activity identification as our example task.

A Federated Learning approach consists of creating an AI model from multiple data sources, without moving large amounts of data across to a central environment. Federated learning can be very useful in a tactical coalition environment, where data can be collected individually by each of the coalition partners, but network connectivity is inadequate to move the data to a central environment. However, such data collected is often dirty and imperfect. The data can be imbalanced, and in some cases, some classes can be completely missing from some coalition partners. Under these conditions, traditional approaches for federated learning can result in models that are highly inaccurate. In this paper, we propose approaches that can result in good machine learning models even in the environments where the data may be highly skewed, and study their performance under different environments.

The Army Rapid Capabilities Office (RCO) sponsored a Blind Signal Classification Competition seeking algorithms to automatically identify the modulation schemes of RF signal from complex-valued IQ (in-phase quadrature) samples. Traditional spectrum sensing technology uses energy detection to detect the existence of RF signals but the RCO competition further aimed to detect the modulation scheme of signals without prior information. Machine Learning (ML) technologies have been widely used for blind signal classification problem. Traditional ML methods usually have two stages where the first stage is to manually extract the features of the IQ symbols by subject matter experts and the second stage is to feed the features to an ML algorithm (e.g., a support vector machine) to develop the classifier. The state-of-art technology is to apply deep learning technologies such as Convolutional Neural Network (CNN) or Recurrent Neural Network (RNN) directly to the complex-value IQ symbols to train a multi-class classifier. Our team, dubbed Deep Dreamers, participated in the RCO competition and placed 3rd out of 42 active teams across industry, academia, and government. In this work we share our experience and lessons learned from the competition. Deep learning methods such as CNN, Residual Neural Network (ResNet), and Long Short-Term Memory (LSTM) are the fundamental neural network layers we used to develop a multi-class classifier. None of our individual models were able to achieve a competitively high ranking in the competition. The key to our success was to use ensemble learning to average the outputs of multiple diverse classifiers. In order for ensemble methods to be more accurate than any of its base models; the base learners have to be as accurate as possible. We found that while ResNet was more accurate than the LSTM; the LSTM was less sensitive to deviations in the test set.

Tactical edge environments are highly distributed with a large number of sensing, computational, and communication nodes spread across large geographical regions, governments, and situated in unique operational environments. In such settings, a large number of observations and actions may occur across a large number of nodes but each node may only have a small number of these data locally. Further, there may be technical as well as policy constraints in aggregating all observations to a single node. Learning from all of the data may uncover critical correlations and insights. However, without having access to all the data, this is not possible. Recently proposed federated averaging approaches allow for learning a single model from data spread across multiple nodes and achieve good results on image classification tasks. However, this still assumes a sizable amount of data on each node and a small number of nodes. This paper investigates the properties of federated averaging for neural networks relative to batch sizes and number of nodes. Experimental results on a human activity dataset finds that (1) accuracy indeed drops as the number of nodes increase but only slightly, however (2) accuracy is highly sensitive to the batch size only in the federated averaging case.

Validation accuracy and test accuracy are necessary, but not sufficient, measures of a neural network classifier’s quality. A model judged successful by these metrics alone may nevertheless reveal serious flaws upon closer examination, such as vulnerability to adversarial attacks or a tendency to misclassify (with high confidence) real-world data different than that in its training set. It may also be incomprehensible to a human, basing its decisions on seemingly arbitrary criteria or overemphasizing one feature of the dataset while ignoring others of equal importance. While these problems have been the focus of a substantial amount of recent research, they are not prioritized during the model development process, which almost always maximizes validation accuracy to the exclusion of everything else. The product of such an approach is likely to fail in unexpected ways outside of the training environment. We believe that, in addition to validation accuracy, the model development process must give equal weight to other performance metrics such as explainability, resistance to adversarial attacks, and classification of out-of-distribution data. We incorporate these assessments into the model design process using free, readily available tools to differentiate between convolutional neural network classifiers trained on the notMNIST character dataset. Specifically, we show that ensemble and ensemble-like models with high cardinality outperform simpler models with identical validation accuracy by up to a factor of 5 on these other metrics.

Real-time object recognition systems are critical for several UAV applications since they provide fundamental semantic information of the aerial scene. In this study, we describe how image quality limits object detection frame-works such as YOLO which can distinguish 80 different object classes. This paper will focus on vehicles such as cars, trucks and buses. Pristine high-resolution images are degraded using different blurring functions, spatial resolution, reduced image contrast, additive noise and lossy compression. Object recognition results are significantly better after applying an image super-resolution algorithm to realistically simulated under-sampled imagery.

Recent technological advances provide the opportunities to bridge the physical world with cyber-space that leads to complex and multi-domain cyber physical systems (CPS) where physical systems are monitored and controlled using numerous smart sensors and cyber space to respond in real-time based on their operating environment. However, the rapid adoption of smart, adaptive and remotely accessible connected devices in CPS makes the cyberspace more complex and diverse as well as more vulnerable to multitude of cyber-attacks and adversaries. In this paper, we aim to design, develop and evaluate a distributed machine learning algorithm for adversarial resiliency where developed algorithm is expected to provide security in adversarial environment for critical mobile CPS.

Emerging cyber physical system (CPS) are expected to enhance the overall performance of the networked systems to provide reliable services and applications to their users. However, massive number of connectivities in CPS bring security vulnerabilities and the mobility adds more complexity for securing the mobile CPS. Any mobile CPS can be represented as a graph with connectivity as well as with interactions among a group of mobile CPS nodes that plays a major role as a medium for the propagation of wrong/right information, and influence its members in the mobile CPS. This problem has wide spread applications in viral information disseminating in mobile CPS, where a malicious mobile CPS node may wish to spread the rumor via the most influential individuals in mobile CPS. In this paper, we design, develop and evaluate a machine learning approach that is based on a set theoretic approach for optimizing the influence in mobile CPS. This problem has applications in civilian and military systems.

Target detection systems identify targets by localizing their coordinates on the input image of interest. This is ideally achieved by labeling each pixel in an image as a background or a potential target pixel. Deep Convolutional Neural Network (DCNN) classifiers have proven to be successful tools for computer vision applications. However, prior research confirms that even state of the art classifier models are susceptible to adversarial attacks. In this paper, we show how to generate adversarial infrared images by adding small perturbations to the targets region to deceive a DCNN-based target detector at remarkable levels. We demonstrate significant progress in developing visually imperceptible adversarial infrared images where the targets are visually recognizable by an expert but a DCNN-based target detector cannot detect the targets in the image.

Recently, intelligent machine agents, such as a deep neural network (DNN), have been showing unparalleled capabilities in recognizing visual patterns, objects, semantic activities/events embedded in real-world images and videos. Hence, there has been an increasing need to deploy DNNs, to a battlefield to provide the Solider with realtime situational understanding by capturing a holistic view of battlespace. Soldiers engaged in tactical operations can greatly benefit from leveraging advanced at-the-point-of-need data analytics running on multimodal and heterogeneous platforms in distributed and constrained network environments. The proposed work aims to decompose DNNs and then distribute over edge nodes in such a way that a trade-off between resources available in the constrained network and recognition performance can be optimized. In this work, we decompose DNNs into two stages: an initial stage on an edge device and the remaining portion running on an edge cloud. To effectively and efficiently divide DNNs into two separate stages, we will rigorously analyze multiple widely used DNN architectures with respect to its memory size and FLOPs (Floating Point Operations) per each layer. Based on these analyses, we will develop advanced splitting strategies for DNNs to handle various network constraints.

This paper presents the algorithms for tracking a moving object through video data using template matching. As the object translates and rotates, the template is adaptively updated so that the object is never lost while in frame. The algorithms were developed in MATLAB and applied to a video of a quadcopter in flight in both visible and infrared imagery. The normalized cross-correlation algorithm is the core of the research, providing an invariant of scale method to perform the template match. Then a bounding box is applied to the matched area and center of mass centroiding allow the object to be tracked frame-to-frame.

Machine learning based perception algorithms are increasingly being used for the development of autonomous navigation systems of self-driving vehicles. These vehicles are mainly designed to operate on structured roads or lanes and the ML algorithms are primarily used for functionalities such as object tracking, lane detection and semantic understanding. On the other hand, Autonomous/ Unmanned Ground Vehicles (UGV) being developed for military applications need to operate in unstructured, combat environment including diverse off-road terrain, inclement weather conditions, water hazards, GPS denied environment, smoke etc. Therefore, the perception algorithm requirements are different and have to be robust enough to account for several diverse terrain conditions and degradations in visual environment. In this paper, we present military-relevant requirements and challenges for scene perception that are not met by current state-of-the-art algorithms, and discuss potential strategies to address these capability gaps. We also present a survey of ML algorithms and datasets that could be employed to support maneuver of autonomous systems in complex terrains, focusing on techniques for (1) distributed scene perception using heterogeneous platforms, (2) computation in resource constrained environment (3) object detection in degraded visual imagery.

In recent years, the field of automated machine learning (autoML) has quickly attracted significant attention both in academia and industry. The driving force is to reduce the amount of human intervention required to process data and create models for classification and prediction, a tedious and arbitrary process for data scientists that may not often result in achieving a global optimum with respect to multiple objectives. Moreover, existing autoML techniques rely on extremely large collections of relatively clean training data, which is not typical of Multi-Domain Battle (MDB) applications. In this paper, we describe a methodology to optimize underwater seafloor detection for airborne bathymetric lidar, an application domain with sparse truth data, leveraging evolutionary algorithms and genetic programming. Our methodology uses the Evolutionary Multi-objective Algorithm Design Engine (EMADE) and a radiometric waveform simulator generating millions of waveforms from which genetic programming techniques select optimal signal processing techniques and their parameters given the goal of reducing Total Propagated Uncertainty (TPU). The EMADE affords several benefits not found in other autoML solutions, including the ability to stack machine learning models, process time-series data using dozens of signal-processing techniques, and efficiently evaluate algorithms on multiple objectives. Given the lack of truth data, we tune EMADE to produce detection algorithms that improve accuracy and reduce relevant measurement uncertainties for a wide variety of operational and environmental scenarios. Preliminary testing indicates successfully reducing TPU and reducing over- and under-prediction errors by 13.8% and 68.2% respectively, foreshadowing using EMADE to assist in future MDB-application algorithm development.

Better understanding of Multi Domain Battle (MDB) challenges in complex military environments may start by gaining a basic scientific appreciation of the level of generalization and scalability offered by Machine Learning (ML) solutions designed, trained and optimized to achieve a single, specific task, continuously daytime and nighttime. We examine the generalization and scalability promises of a modern deep ML solution, applied to a unique spatial-spectral dataset that consists of blackbody calibrated, longwave infrared spectra of a fixed target site containing three painted metal surrogate tanks deployed in a field of mixed vegetation. Data was collected at roughly six minute intervals, nearly continuously, for over a year. This includes collection in many atmospheric conditions (rain, snow, sleet, fog, etc.) throughout the year. This paper focuses on data collected by a Telops Hyper-Cam from a 65 meter observation tower located at slant range of roughly 550 meters, from the targets. The dataset is very complex. There are no obvious spectral signatures from the target surfaces. The complexity is due in part to the natural variations of the various types of vegetation, cloud presence, and the changing solar loading conditions over time. This is precisely the environment MDB applications must function in. We detail some of the many training sets extracted to train different deep learning stacked auto encoder networks. We present performance results with receiver operator characteristic curves, confusion matrices, metric-vs-time plots, and classification maps. We show performance of ML models trained with data from various time windows, including over complete diurnal cycles and their performance processing data from different days and environmental conditions.

Increasingly the design of chemical detection alarm algorithms to alert Soldiers of danger grows more complex as new threats emerge. These algorithms need to be robust enough to prevent false alarms to interferents and sensitive enough to alarm to the incredibly small doses that could prove lethal. The design of these algorithms have left a plethora of data that can be leveraged and utilized in a variety of machine learning (ML) techniques. ML is a field of computer science that uses a set of programming and statistical techniques to enable computers to “learn” from input data without being explicitly programmed. Presented is an application of ML to change two independent fielded chemical detectors into an orthogonal system to improve detection algorithms. The approach models the data from an ion-mobility spectrometer (IMS) and a photoionization detector containing electrochemical sensors (PIDECS) to train a ML model (MLA). The semi-supervised MLA is trained using a supervised learning data set, composed of partially labeled data from the heterogeneous instruments, and then fine-tuned using an unsupervised learning algorithm. The MLA correctly identifies two chemical species with over-lapping IMS detection windows. ML can be utilized to improve the ability of currently fielded detectors or future devices to accurately label chemical unknowns given the parameters of detection. The techniques discussed here presents a starting point for improving current and future alarm algorithms with ML.

There is a need for Radio Frequency Signal Classification (RF-Class) toolbox which can monitor, detect, and classify wireless signals. Rapid developments in the unmanned aerial systems (UAS) have made its usage in a variety of offensive as well as defensive applications especially in military, high priority and sensitive government sites. The ability to accurately classify over-the-air radio signals will provide insights into spectrum utilization, device fingerprinting and protocol identification. These insights can help the Warfighter to constantly be informed about adversarys transmitters capabilities without their knowledge. Recently, few researches have proposed feature-based machine learning techniques to classify RF signals. However, these researches are mostly evaluated on simulated environments, less accurate, and failed to explore advance machine learning techniques. In this research, we proposed a feature-engineering based signal classification (RF-class) toolbox which implements RF signal detection, Cyclostationary Features Extraction and Feature engineering, Automatic Modulation Recognition to automatically recognize modulation as well as sub-modulation types of the received signal. To demonstrate the feasibility and accuracy of our approach, we have evaluated the performance on a real environment with an UAS (Drone DJI Phantom 4). Our initial experimental result showed that we were able to detect presence of drone signal successfully when power on and transmitting. And further experiments are under progress.

Artificial Intelligence (AI) technology is being applied successfully in a number of domains. Advances in low cost, high performance computing platforms have made AI approaches sufficiently scalable to be applied in high volume, commercial applications. The true promise of AI in modeling human intelligence remains elusive. Current approaches can simulate a small subset of the many processes that make up human cognition, and yet it would be of huge benefit to be able to integrate expert human decision making in AI applications. In this paper, we present a pragmatic approach that can be used to capture expert human decision making within a limited domain of expertise. We propose an approach that automates the Analytic Hierarchy Process in order to capture a model of expert decision making from observational data. While this is not a general solution, it provides a workable approach for AI applications dealing with well defined, limited domains of knowledge.