This PDF file contains the front matter associated with SPIE Proceedings Volume 12099, including the Title Page, Copyright information, Table of Contents, and Conference Committee listings.

As more humans settle in dense urban areas, the effect of natural or anthropogenically induced shocks at these locations has an increased potential to impact larger numbers of individuals. In particular, a disruption to the delivery of goods and services can leave large portions of the population in a vulnerable state. Research suggests that resilience to shocks is a function of physical fortifications and social processes, such as levees and critical infrastructure, the strength of social networks, or community efficacy, and trust. While physical fortifications are relatively easy to identify and catalog, the measurement of social processes is more difficult due to data limitations and geographic constraints. Recent work has shown that certain types of infrastructure may correlate with social processes that enhance community resilience; however, the ability to assess where and to what extent that infrastructure exists depends on a complete representation of the built environment. OpenStreetMap (OSM) and Google Places are two sources of data commonly used to identify the location and type of infrastructure but can display varying degrees of completeness depending on geographic location. We address this limitation by applying a Convolution Neural Network (CNN) to remotely sensed data from Sentinel-2 to estimate the density and type of infrastructure. We compare the classification results to known infrastructure locations from OSM data. Our results show that the CNN classifier performs well and may be used to augment incomplete data sets for a deeper understanding of the prevalence of infrastructure associated with social processes that enhance community resilience.

This paper proposes an automatic ship detection approach in Synthetic Aperture Radar (SAR) Images using YOLO deep learning framework. The You Only Look Once (YOLO) model was initially introduced as the first object detection model that combined bounding box prediction and objects classification into a single end-to-end differentiable network. We train the YOLO model on our dataset in this paper for our detector to learn to detect objects in SAR images such as ships. YOLO test results showed an increase in the accuracy of ship detection at Cyprus’s Coast and can be applied in the field of ship detection.

We propose Cross-KD, a novel framework for knowledge distillation across modalities from Electro-Optical (EO) to Synthetic Aperture Radar (SAR). The Cross-KD approach is response-based and takes into consideration the differences in network size and feature representations in the two modalities. The proposed training includes two stages consisting of i) EO network training and ii) SAR network training with transfer learning and knowledge distillation from the EO network. Knowledge distillation (KD) is performed at the soft output level, allowing features in the EO and SAR networks to be different. The Cross-KD model is agnostic in the selection of network backbone and does not place any constraints on the network architecture, thus making knowledge transfer applicable even from a smaller network to a larger network. We test our framework on a recent EO-SAR coupled dataset with promising results on SAR image classification. Cross-KD achieves performance gains for each component of the model, as evidenced in our ablation studies resulting in 93.98% mean per class accuracy.

EXP’s IC3D system classifies SAR or LiDAR derived 3D point clouds using a deep representation learning approach, producing as output a vector of categorical posterior probabilities of target classifications. Such posterior probabilities are suitable observational inputs to a Bayesian belief network (BBN), such as the EXP Shadow Compass system. In concert with conditional probabilities of intermediate events depending on the observation states, the Bayesian network computes posterior probabilities for events conditionally dependent on those intermediate event states. We demonstrate this approach by computing posterior event probabilities for sample analyst scenarios with intermediate event conditional probabilities specified by analysts. Future work could include extending the Bayesian network approach to discovery of the network topology from analyst data.

Point cloud completion aims to infer missing regions of a point cloud, given an incomplete point cloud. Like image inpainting, in the 2D domain, point cloud completion offers a way to recreate an entire point cloud, given only a subset of the information. However, current applications study only synthetic datasets with artificial point removal, such as the Completion3D dataset. Although these datasets are valuable, they are an artificial problem set that we can not apply to real-world data. This paper draws a parallel between point cloud completion and occlusion reduction in aerial lidar scenes. We propose a crucial change in the hierarchical sampling using selforganizing maps to propose new points representing the scene in a reduced resolution. These new points are a weighted combination of the original set using spatial and feature information. A new set of proposed points is more powerful than simply sampling existing points. We demonstrate this sampling technique by replacing the farthest point sampling in the Skip-attention Network with Hierarchical Folding (SA-Net) and show a significant increase in the overall results using the Chamfers distance as our metric. We also show that we can use this sampling method in the context of any technique which uses farthest point sampling.

Deep learning-based object detection and classification in 3D point clouds has numerous applications including defense, autonomous driving, and augmented reality. A challenge in applying deep learning to point clouds is the frequent scarcity of labeled data. Often, one must manually label a large quantity of data for the model to be useful in application. To overcome this challenge, active learning provides a means of minimizing the manual labeling required. The crux of active learning algorithms is defining and calculating the potential added “value” of labeling each unlabeled sample. We introduce a novel active learning algorithm, LOCAL, with an anchorbased object detection architecture, a modified object matching strategy, and an acquisition metric designed for object detection in any dimension. We compare the performance of common acquisition functions to our novel metric that utilizes all of the model outputs—including both bounding box localizations and softmax classification scores—to capture both the classification and spatial uncertainty in the model. Finally, we identify opportunities for further exploration, such as alternative measures of spatial uncertainty as well as increasing the stochasticity of the model in order to improve robustness of the algorithm.

A number of real-time object detection, tracking, and autonomy artificial intelligence (AI) and machine learning (ML) algorithms are being proposed for unmanned aerial vehicles (UAVs). A big challenge is can we stress test these algorithms, identify their strengths and weaknesses, and assess if the UAV is safe and trustworthy? The process of collecting real-world UAV data is costly, time consuming, and riddled by lack of quality geospatial ground truth and metadata. Herein, we outline a fully automated framework and work ow to address the above challenges using free or low-cost assets, the photorealistic Unreal Engine (UE), and AirSim aerial platform simulator. Specifically, we discuss the rapid prototyping of an outdoor environment combined with the robotic operating system (ROS) for abstracting UAV data collection, control, and processing. Real and accurate ground truth is collected and metrics are presented for individual frame and entire flight collection evaluation. Metrics recorded and analyzed include percentage of scene mapped, 3D mapping accuracy, time to complete task, object detection and tracking statistics, battery usage, altitude (from ground), collisions, and other statistics. These metrics are computed in general and with respect to context, e.g., clutter, view angle, etc. Overall, the proposed work is an automated way to explore UAV operation before real-world testing or deployment. Promising preliminary results are discussed for an outdoor environment with vegetation, short and long range objects, buildings, people, vehicles, and other features for a UAV performing loitering and interrogation.

For many intelligence sources, reliable independent algorithms exist for interpreting the data and reporting relevant information to analysts. However, achieving the necessary cross-source data fusion from these sources and algorithmic outputs to achieve true sensemaking can be challenging. This is especially true at the individual object level, given the sources' highly variable spatiotemporal resolutions and uncertainties. We have developed a framework for merging automatic target recognition (ATR) algorithms and their outputs to produce a sensor-agnostic means of object level change detection to establish the necessary patterns-of-life for big picture sensemaking, activity-based intelligence, and autonomous decision making.

Sea navigation and operations within areas of interest has been a major focus of naval research. Documents such as Raster Navigational Charts (RNC) that help with sea navigation tasks are critically important. A RNC is a copy of a navigational paper chart in image form. Therefore, RNC contains important information such as navigational channels, water depths, rocky areas etc. However, a RNC is hard to interpret by computers and even humans as it contains very dense information due to the different layers of drawings from the information mentioned above. In this paper, we introduce a reverse engineering approach using computer vision to extract features from the RNC image. We use optical character recognition to extract text features and templates matching for symbolic features. With the new approach, we show that RNC will become machine readable, and the features extracted can be used to draw tactical regions of interest.

Machine learning has exhibited significant advances through the development and application of new deep learning methods. Convolutional neural networks (CNNs) in particular are a powerful tool for object recognition in imagery and are widely used in for computer vision applications. Implementing a CNN solution involves decisions about the parameters defining the architecture, including the convolutional filter window size and number of filters, the size of the latent space, and the number of hidden layers. Generally, developers have chosen these parameters by relying on heuristics or empirical investigations. In this study, we build on previous research to understand the trade-offs associated with these design choices for a CNN. The approach explicitly models the performance, as measured by the correct classification rate, and the cost, as measured by computer times for training and testing. We develop a performance model that captures these measures as a function of the design parameters and can guide developers in assessing the tradeoffs.

Because of weight, power, and cost constraints, most unmanned aircraft systems (UAS) contain monocular camera systems. Real-time structure from motion (SfM) algorithms are required for monocular UAS systems to sense and autonomously navigate 3D environments. The SfM algorithm must be designed to work near real-time and handle the wide variety of possible extrinsic parameters produced by UAS image pairs. Common rigid epipolar rectification techniques (homography-based rectification) do not accurately model epipolar geometries with epipoles close to or contained within the image bounds. Common UAS movement types, translation along lens axis, tight radial turns, circular patterns with GPS locked camera focus, can all produce epipolar geometries with epipoles inside the camera frame. Using a generalized epipolar rectification technique, all extrinsic UAS movement types can be handled, and optimized image block matching techniques can be used to produce disparity/depth maps. The majority of UASs contain GPS/IMU/magnetometer modules. These modules provide absolute camera extrinsic parameters for every image. The essential and fundamental matrices between image pairs can be calculated from these absolute extrinsics. SfM is performed, depth images are produced, and the camera space pixel values can be projected as point clouds in three-dimensional space. These point clouds provide scene understanding that can be used for autonomous reasoning systems.

The machine learning community has seen an explosion in the sophistication of adversarial attacks against deep neural network-based computer vision models. In particular, researchers have successfully used adversarial patterns to trigger false positive or false negative results in both research and real-world settings. However, researchers have not yet codified performance metrics for evaluating the efficacy of attack techniques. This evaluation is needed to adequately assess performance improvements of novel adversarial attack methods. This study aims to contribute the following: adversarial pattern performance metrics, demonstration of each metric’s strengths and contributions on a case study, and an initial standardized performance evaluation strategy for novel adversarial pattern attacks. We train state-of-the-art deep neural network-based object detection models on an open-source dataset. We then use these trained models to evaluate trained adversarial patterns for both false positive and false negative attacks and evaluate their performance using our suite of metrics in order to establish and codify a workflow to be used when evaluating future adversarial pattern algorithms.