Maritime platform defense with deep reinforcement learning

Jared Markowitz; Ryan Sheffield; Galen Mullins

doi:10.1117/12.2618808

6 June 2022 Maritime platform defense with deep reinforcement learning

Jared Markowitz, Ryan Sheffield, Galen Mullins

Proceedings Volume 12113, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV; 121131B (2022) https://doi.org/10.1117/12.2618808
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

We present a method for applying deep reinforcement learning to maritime platform defense, showing how to successfully train agents to schedule countermeasures for defending a fleet of ships against stochastic raids in a simulated environment. Our Schedule Evaluation Simulation (SEvSim) environment was developed using extensive input from subject matter experts and contains realistic threat characteristics, weapon efficacies, and constraints among weapons. Our approach includes novelty in both the representation of the system state and the neural network architecture: threats are represented as vectors containing information on the projected effect of different scheduling actions on their viability and fed to network input “slots” in randomized locations. Agents are trained using Proximal Policy Optimization, a state-of-the-art method for model-free learning. We evaluate the performance of our approach, finding that it learns scheduling strategies that both reliably neutralize threats and conserve inventory. We subsequently discuss the remaining challenges involved in bringing neural-network-based control to realization in this application space. Among these challenges are the needs to integrate humans into the loop, provide safety assurances, and enable continual learning.

Conference Presentation

Citation Download Citation

Jared Markowitz, Ryan Sheffield, and Galen Mullins "Maritime platform defense with deep reinforcement learning", Proc. SPIE 12113, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications IV, 121131B (6 June 2022); https://doi.org/10.1117/12.2618808

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available