Drone based user and heading detection using deep learning and stereo vision

Lynne Grewe; Garrett Stevenson

doi:10.1117/12.2518266

7 May 2019 Drone based user and heading detection using deep learning and stereo vision

Lynne Grewe, Garrett Stevenson

Proceedings Volume 11018, Signal Processing, Sensor/Information Fusion, and Target Recognition XXVIII; 110180X (2019) https://doi.org/10.1117/12.2518266
Event: SPIE Defense + Commercial Sensing, 2019, Baltimore, MD, United States

Abstract

The problem of assisting a low-vision person with environment awareness using a drone is addressed. Specifically, the first stage task of detecting the User and their heading which does not require any user adaptive training is tackled. The modalities of 3D and 2D vision on a drone are compared for this task. 3D data is provided using a stereo sensor mounted on the drone that communicates using RF to a mobile device based android application. For the task of localization, a Single Shot MulitBox Detector is utilized. Different networks in terms of input modalities and related structure are developed including a 2D only network and a 3D+2D fused network. Performance of these networks are compared and results discused. In addition, a comparison of retrained networks versus training from scratch is made. In all cases, approximately 34,000 user heading samples were collected for training. Real data from outdoor drone flights that communicate with the Android based application are shown. Detecting both the user in the scene and their heading is an important first step necessary in a drone-based system that helps low-vision persons with environment awareness. The success and challenges faced are presented along with future avenues of work.

Citation Download Citation

Lynne Grewe and Garrett Stevenson "Drone based user and heading detection using deep learning and stereo vision", Proc. SPIE 11018, Signal Processing, Sensor/Information Fusion, and Target Recognition XXVIII, 110180X (7 May 2019); https://doi.org/10.1117/12.2518266

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available