Modeling and decoding event-based sensor lightning response

Brian J. McReynolds; Rui Graca; Daniel O'Keefe; Rachel Oliver; Richard Balthazor; Nathaniel George; Matthew McHarg

doi:10.1117/12.2674435

3 October 2023 Modeling and decoding event-based sensor lightning response

Brian J. McReynolds, Rui Graca, Daniel O'Keefe, Rachel Oliver, Richard Balthazor, Nathaniel George, Matthew McHarg

Proceedings Volume 12693, Unconventional Imaging, Sensing, and Adaptive Optics 2023; 1269313 (2023) https://doi.org/10.1117/12.2674435
Event: SPIE Optical Engineering + Applications, 2023, San Diego, California, United States

Abstract

Neuromorphic cameras, or Event-based Vision Sensors (EVS), operate in a fundamentally different way than conventional frame-based cameras. Their unique operational paradigm results in a sparse stream of high temporal resolution output events which encode pixel-level brightness changes with low-latency and wide dynamic range. Recently, interest has grown in exploiting these capabilities for scientific studies; however, accurately reconstructing signals from the output event stream presents a challenge due to physical limitations of the analog circuits that implement logarithmic change detection. In this paper, we present simultaneous recordings of lightning strikes using both an event camera and frame-based high-speed camera. To our knowledge, this is the first side-by-side recording using these two sensor types in a real-world scene with challenging dynamics that include very fast and bright illumination changes. Our goal in this work is to accurately map the illumination to EVS output in order to better inform modeling and reconstruction of events from a real-scene. We first combine lab measurements of key performance metrics to inform an existing pixel model. We then use the high-speed frames as signal ground truth to simulate an event stream and refine parameter estimates to optimally match the event-based sensor response for several dozen pixels representing different regions of the scene. These results will be used to predict sensor response and develop methods to more precisely reconstruct lightning and sprite signals for Falcon ODIN, our upcoming International Space Station neuromorphic sensing mission.

Conference Presentation

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Brian J. McReynolds, Rui Graca, Daniel O'Keefe, Rachel Oliver, Richard Balthazor, Nathaniel George, and Matthew McHarg "Modeling and decoding event-based sensor lightning response", Proc. SPIE 12693, Unconventional Imaging, Sensing, and Adaptive Optics 2023, 1269313 (3 October 2023); https://doi.org/10.1117/12.2674435

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available