Augmenting real-time video with virtual models for enhanced visualization for simulation, teaching, training and guidance

Michael Potter; Alexander Bensch; Alexander Dawson-Elli; Cristian A. Linte

doi:10.1117/12.2082545

Translator Disclaimer

17 March 2015 Augmenting real-time video with virtual models for enhanced visualization for simulation, teaching, training and guidance

Michael Potter, Alexander Bensch, Alexander Dawson-Elli, Cristian A. Linte

Author Affiliations +

Proceedings Volume 9416, Medical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment; 941615 (2015) https://doi.org/10.1117/12.2082545
Event: SPIE Medical Imaging, 2015, Orlando, Florida, United States

Abstract

In minimally invasive surgical interventions direct visualization of the target area is often not available. Instead, clinicians rely on images from various sources, along with surgical navigation systems for guidance. These spatial localization and tracking systems function much like the Global Positioning Systems (GPS) that we are all well familiar with. In this work we demonstrate how the video feed from a typical camera, which could mimic a laparoscopic or endoscopic camera used during an interventional procedure, can be used to identify the pose of the camera with respect to the viewed scene and augment the video feed with computer-generated information, such as rendering of internal anatomy not visible beyond the imaged surface, resulting in a simple augmented reality environment. This paper describes the software and hardware environment and methodology for augmenting the real world with virtual models extracted from medical images to provide enhanced visualization beyond the surface view achieved using traditional imaging. Following intrinsic and extrinsic camera calibration, the technique was implemented and demonstrated using a LEGO structure phantom, as well as a 3D-printed patient-specific left atrial phantom. We assessed the quality of the overlay according to fiducial localization, fiducial registration, and target registration errors, as well as the overlay offset error. Using the software extensions we developed in conjunction with common webcams it is possible to achieve tracking accuracy comparable to that seen with significantly more expensive hardware, leading to target registration errors on the order of 2 mm.

Citation Download Citation

Michael Potter, Alexander Bensch, Alexander Dawson-Elli, and Cristian A. Linte "Augmenting real-time video with virtual models for enhanced visualization for simulation, teaching, training and guidance", Proc. SPIE 9416, Medical Imaging 2015: Image Perception, Observer Performance, and Technology Assessment, 941615 (17 March 2015); https://doi.org/10.1117/12.2082545

ACCESS THE FULL ARTICLE