Dense surface reconstruction using a learning-based monocular vSLAM model for laparoscopic surgery

James Yu; Kelden Pruitt; Nati Nawawithan; Brett A. Johnson; Jeffrey Gahan; Baowei Fei

doi:10.1117/12.3008768

29 March 2024 Dense surface reconstruction using a learning-based monocular vSLAM model for laparoscopic surgery

James Yu, Kelden Pruitt, Nati Nawawithan, Brett A. Johnson, Jeffrey Gahan, Baowei Fei

Proceedings Volume 12928, Medical Imaging 2024: Image-Guided Procedures, Robotic Interventions, and Modeling; 129280J (2024) https://doi.org/10.1117/12.3008768
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Abstract

Augmented reality (AR) has seen increased interest and attention for its application in surgical procedures. AR-guided surgical systems can overlay segmented anatomy from pre-operative imaging onto the user’s environment to delineate hard-to-see structures and subsurface lesions intraoperatively. While previous works have utilized pre-operative imaging such as computed tomography or magnetic resonance images, registration methods still lack the ability to accurately register deformable anatomical structures without fiducial markers across modalities and dimensionalities. This is especially true of minimally invasive abdominal surgical techniques, which often employ a monocular laparoscope, due to inherent limitations. Surgical scene reconstruction is a critical component towards accurate registrations needed for AR-guided surgery and other downstream AR applications such as remote assistance or surgical simulation. In this work, we utilize a state-of-the-art (SOTA) deep-learning-based visual simultaneous localization and mapping (vSLAM) algorithm to generate a dense 3D reconstruction with camera pose estimations and depth maps from video obtained with a monocular laparoscope. The proposed method can robustly reconstruct surgical scenes using real-time data and provide camera pose estimations without stereo or additional sensors, which increases its usability and is less intrusive. We also demonstrate a framework to evaluate current vSLAM algorithms on non-Lambertian, low-texture surfaces and explore using its outputs on downstream tasks. We expect these evaluation methods can be utilized for the continual refinement of newer algorithms for AR-guided surgery.

Conference Presentation

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

James Yu, Kelden Pruitt, Nati Nawawithan, Brett A. Johnson, Jeffrey Gahan, and Baowei Fei "Dense surface reconstruction using a learning-based monocular vSLAM model for laparoscopic surgery", Proc. SPIE 12928, Medical Imaging 2024: Image-Guided Procedures, Robotic Interventions, and Modeling, 129280J (29 March 2024); https://doi.org/10.1117/12.3008768

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available