Dense 3D organ modeling from a laparoscopic video

Ryoshuke Maekawa; Hidehiko Shishido; Yoshinari Kameda; Itaru Kitahara

doi:10.1117/12.2590732

20 April 2021 Dense 3D organ modeling from a laparoscopic video

Ryoshuke Maekawa, Hidehiko Shishido, Yoshinari Kameda, Itaru Kitahara

Proceedings Volume 11792, International Forum on Medical Imaging in Asia 2021; 117920L (2021) https://doi.org/10.1117/12.2590732
Event: International Forum on Medical Imaging in Asia 2021, 2021, Taipei, Taiwan

Abstract

Laparoscopic surgery provides for patients such advantages as a small incision range and quick postoperative recovery. Unfortunately, surgeons struggle to grasp 3D spatial relationships in the abdominal cavity. Methods have been proposed to present the 3D information of the abdominal cavity using AR or VR. Although 3D geometrical information is crucial to perform such methods, it is difficult to reconstruct dense 3D organ shapes using a feature-point-based 3D reconstruction method such as structure from motion (SfM) due to the appearance characteristics of organs (e.g., texture-less and glossy). Our research solves this problem by estimating depth information from laparoscopic images using deep learning. We constructed a training dataset from both RGB and depth images with an RGB-D camera, implemented a depth image generator by applying a generative adversarial network (GAN), and generated a depth image from a single-shot RGB image. By calibration with a laparoscopic camera and an RGB-D camera, the laparoscopic image was transformed to an RGB image. We generated depth images by inputting the transformed laparoscopic images into a GAN generator. The scale parameter of the depth image with real-world dimensions was calculated by comparing the depth value and the 3D information estimated by SfM. Consequently, the density of the organ model increased by back-projecting the depth image to the 3D space.

Citation Download Citation

Ryoshuke Maekawa, Hidehiko Shishido, Yoshinari Kameda, and Itaru Kitahara "Dense 3D organ modeling from a laparoscopic video", Proc. SPIE 11792, International Forum on Medical Imaging in Asia 2021, 117920L (20 April 2021); https://doi.org/10.1117/12.2590732

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available