Probabilistic visual and electromagnetic data fusion for robust drift-free sequential mosaicking: application to fetoscopy

Marcel Tella-Amo; Loic Peter; Dzhoshkun I. Shakir; Jan Deprest; Danail Stoyanov; Juan E. Iglesias; Tom Vercauteren; Sébastien Ourselin

doi:10.1117/1.JMI.5.2.021217

22 February 2018 Probabilistic visual and electromagnetic data fusion for robust drift-free sequential mosaicking: application to fetoscopy

Marcel Tella-Amo, Loic Peter, Dzhoshkun I. Shakir, Jan Deprest, Danail Stoyanov, Juan E. Iglesias, Tom Vercauteren, Sébastien Ourselin

Author Affiliations +

Journal of Medical Imaging, Vol. 5, Issue 2, 021217 (February 2018). https://doi.org/10.1117/1.JMI.5.2.021217

Abstract

The most effective treatment for twin-to-twin transfusion syndrome is laser photocoagulation of the shared vascular anastomoses in the placenta. Vascular connections are extremely challenging to locate due to their caliber and the reduced field-of-view of the fetoscope. Therefore, mosaicking techniques are beneficial to expand the scene, facilitate navigation, and allow vessel photocoagulation decision-making. Local vision-based mosaicking algorithms inherently drift over time due to the use of pairwise transformations. We propose the use of an electromagnetic tracker (EMT) sensor mounted at the tip of the fetoscope to obtain camera pose measurements, which we incorporate into a probabilistic framework with frame-to-frame visual information to achieve globally consistent sequential mosaics. We parametrize the problem in terms of plane and camera poses constrained by EMT measurements to enforce global consistency while leveraging pairwise image relationships in a sequential fashion through the use of local bundle adjustment. We show that our approach is drift-free and performs similarly to state-of-the-art global alignment techniques like bundle adjustment albeit with much less computational burden. Additionally, we propose a version of bundle adjustment that uses EMT information. We demonstrate the robustness to EMT noise and loss of visual information and evaluate mosaics for synthetic, phantom-based and ex vivo datasets.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Marcel Tella-Amo, Loic Peter, Dzhoshkun I. Shakir, Jan Deprest, Danail Stoyanov, Juan E. Iglesias, Tom Vercauteren, and Sébastien Ourselin "Probabilistic visual and electromagnetic data fusion for robust drift-free sequential mosaicking: application to fetoscopy," Journal of Medical Imaging 5(2), 021217 (22 February 2018). https://doi.org/10.1117/1.JMI.5.2.021217

Received: 15 September 2017; Accepted: 23 January 2018; Published: 22 February 2018

ACCESS THE FULL ARTICLE

JOURNAL ARTICLE
16 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY