The development of an augmented reality environment that combines laparoscopic video and ultrasound imaging
for image-guided minimally invasive abdominal surgical procedures, such as partial nephrectomy and radical
prostatectomy, is an ongoing project in our laboratory. Our system overlays magnetically tracked ultrasound
images onto endoscopic video to create a more intuitive visualization for mapping lesions intraoperatively and to
give the ultrasound image context in 3D space. By presenting data in a common environment, this system will
allow surgeons to visualize the multimodality information without having to switch between different images.
A stereoscopic laparoscope from Visionsense Limited enhances our current system by providing surgeons with
additional visual information through improved depth perception. In this paper, we develop and validate a
calibration method that determines the transformation between the images from the stereoscopic laparoscope
and the 3D locations of structures represented by a tracked laparoscopic ultrasound probe. We first calibrate
the laparoscope with a checkerboard pattern and measure how accurate the transformation from image space
to tracking space is. We then perform a target localization task using our fused environment. Our initial
experience has demonstrated an RMS registration accuracy in 3D of 2.21mm for the laparoscope and 1.16mm for
the ultrasound in a working volume of 0.125m<sup>3</sup>, indicating that magnetically tracked stereoscopic laparoscope
and ultrasound images may be appropriately combined using magnetic tracking as long as steps are taken to
ensure that the magnetic field generated by the system is not distorted by surrounding objects close to the
A 2D ultrasound enhanced virtual reality surgical guidance system has been under development for some time in
our lab. The new surgical guidance platform has been shown to be effective in both the laboratory and clinical
settings, however, the accuracy of the tracked 2D ultrasound has not been investigated in detail in terms of the
applications for which we intend to use it (i.e., mitral valve replacement and atrial septal defect closure). This
work focuses on the development of an accuracy assessment protocol specific to the assessment of the calibration
methods used to determine the rigid transformation between the ultrasound image and the tracked sensor.
Specifically, we test a Z-bar phantom calibration method and a phantomless calibration method and compared
the accuracy of tracking ultrasound images from neuro, transesophageal, intracardiac and laparoscopic ultrasound
transducers. This work provides a fundamental quantitative description of the image-guided accuracy that can
be obtained with this new surgical guidance system.
To completely remove a tumor from a diseased kidney, while minimizing the resection of healthy tissue, the surgeon must be able to accurately determine its location, size and shape. Currently, the surgeon mentally estimates these parameters by examining pre-operative Computed Tomography (CT) images of the patient's anatomy. However, these images do not reflect the state of the abdomen or organ during surgery. Furthermore, these images can be difficult to place in proper clinical context. We propose using Ultrasound (US) to acquire images of the tumor and the surrounding tissues in real-time, then segmenting these US images to present the tumor as a three dimensional (3D) surface. Given the common use of laparoscopic procedures that inhibit the range of motion of the operator, we propose segmenting arbitrarily placed and oriented US slices individually using a tracked US probe. Given the known location and orientation of the US probe, we can assign 3D coordinates to the segmented slices and use them as input to a 3D surface reconstruction algorithm. We have implemented two approaches for 3D segmentation from freehand 2D ultrasound. Each approach was evaluated on a tissue-mimicking phantom of a kidney tumor. The performance of our approach was determined by measuring RMS surface error between the segmentation and the known gold standard and was found to be below 0.8 mm.