This PDF file contains the front matter associated with SPIE Proceedings Volume 8316, including the Title Page, Copyright information, Table of Contents, and the Conference Committee listing.

Intraoperative cone-beam CT (CBCT) could offer an important advance to thoracic surgeons in directly localizing subpalpable nodules during surgery. An image-guidance system is under development using mobile C-arm CBCT to directly localize tumors in the OR, potentially reducing the cost and logistical burden of conventional preoperative localization and facilitating safer surgery by visualizing critical structures surrounding the surgical target (e.g., pulmonary artery, airways, etc.). To utilize the wealth of preoperative image/planning data and to guide targeting under conditions in which the tumor may not be directly visualized, a deformable registration approach has been developed that geometrically resolves images of the inflated (i.e., inhale or exhale) and deflated states of the lung. This novel technique employs a coarse model-driven approach using lung surface and bronchial airways for fast registration, followed by an image-driven registration using a variant of the Demons algorithm to improve target localization to within ~1 mm. Two approaches to model-driven registration are presented and compared - the first involving point correspondences on the surface of the deflated and inflated lung and the second a mesh evolution approach. Intensity variations (i.e., higher image intensity in the deflated lung) due to expulsion of air from the lungs are accounted for using an a priori lung density modification, and its improvement on the performance of the intensity-driven Demons algorithm is demonstrated. Preliminary results of the combined model-driven and intensity-driven registration process demonstrate accuracy consistent with requirements in minimally invasive thoracic surgery in both target localization and critical structure avoidance.

This paper proposes to utilize a patient-specific prior to augment intraoperative sparse-scan data to accurately reconstruct the aspects of the region that have changed by a surgical procedure in image-guided surgeries. When anatomical changes are introduced by a surgical procedure, only a sparse set of x-ray images are acquired, and the prior volume is registered to these data. Since all the information of the patient anatomy except for the surgical change is already known from the prior volume, we highlight only the change by creating difference images between the new scan and digitally reconstructed radiographs (DRR) computed from the registered prior volume. The region of change (RoC) is reconstructed from these sparse difference images by a penalized likelihood (PL) reconstruction method regularized by a compressed sensing penalty. When the surgical changes are local and relatively small, the RoC reconstruction involves only a small volume size and a small number of projections, allowing much faster computation and lower radiation dose than is needed to reconstruct the entire surgical volume. The reconstructed RoC merges with the prior volume to visualize an updated surgical field. We apply this novel approach to sacroplasty phantom data obtained from a conebeam CT (CBCT) test bench and vertebroplasty data with a fresh cadaver acquired from a C-arm CBCT system with a flat-panel detector (FPD).

This paper presents a parallel algorithm for the top of the line among the fuzzy connectedness algorithm family, namely the iterative relative fuzzy connectedness (IRFC) segmentation method. The algorithm of IRFC, realized via image foresting transform (IFT), is implemented by using NVIDIA's compute unified device architecture (CUDA) platform for segmenting large medical image data sets. In the IRFC algorithm, there are two major computational tasks: (i) computing the fuzzy affinity relations, and (ii) computing the fuzzy connectedness relations and tracking labels for objects of interest. Both tasks are implemented as CUDA kernels, and a substantial improvement in speed for both tasks is achieved. Our experiments based on three data sets of small, medium, and large data size demonstrate the efficiency of the parallel algorithm, which achieves a speed-up factor of 2.4x, 17.0x, and 42.7x, correspondingly, for the three data sets on the NVIDIA Tesla C1060 over the implementation of the algorithm in CPU.

To make Quantitative Radiology a reality in routine radiological practice, computerized automatic anatomy recognition (AAR) during radiological image reading becomes essential. As part of this larger goal, last year at this conference we presented a fuzzy strategy for building body-wide group-wise anatomic models. In the present paper, we describe the further advances made in fuzzy modeling and the algorithms and results achieved for AAR by using the fuzzy models. The proposed AAR approach consists of three distinct steps: (a) Building fuzzy object models (FOMs) for each population group G. (b) By using the FOMs to recognize the individual objects in any given patient image I under group G. (c) To delineate the recognized objects in I. This paper will focus mostly on (b). FOMs are built hierarchically, the smaller sub-objects forming the offspring of larger parent objects. The hierarchical pose relationships from the parent to offspring are codified in the FOMs. Several approaches are being explored currently, grouped under two strategies, both being hierarchical: (ra1) those using search strategies; (ra2) those strategizing a one-shot approach by which the model pose is directly estimated without searching. Based on 32 patient CT data sets each from the thorax and abdomen and 25 objects modeled, our analysis indicates that objects do not all scale uniformly with patient size. Even the simplest among the (ra2) strategies of recognizing the root object and then placing all other descendants as per the learned parent-to-offspring pose relationship bring the models on an average within about 18 mm of the true locations.

Interventional C-arm systems allow the efficient acquisition of 3D cone beam CT images. They can be used for intervention planning, navigation, and outcome assessment. We present a fast and completely automated volume of interest (VOI) delineation for cardiac interventions, covering the whole visceral cavity including mediastinum and lungs but leaving out rib-cage and spine. The problem is addressed in a model based approach. The procedure has been evaluated on 22 patient cases and achieves an average surface error below 2mm. The method is able to cope with varying image intensities, varying truncations due to the limited reconstruction volume, and partially with heavy metal and motion artifacts.

Probe guides are surgical fixtures that are rigidly attached to bone anchors in order to place a probe at a target with high accuracy (RMS error < 1 mm). Applications include needle biopsy, the placement of electrodes for deep-brain stimulation (DBS), spine surgery, and cochlear implant surgery. Targeting is based on pre-operative images, but targeting errors can arise from three sources: (1) anchor localization error, (2) guide fabrication error, and (3) external forces and torques. A well-established theory exists for the statistical prediction of target registration error (TRE) when targeting is accomplished by means of tracked probes, but no such TRE theory is available for fixtured probe guides. This paper provides that theory and shows that all three error sources can be accommodated in a remarkably simple extension of existing theory. Both the guide and the bone with attached anchors are modeled as objects with rigid sections and elastic sections, the latter of which are described by stiffness matrices. By relating minimization of elastic energy for guide attachment to minimization of fiducial registration error for point registration, it is shown that the expression for targeting error for the guide is identical to that for weighted rigid point registration if the weighting matrices are properly derived from stiffness matrices and the covariance matrices for fiducial localization are augmented with offsets in the anchor positions. An example of the application of the theory is provided for ear surgery.

Electromagnetic tracking (EMT) systems are gaining increased attention in various fields of image-guided surgery. One of the main problems related to EMT systems is their vulnerability to distortion due to metallic objects. Several methods have been introduced to evaluate electromagnetic trackers, yet, the data acquisition has to be manually performed in a time consuming procedure, which often leads to a sparse volume coverage. The aim of this work is to present a fully automatic calibration system. It consists of a novel, parallel robotic arm and has the potential to collect a very large number of tracking data while scanning the entire tracking volume of a field generator. To prove the feasibility of our system, we evaluate two electromagnetic field generators (NDI Planar and Tabletop) in an ideal metal-free environment and in a clinical setup. Our proposed calibration robot successfully performed throughout the experiments and examined 1,000 positions in the tracking volume of each field generator (FG). According to the results both FGs are highly accurate in an ideal environment. However, in the examined clinical setup, the Planar FG is strongly distorted by metallic objects. Whereas the Tabletop FG provided very robust and accurate tracking, even if metallic objects where lying directly underneath the FG.

Conventional surgical tracking configurations carry a variety of limitations in line-of-sight, geometric accuracy, and mismatch with the surgeon's perspective (for video augmentation). With increasing utilization of mobile C-arms, particularly those allowing cone-beam CT (CBCT), there is opportunity to better integrate surgical trackers at bedside to address such limitations. This paper describes a tracker configuration in which the tracker is mounted directly on the Carm. To maintain registration within a dynamic coordinate system, a reference marker visible across the full C-arm rotation is implemented, and the "Tracker-on-C" configuration is shown to provide improved target registration error (TRE) over a conventional in-room setup - (0.9±0.4) mm vs (1.9±0.7) mm, respectively. The system also can generate digitally reconstructed radiographs (DRRs) from the perspective of a tracked tool ("x-ray flashlight"), the tracker, or the C-arm ("virtual fluoroscopy"), with geometric accuracy in virtual fluoroscopy of (0.4±0.2) mm. Using a video-based tracker, planning data and DRRs can be superimposed on the video scene from a natural perspective over the surgical field, with geometric accuracy (0.8±0.3) pixels for planning data overlay and (0.6±0.4) pixels for DRR overlay across all C-arm angles. The field-of-view of fluoroscopy or CBCT can also be overlaid on real-time video ("Virtual Field Light") to assist C-arm positioning. The fixed transformation between the x-ray image and tracker facilitated quick, accurate intraoperative registration. The workflow and precision associated with a variety of realistic surgical tasks were significantly improved using the Tracker-on-C - for example, nearly a factor of 2 reduction in time required for C-arm positioning, reduction or elimination of dose in "hunting" for a specific fluoroscopic view, and confident placement of the x-ray FOV on the surgical target. The proposed configuration streamlines the integration of C-arm CBCT with realtime tracking and demonstrated utility in a spectrum of image-guided interventions (e.g., spine surgery) benefiting from improved accuracy, enhanced visualization, and reduced radiation exposure.

Purpose: Accurate dose delivery in deep-inspiration breath-hold (DIBH) radiotherapy for patients with breast cancer relies on precise treatment setup and monitoring of the depth of the breath hold. This study entailed performance evaluation of a 3D surface imaging system for image guidance in DIBH radiotherapy by comparison with cone-beam computed tomography (CBCT). Materials and Methods: Fifteen patients, treated with DIBH radiotherapy after breast-conserving surgery, were included. The performance of surface imaging was compared to the use of CBCT for setup verification. Retrospectively, breast surface registrations were performed for CBCT to planning CT as well as for a 3D surface, captured concurrently with CBCT, to planning CT. The resulting setup errors were compared with linear regression analysis. For the differences between setup errors, group mean, systematic and random errors were calculated. Furthermore, a residual error after registration (RRE) was assessed for both systems by investigating the root-mean-square distance between the planning CT surface and registered CBCT/captured surface. Results: Good correlation between setup errors was found: R²=0.82, 0.86, 0.82 in left-right, cranio-caudal and anteriorposterior direction, respectively. Systematic and random errors were ≤0.16cm and ≤0.13cm in all directions, respectively. RRE values for surface imaging and CBCT were on average 0.18 versus 0.19cm with a standard deviation of 0.10 and 0.09cm, respectively. Wilcoxon-signed-ranks testing showed that CBCT registrations resulted in higher RRE values than surface imaging registrations (p=0.003). Conclusion: This performance evaluation study shows very promising results

In the real-time tumor-tracking radiotherapy (RTRT) system, the fiducial markers are inserted in or near the target tumor in order monitor the respiratory-induced motion of tumors. During radiation treatment, the markers are detected by continuous fluoroscopy operated at 30 frames/sec. The marker position is determined by means of a template pattern matching technique which is based on the normalized cross correlation. With high tube voltage, large current and long exposure, the fiducial marker will be recognized accurately, however, the radiation dose due to X-ray fluoroscopy increases. On the other hand, by decreasing the fluoroscopy parameter settings, the fiducial marker could be lost because the effect of statistical noise is increased. In the respiratory-gated radiotherapy, the error of the image guidance will induce the reduction of the irradiation efficiency and accuracy. In order to track the marker stably and accurately in low dose fluoroscopy, we propose the application of a recursive filter. The effectiveness of the image processing is investigated by tracking the static marker and the dynamic marker. The results suggest that the stability and the accuracy of the marker tracking can be improved by applying the recursive image filter in low dose imaging.

Although 4D CT imaging becomes available in an increasing number of radiotherapy facilities, 3D imaging and planning is still standard in current clinical practice. In particular for lung tumors, respiratory motion is a known source of uncertainty and should be accounted for during radiotherapy planning - which is difficult by using only a 3D planning CT. In this contribution, we propose applying a statistical lung motion model to predict patients' motion patterns and to estimate dosimetric motion effects in lung tumor radiotherapy if only 3D images are available. Being generated based on 4D CT images of patients with unimpaired lung motion, the model tends to overestimate lung tumor motion. It therefore promises conservative risk assessment regarding tumor dose coverage. This is exemplarily evaluated using treatment plans of lung tumor patients with different tumor motion patterns and for two treatment modalities (conventional 3D conformal radiotherapy and step-&- shoot intensity modulated radiotherapy). For the test cases, 4D CT images are available. Thus, also a standard registration-based 4D dose calculation is performed, which serves as reference to judge plausibility of the modelbased 4D dose calculation. It will be shown that, if combined with an additional simple patient-specific breathing surrogate measurement (here: spirometry), the model-based dose calculation provides reasonable risk assessment of respiratory motion effects.

Computer-Integrated Interventional Medicine (CIIM) promises to have a profound impact on health care in the next 20 years, much as and for many of the same reasons that the marriage of computers and information processing methods with other technology have had on manufacturing, transportation, and other sectors of our society. Our basic premise is that the steps of creating patient-specific computational models, using these models for planning, registering the models and plans with the actual patient in the operating room, and using this information with appropriate technology to assist in carrying out and monitoring the intervention are best viewed as part of a complete patient-specific intervention process that occurs over many time scales. Further, the information generated in computer-integrated interventions can be captured and analyzed statistically to improve treatment processes. This paper will explore these themes briefly, using examples drawn from our work at the Engineering Research Center for Computer-Integrated Surgical Systems and Technology (CISST ERC).

Objective: Robotic assistance during surgery has been shown to be a useful resource to both augment the surgical skills of the surgeon through tele-operation, and to assist the surgeon handling the surgical instruments to the surgeon, similar to a surgical tech. We evaluated the performance and effect of a gesture driven surgical robotic nurse in the context of economy of movements, during an abdominal incision and closure exercise with a simulator. Methods: A longitudinal midline incision (100 mm) was performed on the simulated abdominal wall to enter the peritoneal cavity without damaging the internal organs. The wound was then closed using a blunt needle ensuring that no tissue is caught up by the suture material. All the instruments required to complete this task were delivered by a robotic surgical manipulator directly to the surgeon. The instruments were requested through voice and gesture recognition. The robotic system used a low end range sensor camera to extract the hand poses and for recognizing the gestures. The instruments were delivered to the vicinity of the patient, at chest height and at a reachable distance to the surgeon. Task performance measures for each of three abdominal incision and closure exercises were measured and compared to a human scrub nurse instrument delivery action. Picking instrument position variance, completion time and trajectory of the hand were recorded for further analysis. Results: The variance of the position of the robotic tip when delivering the surgical instrument is compared to the same position when a human delivers the instrument. The variance was found to be 88.86% smaller compared to the human delivery group. The mean task completion time to complete the surgical exercise was 162.7± 10.1 secs for the human assistant and 191.6± 3.3 secs (P<.01) when using the robotic standard display group. Conclusion: Multimodal robotic scrub nurse assistant improves the surgical procedure by reducing the number of movements (lower variance in the picking position). The variance of the picking point is closely related to the concept of economy of movements in the operating room. Improving the effectiveness of the operating room can potentially enhance the safety of surgical interventions without affecting the performance time.

Objectives: The purpose of this study was to determine the effect of three-dimensional (3D) versus two-dimensional (2D) visualization on the amount of force applied to mitral valve tissue during robotics-assisted mitral valve annuloplasty, and the time to perform the procedure in an ex vivo animal model. In addition, we examined whether these effects are consistent between novices and experts in robotics-assisted cardiac surgery. Methods: A cardiac surgery test-bed was constructed to measure forces applied by the da Vinci surgical system (Intuitive Surgical, Sunnyvale, CA) during mitral valve annuloplasty. Both experts and novices completed roboticsassisted mitral valve annuloplasty with 2D and 3D visualization. Results: The mean time for both experts and novices to suture the mitral valve annulus and to tie sutures using 3D visualization was significantly less than that required to suture the mitral valve annulus and to tie sutures using 2D vision (p∠0.01). However, there was no significant difference in the maximum force applied by novices to the mitral valve during suturing (p = 0.3) and suture tying (p = 0.6) using either 2D or 3D visualization. Conclusion: This finding suggests that 3D visualization does not fully compensate for the absence of haptic feedback in robotics-assisted cardiac surgery. Keywords: Robotics-assisted surgery, visualization, cardiac surgery

This paper evaluates deformation accuracy of a virtual pneumoperitoneum method by utilizing measurement data of real deformations of patient bodies. Laparoscopic surgery is an option of surgical operations that is less invasive technique as compared with traditional surgical operations. In laparoscopic surgery, the pneumoperitoneum process is performed to create a viewing and working space. Although a virtual pneumoperitoneum method based on 3D CT image deformation has been proposed for patient-specific laparoscopy simulators, quantitative evaluation based on measurements obtained in real surgery has not been performed. In this paper, we evaluate deformation accuracy of the virtual pneumoperitoneum method based on real deformation data of the abdominal wall measured in operating rooms (ORs.) The evaluation results are used to find optimal deformation parameters of the virtual pneumoperitoneum method. We measure landmark positions on the abdominal wall on a 3D CT image taken before performing a pneumoperitoneum process. The landmark positions are defined based on anatomical structure of a patient body. We also measure the landmark positions on a 3D CT image deformed by the virtual pneumoperitoneum method. To measure real deformations of the abdominal wall, we measure the landmark positions on the abdominal wall of a patient before and after the pneumoperitoneum process in the OR. We transform the landmark positions measured in the OR from the tracker coordinate system to the CT coordinate system. A positional error of the virtual pneumoperitoneum method is calculated based on positional differences between the landmark positions on the 3D CT image and the transformed landmark positions. Experimental results based on eight cases of surgeries showed that the minimal positional error was 13.8 mm. The positional error can be decreased from the previous method by calculating optimal deformation parameters of the virtual pneumoperitoneum method from the experimental results.

Real-time surgical simulation is becoming an important component of surgical training. To meet the realtime requirement, however, the accuracy of the biomechancial modeling of soft tissue is often compromised due to computing resource constraints. Furthermore, haptic integration presents an additional challenge with its requirement for a high update rate. As a result, most real-time surgical simulation systems employ a linear elasticity model, simplified numerical methods such as the boundary element method or spring-particle systems, and coarse volumetric meshes. However, these systems are not clinically realistic. We present here an ongoing work aimed at developing an efficient and physically realistic neurosurgery simulator using a non-linear finite element method (FEM) with haptic interaction. Real-time finite element analysis is achieved by utilizing the total Lagrangian explicit dynamic (TLED) formulation and GPU acceleration of per-node and per-element operations. We employ a virtual coupling method for separating deformable body simulation and collision detection from haptic rendering, which needs to be updated at a much higher rate than the visual simulation. The system provides accurate biomechancial modeling of soft tissue while retaining a real-time performance with haptic interaction. However, our experiments showed that the stability of the simulator depends heavily on the material property of the tissue and the speed of colliding objects. Hence, additional efforts including dynamic relaxation are required to improve the stability of the system.

A biomechanical model is proposed to predict deflated lung tumor motion caused by diaphragm respiratory motion. This model can be very useful for targeting the tumor in tumor ablative procedures such as lung brachytherapy. To minimize motion within the target lung, these procedures are performed while the lung is deflated. However, significant amount of tissue deformation still occurs during respiration due to the diaphragm contact forces. In the absence of effective realtime image guidance, biomechanical models can be used to estimate tumor motion as a function of diaphragm's position. To develop this model, Finite Element Method (FEM) was employed. To demonstrate the concept, we conducted an animal study of an ex-vivo porcine deflated lung with a tumor phantom. The lung was deformed by compressing a diaphragm mimicking cylinder against it. Before compression, 3D-CT image of this lung was acquired, which was segmented and turned into FE mesh. The lung tissue was modeled as hyperelastic material with a contact loading to calculate the lung deformation and tumor motion during respiration. To validate the results from FE model, the motion of a small area on the surface close to the tumor was tracked while the lung was being loaded by the cylinder. Good agreement was demonstrated between the experiment results and simulation results. Furthermore, the impact of tissue hyperelastic parameters uncertainties in the FE model was investigated. For this purpose, we performed in-silico simulations with different hyperelastic parameters. This study demonstrated that the FEM was accurate and robust for tumor motion prediction.

This paper presents a method for simulating the behavior of stomach with large-scale deformation. This simulator is generated by the real-time FEM-based analysis by using a neural network.⁴ There are various deformation patterns of hollow organs by changing both its shape and volume. In this case, one network can not learn the stomach deformation with a huge number of its deformation pattern. To overcome the problem, we propose a method of constructing the simulator composed of multiple neural networks by 1)partitioning a training dataset into several subsets, and 2)selecting the data included in each subset. From our experimental results, we can conclude that our method can speed up the training process of a neural network while keeping acceptable accuracy.

Pectus excavatum is the most common congenital deformity of the anterior chest wall, in which an abnormal formation of the rib cage gives the chest a caved-in or sunken appearance. Today, the surgical correction of this deformity is carried out in children and adults through Nuss technic, which consists in the placement of a prosthetic bar under the sternum and over the ribs. Although this technique has been shown to be safe and reliable, not all patients have achieved adequate cosmetic outcome. This often leads to psychological problems and social stress, before and after the surgical correction. This paper targets this particular problem by presenting a method to predict the patient surgical outcome based on pre-surgical imagiologic information and chest skin dynamic modulation. The proposed approach uses the patient pre-surgical thoracic CT scan and anatomical-surgical references to perform a 3D segmentation of the left ribs, right ribs, sternum and skin. The technique encompasses three steps: a) approximation of the cartilages, between the ribs and the sternum, trough b-spline interpolation; b) a volumetric mass spring model that connects two layers - inner skin layer based on the outer pleura contour and the outer surface skin; and c) displacement of the sternum according to the prosthetic bar position. A dynamic model of the skin around the chest wall region was generated, capable of simulating the effect of the movement of the prosthetic bar along the sternum. The results were compared and validated with patient postsurgical skin surface acquired with Polhemus FastSCAN system.

Minimally invasive surgery is medically complex and can heavily benefit from computer assistance. One way to help the surgeon is to integrate preoperative planning data into the surgical workflow. This information can be represented as a customized preoperative model of the surgical site. To use it intraoperatively, it has to be updated during the intervention due to the constantly changing environment. Hence, intraoperative sensor data has to be acquired and registered with the preoperative model. Haptic information which could complement the visual sensor data is still not established. In addition, biomechanical modeling of the surgical site can help in reflecting the changes which cannot be captured by intraoperative sensors. We present a setting where a force sensor is integrated into a laparoscopic instrument. In a test scenario using a silicone liver phantom, we register the measured forces with a reconstructed surface model from stereo endoscopic images and a finite element model. The endoscope, the instrument and the liver phantom are tracked with a Polaris optical tracking system. By fusing this information, we can transfer the deformation onto the finite element model. The purpose of this setting is to demonstrate the principles needed and the methods developed for intraoperative sensor data fusion. One emphasis lies on the calibration of the force sensor with the instrument and first experiments with soft tissue. We also present our solution and first results concerning the integration of the force sensor as well as accuracy to the fusion of force measurements, surface reconstruction and biomechanical modeling.

The pigtail catheter is a type of catheter inserted into the human body during interventional surgeries such as the transcatheter aortic valve implantation (TAVI). The catheter is characterized by a tightly curled end in order to remain attached to a valve pocket during the intervention, and it is used to inject contrast agent for the visualization of the vessel in fluoroscopy. Image-based detection of this catheter is used during TAVI, in order to overlay a model of the aorta and enhance visibility during the surgery. Due to the different possible projection angles in fluoroscopy, the pigtail tip can appear in a variety of different shapes spanning from pure circular to ellipsoid or even line. Furthermore, the appearance of the catheter tip is radically altered when the contrast agent is injected during the intervention or when it is occluded by other devices. All these factors make the robust real-time detection and tracking of the pigtail catheter a challenging task. To address these challenges, this paper proposes a new tree-structured, hierarchical detection scheme, based on a shape categorization of the pigtail catheter tip, and a combination of novel Haar features. The proposed framework demonstrates improved detection performance, through a validation on a data set consisting of 272 sequences with more than 20,000 images. The detection framework presented in this paper is not limited to pigtail catheter detection, but it can also be applied successfully to any other shape-varying object with similar characteristics.

Localization of target vertebrae is an essential step in minimally invasive spine surgery, with conventional methods relying on "level counting" - i.e., manual counting of vertebrae under fluoroscopy starting from readily identifiable anatomy (e.g., the sacrum). The approach requires an undesirable level of radiation, time, and is prone to counting errors due to the similar appearance of vertebrae in projection images; wrong-level surgery occurs in 1 of every ~3000 cases. This paper proposes a method to automatically localize target vertebrae in x-ray projections using 3D-2D registration between preoperative CT (in which vertebrae are preoperatively labeled) and intraoperative fluoroscopy. The registration uses an intensity-based approach with a gradient-based similarity metric and the CMA-ES algorithm for optimization. Digitally reconstructed radiographs (DRRs) and a robust similarity metric are computed on GPU to accelerate the process. Evaluation in clinical CT data included 5,000 PA and LAT projections randomly perturbed to simulate human variability in setup of mobile intraoperative C-arm. The method demonstrated 100% success for PA view (projection error: 0.42mm) and 99.8% success for LAT view (projection error: 0.37mm). Initial implementation on GPU provided automatic target localization within about 3 sec, with further improvement underway via multi-GPU. The ability to automatically label vertebrae in fluoroscopy promises to streamline surgical workflow, improve patient safety, and reduce wrong-site surgeries, especially in large patients for whom manual methods are time consuming and error prone.

Prostate biopsy is the clinical standard for prostate cancer diagnosis. To improve the accuracy of targeting suspicious locations, systems have been developed that can plan and record biopsy locations in a 3D TRUS image acquired at the beginning of the procedure. Some systems are designed for maximum compatibility with existing ultrasound equipment and are thus designed around the use of a conventional 2D TRUS probe, using controlled axial rotation of this probe to acquire a 3D TRUS reference image at the start of the biopsy procedure. Prostate motion during the biopsy procedure causes misalignments between the prostate in the live 2D TRUS images and the pre-acquired 3D TRUS image. We present an image-based rigid registration technique that aligns live 2D TRUS images, acquired immediately prior to biopsy needle insertion, with the pre-acquired 3D TRUS image to compensate for this motion. Our method was validated using 33 manually identified intrinsic fiducials in eight subjects and the target registration error was found to be 1.89 mm. We analysed the suitability of two image similarity metrics (normalized cross correlation and mutual information) for this task by plotting these metrics as a function of varying parameters in the six degree-of-freedom transformation space, with the ground truth plane obtained from registration as the starting point for the parameter exploration. We observed a generally convex behaviour of the similarity metrics. This encourages their use for this registration problem, and could assist in the design of a tool for the detection of misalignment, which could trigger the execution of a non-real-time registration, when needed during the procedure.

The Camera-Augmented Mobile C-arm (CamC) augments X-ray by optical camera images and is used as an advanced visualization and guidance tool in trauma and orthopedic surgery. However, in its current form the calibration is suboptimal. We investigated and compared calibration and distortion correction between: (i) the existing CamC calibration framework (ii) Zhang's calibration for video images, and (iii) the traditional C-arm fluoroscopy calibration technique. Accuracy of the distortion correction for each of the three methods is compared by analyzing the error based on a synthetic model and the linearity and cross-ratio properties. Also, the accuracy of calibrated X-ray projection geometry is evaluated by performing C-arm pose estimation using a planar pattern with known geometry. The RMS errors based on a synthetic model and pose estimation shows that the traditional C-arm method (μ=0.39 pixels) outperforms both Zhang (μ=0.68 pixels) and original CamC (μ=1.07 pixels) methods. The relative pose estimation comparison shows that the translation error of the traditional method (μ=0.25mm) outperforms Zhang (μ=0.41mm) and CamC (μ=1.13mm) method. In conclusion, we demonstrated that the traditional X-ray calibration procedure outperforms the existing CamC solution and Zhang's method for the calibration of C-arm X-ray projection geometry.

Minimally invasive abdominal aortic aneurysm (AAA) stenting can be greatly facilitated by overlaying the preoperative 3-D model of the abdominal aorta onto the intra-operative 2-D X-ray images. Accurate 2-D/3-D registration in 3-D space makes the 2-D/3-D overlay robust to the change of C-Arm angulations. By far, the 2-D/3-D registration methods based on simulated X-ray projection images using multiple image planes have been shown to be able to provide satisfactory 3-D registration accuracy. However, one drawback of the intensity-based 2-D/3-D registration methods is that the similarity measure is usually highly non-convex and hence the optimizer can easily be trapped into local minima. User interaction therefore is often needed in the initialization of the position of the 3-D model in order to get a successful 2-D/3-D registration. In this paper, a novel 3-D pose initialization technique is proposed, as an extension of our previously proposed bi-plane 2-D/3-D registration method for AAA intervention [4]. The proposed method detects vessel bifurcation points and spine centerline in both 2-D and 3-D images, and utilizes landmark information to bring the 3-D volume into a 15mm capture range. The proposed landmark detection method was validated on real dataset, and is shown to be able to provide a good initialization for 2-D/3-D registration in [4], thus making the workflow fully automatic.

The objective of our work is a system that enables both mechanically and electronically shapable thermal energy deposition in soft tissue ablation. The overall goal is a system that can percutaneously (and through a single organ surface puncture) treat tumors that are large, multiple, geometrically complex, or located too close to vital structures for traditional resection. This paper focuses on mechanical steering and image guidance aspects of the project. Mechanical steering is accomplished using an active cannula that enables repositioning of the ablator tip without complete retraction. We describe experiments designed to evaluate targeting accuracy of the active cannula (also known as a concentric tube robot) in soft tissues under tracked 3D ultrasound guidance.

There are approximately 6000 hospitals in the United States, of which approximately 5400 employ minimally invasive surgical robots for a variety of procedures. Furthermore, 95% of these robots require extensive registration before they can be fitted into the operating room. These "registrations" are performed by surgical navigation systems, which allow the surgical tools, the robot and the surgeon to be synchronized together-hence operating in concert. The most common surgical navigation modalities include: electromagnetic (EM) tracking and optical tracking. Currently, these navigation systems are large, intrusive, come with a steep learning curve, require sacrifices on the part of the attending medical staff, and are quite expensive (since they require several components). Recently, photoacoustic (PA) imaging has become a practical and promising new medical imaging technology. PA imaging only requires the minimal equipment standard with most modern ultrasound (US) imaging systems as well as a common laser source. In this paper, we demonstrate that given a PA imaging system, as well as a stereocamera (SC), the registration between the US image of a particular anatomy and the SC image of the same anatomy can be obtained with reliable accuracy. In our experiments, we collected data for N = 80 trials of sample 3D US and SC coordinates. We then computed the registration between the SC and the US coordinates. Upon validation, the mean error and standard deviation between the predicted sample coordinates and the corresponding ground truth coordinates were found to be 3.33 mm and 2.20 mm respectively.

Visualization of anatomical data for disease diagnosis, surgical planning, or orientation during interventional therapy is an integral part of modern health care. However, as anatomical information is typically shown on monitors provided by a radiological work station, the physician has to mentally transfer internal structures shown on the screen to the patient. To address this issue, we recently presented a new approach to on-patient visualization of 3D medical images, which combines the concept of augmented reality (AR) with an intuitive interaction scheme. Our method requires mounting a range imaging device, such as a Time-of-Flight (ToF) camera, to a portable display (e.g. a tablet PC). During the visualization process, the pose of the camera and thus the viewing direction of the user is continuously determined with a surface matching algorithm. By moving the device along the body of the patient, the physician is given the impression of looking directly into the human body. In this paper, we present and evaluate a new method for camera pose estimation based on an anisotropic trimmed variant of the well-known iterative closest point (ICP) algorithm. According to in-silico and in-vivo experiments performed with computed tomography (CT) and ToF data of human faces, knees and abdomens, our new method is better suited for surface registration with ToF data than the established trimmed variant of the ICP, reducing the target registration error (TRE) by more than 60%. The TRE obtained (approx. 4-5 mm) is promising for AR visualization, but clinical applications require maximization of robustness and run-time.

This work explores the suitability of low-cost sensors for "serious" medical applications, such as tracking of interventional tools in the OR, for simulation, and for education. Although such tracking - i.e. the acquisition of pose data e.g. for ultrasound probes, tissue manipulation tools, needles, but also tissue, bone etc. - is well established, it relies mostly on external devices such as optical or electromagnetic trackers, both of which mandate the use of special markers or sensors attached to each single entity whose pose is to be recorded, and also require their calibration to the tracked entity, i.e. the determination of the geometric relationship between the marker's and the object's intrinsic coordinate frames. The Microsoft Kinect sensor is a recently introduced device for full-body tracking in the gaming market, but it was quickly hacked - due to its wide range of tightly integrated sensors (RGB camera, IR depth and greyscale camera, microphones, accelerometers, and basic actuation) - and used beyond this area. As its field of view and its accuracy are within reasonable usability limits, we describe a medical needle-tracking system for interventional applications based on the Kinect sensor, standard biopsy needles, and no necessary attachments, thus saving both cost and time. Its twin cameras are used as a stereo pair to detect needle-shaped objects, reconstruct their pose in four degrees of freedom, and provide information about the most likely candidate.

Needles provide an effective way to reach lesions in soft tissue and are frequently used for diagnosis and treatment. Examples include biopsies, tumor ablation, and brachytherapy. Yet, precise localization of the needle with respect to the target is complicated by motion and deformation of the tissue during insertion. We have developed a prototypical needle with an embedded optical fiber allowing to obtain optical coherence tomography images of the tissue in front of the needle tip. Using the data and particularly the Doppler information it is possible to estimate the motion of the needle tip with respect to the surrounding soft tissue. We studied whether it is feasible to approximate the depth in tissue by integrating over the relative velocity. To validate the approach, the needle was driven into tissue phantoms using an articulated robotic arm. The time when the needle entered and left the phantom was observed with optical cameras, and the total motion of the robot was compared with the values computed from the Doppler OCTmeasurements. Our preliminary results indicate that the Doppler data can provide additional information on the needle position inside soft tissue. It could be used in addition to other image data to improve precise needle navigation, particularly when other image modalities are subject to artifacts caused by the needles.

PURPOSE: Evaluation of surgical performance in image-guided needle insertions is of emerging interest, to both promote patient safety and improve the efficiency and effectiveness of training. The purpose of this study was to determine if a Markov model-based algorithm can more accurately segment a needle-based surgical procedure into its five constituent tasks than a simple threshold-based algorithm. METHODS: Simulated needle trajectories were generated with known ground truth segmentation by a synthetic procedural data generator, with random noise added to each degree of freedom of motion. The respective learning algorithms were trained, and then tested on different procedures to determine task segmentation accuracy. In the threshold-based algorithm, a change in tasks was detected when the needle crossed a position/velocity threshold. In the Markov model-based algorithm, task segmentation was performed by identifying the sequence of Markov models most likely to have produced the series of observations. RESULTS: For amplitudes of translational noise greater than 0.01mm, the Markov model-based algorithm was significantly more accurate in task segmentation than the threshold-based algorithm (82.3% vs. 49.9%, p<0.001 for amplitude 10.0mm). For amplitudes less than 0.01mm, the two algorithms produced insignificantly different results. CONCLUSION: Task segmentation of simulated needle insertion procedures was improved by using a Markov model-based algorithm as opposed to a threshold-based algorithm for procedures involving translational noise.

During endoscopic procedures it is often desirable to determine the distance between anatomical features. One such clinical application is percutaneous cochlear implantation (PCI), which is a minimally invasive approach to the cochlea via a single, straight drill path and can be achieved accurately using bone-implanted markers and customized microstereotactic frame. During clinical studies to validate PCI, traditional open-field cochlear implant surgery was performed and prior to completion of the surgery, a customized microstereotactic frame designed to achieve the desired PCI trajectory was attached to the bone-implanted markers. To determine whether this trajectory would have safely achieved the target, a sham drill bit is passed through the frame to ensure that the drill bit would reach the cochlea without damaging vital structures. Because of limited access within the facial recess, the distances from the bit to anatomical features could not be measured with calipers. We hypothesized that an endoscope mounted on a sliding stage that translates only along the trajectory, would provide sufficient triangulation to accurately measure these distances. In this paper, the design, fabrication, and testing of such a system is described. The endoscope is mounted so that its optical axis is approximately aligned with the trajectory. Several images are acquired as the stage is moved, and threedimensional reconstruction of selected points allows determination of distances. This concept also has applicability in a large variety of rigid endoscopic interventions including bronchoscopy, laparoscopy, and sinus endoscopy.

Motion estimation based on point correspondences in two views is a classic problem in computer vision. In the field of laparoscopic video sequences - even with state of the art algorithms - a stable motion estimation can not be guaranteed generally. Typically, a video from a laparoscopic surgery contains sequences where the surgeon barely moves the endoscope. Such restricted movement causes a small ratio between baseline and distance leading to unstable estimation results. Exploiting the fact that the entire sequence is known a priori, we propose an algorithm for keyframe selection in a sequence of images. The key idea can be expressed as follows: if all combination of frames in a sequence are scored, the optimal solution can be described as a weighted directed graph problem. We adapt the widely known Dijkstras Algorithm to find the best selection of frames.¹ The framework for keyframe selection can be used universally to find the best combination of frames for any reliable scoring function. For instance, forward motion ensures the most accurate camera position estimation, whereas sideward motion is preferred in the sense of reconstruction. Based on the distribution and the disparity of point correspondences, we propose a scoring function which is capable of detecting poorly conditioned pairs of frames. We demonstrate the potential of the algorithm focusing on accurate camera positions. A robot system provides ground truth data. The environment in laparoscopic videos is reflected by an industrial endoscope and a phantom.

User interaction during navigated surgery is often a critical issue in the overall procedure, as several complex aspects must be considered, such as sterility, workflow, field of view, and cognitive load. This work introduces a new approach for intraoperative interaction that seamlessly fits the high surgical requirements. A navigation system, typically consisting of a tracking system and a monitor for 3D virtual models, is augmented with a tracked pointer with triggering functionality and a pan-tilt mounted laser. The pointer, which is sterile and can be applied for landmark-based organ registration, is used for wireless interaction with the monitor scene. The laser system enables the calibration of the monitor, which is out of the tracking system's range. Moreover, the laser beam can focus on any organ point defined on the virtual model, which improves targeting or visual feedback during intervention. The calibration of the laser system, monitor, and triggered pointer is achieved by an effective procedure, which can be easily repeated in operating room. The mathematical background of the calibration is based on the Levenberg-Marquardt and Umeyama's algorithms.

In an effort to improve the accuracy of transrectal ultrasound (TRUS)-guided needle biopsies of the prostate, it is important to understand the non-rigid deformation of the prostate. To understand the deformation of the prostate when an endorectal coil (ERC) is inserted, we develop an elastic registration framework to register prostate MR images with and without ERC. Our registration framework uses robust point matching (RPM) to get the correspondence between the surface landmarks in the source and target volumes followed by elastic body spline (EBS) registration based on the corresponding landmark pairs. Together with the manual rigid alignment, we compared our registration framework based on pure surface landmarks to the registration based on both surface and internal landmarks in the center of the prostate. In addition, we assessed the impact of constraining the warping in the central zone of the prostate using a Gaussian weighting function. Our results show that elastic surface-driven prostate registration is feasible, and that internal landmarks further improve the registration in the central zone while they have little impact on the registration in the peripheral zone of the prostate. Results varied case by case depending on the accuracy of the prostate segmentation and the amount of warping present in each image pair. The most accurate results were obtained when using a Gaussian weighting in the central zone to limit the EBS warping driven by surface points. This suggests that a Gaussian constrain of the warping can effectively compensate for the limitations of the isotropic EBS deformation model, and for erroneous warping inside the prostate created by inaccurate surface landmarks driving the EBS.

Motivation: In prostate brachytherapy, intra-operative dosimetry would be ideal to allow for rapid evaluation of the implant quality while the patient is still in the treatment position. Such a mechanism, however, requires 3-D visualization of the currently deposited seeds relative to the prostate. Thus, accurate, robust, and fully-automatic seed segmentation is of critical importance in achieving intra-operative dosimetry. Methodology: Implanted brachytherapy seeds are segmented by utilizing a region-based implicit active contour approach. Overlapping seed clusters are then resolved using a simple yet effective declustering technique. Results: Ground-truth seed coordinates were obtained via a published segmentation technique. A total of 248 clinical C-arm images from 16 patients were used to validate the proposed algorithm resulting in a 98.4% automatic detection rate with a corresponding 2.5% false-positive rate. The overall mean centroid error between the ground-truth and automatic segmentations was measured to be 0.42 pixels, while the mean centroid error for overlapping seed clusters alone was measured to be 0.67 pixels. Conclusion: Based on clinical data evaluation and validation, robust, accurate, and fully-automatic brachytherapy seed segmentation can be achieved through the implicit active contour framework and subsequent seed declustering method.

The fusion of TransRectal Ultrasound (TRUS) and Magnetic Resonance (MR) images of the prostate can aid diagnosis and treatment planning for prostate cancer. Surface segmentations of the prostate are available in both modalities. Our goal is to develop a 3D deformable registration method based on these segmentations and a biomechanical model. The segmented source volume is meshed and a linear finite element model is created for it. This volume is deformed to the target image volume by applying surface forces computed by assuming a negative relative pressure between the non-overlapping regions of the volumes and the overlapping ones. This pressure drives the model to increase the volume overlap until the surfaces are aligned. We tested our algorithm on prostate surfaces extracted from post-operative MR and TRUS images for 14 patients, using a model with elasticity parameters in the range reported in the literature for the prostate. We used three evaluation metrics for validating our technique: the Dice Similarity Coefficient (DSC) (ideally equal to 1.0), which is a measure of volume alignment, the volume change in source surface during registration, which is a measure of volume preservation, and the distance between the urethras to assess the anatomical correctness of the method. We obtained a DSC of 0.96±0.02 and a mean distance between the urethras of 1.5±1.4 mm. The change in the volume of the source surface was 1.5±1.4%. Our results show that this method is a promising tool for physicallybased deformable surface registration.

Systematic transrectal ultrasound (TRUS)-guided biopsy is the standard method for a definitive diagnosis of prostate cancer. However, this biopsy approach uses two-dimensional (2D) ultrasound images to guide biopsy and can miss up to 30% of prostate cancers. We are developing a molecular image-directed, three-dimensional (3D) ultrasound imageguided biopsy system for improved detection of prostate cancer. The system consists of a 3D mechanical localization system and software workstation for image segmentation, registration, and biopsy planning. In order to plan biopsy in a 3D prostate, we developed an automatic segmentation method based wavelet transform. In order to incorporate PET/CT images into ultrasound-guided biopsy, we developed image registration methods to fuse TRUS and PET/CT images. The segmentation method was tested in ten patients with a DICE overlap ratio of 92.4% ± 1.1 %. The registration method has been tested in phantoms. The biopsy system was tested in prostate phantoms and 3D ultrasound images were acquired from two human patients. We are integrating the system for PET/CT directed, 3D ultrasound-guided, targeted biopsy in human patients.

Image guided prostate interventions have been accelerated by Magnetic Resonance Imaging (MRI) and robotic technologies in the past few years. However, transrectal ultrasound (TRUS) guided procedure still remains as vast majority in clinical practice due to engineering and clinical complexity of the MRI-guided robotic interventions. Subsequently, great advantages and increasing availability of MRI have not been utilized at its maximum capacity in clinic. To benefit patients from the advantages of MRI, we developed an MRI-compatible motorized needle guide device "Smart Template" that resembles a conventional prostate template to perform MRI-guided prostate interventions with minimal changes in the clinical procedure. The requirements and specifications of the Smart Template were identified from our latest MRI-guided intervention system that has been clinically used in manual mode for prostate biopsy. Smart Template consists of vertical and horizontal crossbars that are driven by two ultrasonic motors via timing-belt and mitergear transmissions. Navigation software that controls the crossbar position to provide needle insertion positions was also developed. The software can be operated independently or interactively with an open-source navigation software, 3D Slicer, that has been developed for prostate intervention. As preliminary evaluation, MRI distortion and SNR test were conducted. Significant MRI distortion was found close to the threaded brass alloy components of the template. However, the affected volume was limited outside the clinical region of interest. SNR values over routine MRI scan sequences for prostate biopsy indicated insignificant image degradation during the presence of the robotic system and actuation of the ultrasonic motors.

Compared to conventional open-heart surgeries, minimally invasive cardiac interventions cause less trauma and sideeffects to patients. However, the direct view of surgical targets and tools is usually not available in minimally invasive procedures, which makes image-guided navigation systems essential. The choice of imaging modalities used in the navigation systems must consider the capability of imaging soft tissues, spatial and temporal resolution, compatibility and flexibility in the OR, and financial cost. In this paper, we propose a new means of guidance for minimally invasive cardiac interventions using 3D real-time ultrasound images to show the intra-operative heart motion together with preoperative CT image(s) employed to demonstrate high-quality 3D anatomical context. We also develop a method to register intra-operative ultrasound and pre-operative CT images in close to real-time. The registration method has two stages. In the first, anatomical features are segmented from the first frame of ultrasound images and the CT image(s). A feature based registration is used to align those features. The result of this is used as an initialization in the second stage, in which a mutual information based registration is used to register every ultrasound frame to the CT image(s). A GPU based implementation is used to accelerate the registration.

Catheter ablation is an important option to treat ventricular tachycardias (VT). Scar-related VT is among the most difficult to treat, because myocardial scar, which is the underlying arrhythmogenic substrate, is patient-specific and often highly complex. The scar image from preprocedural late gadolinium enhancement magnetic resonance imaging (LGE- MRI) can provide high-resolution substrate information and, if integrated at the early stage of the procedure, can largely facilitate the procedure with image guidance. In clinical practice, however, early MRI integration is difficult because available integration tools rely on matching the MRI surface mesh and electroanatomical mapping (EAM) points, which is only possible after extensive EAM has been performed. In this paper, we propose to use a priori information on patient posture and a multi-sequence MRI integration framework to achieve accurate MRI integration that can be accomplished at an early stage of the procedure. From the MRI sequences, the left ventricular (LV) geometry, myocardial scar characteristics, and an anatomical landmark indicating the origin of the left main coronary artery are obtained preprocedurally using image processing techniques. Thereby the integration can be realized at the beginning of the procedure after acquiring a single mapping point. The integration method has been evaluated postprocedurally in terms of LV shape match and actual scar match. Compared to the iterative closest point (ICP) method that uses high-intensity mapping (225±49 points), our method using one mapping point reached a mean point-to-surface distance of 5.09±1.09 mm (vs. 3.85±0.60 mm, p<0.05), and scar correlation of -0.51±0.14 (vs. -0.50±0.14, p=NS).

One of the fundamental components in all Image Guided Surgery (IGS) applications is a method for presenting information to the surgeon in a simple, effective manner. This paper describes the first steps in our new Augmented Reality (AR) information delivery program. The system makes use of new "off the shelf" AR glasses that are both light-weight and unobtrusive, with adequate resolution for many IGS applications. Our first application is perioperative planning of minimally invasive robot-assisted cardiac surgery. In this procedure, a combination of tracking technologies and intraoperative ultrasound is used to map the migration of cardiac targets prior to selection of port locations for trocars that enter the chest. The AR glasses will then be used to present this heart migration data to the surgeon, overlaid onto the patients chest. The current paper describes the calibration process for the AR glasses, their integration into our IGS framework for minimally invasive robotic cardiac surgery, and preliminary validation of the system. Validation results indicate a mean 3D triangulation error of 2.9 ± 3.3mm, 2D projection error of 2.1 ± 2.1 pixels, and Normalized Stereo Calibration Error of 3.3.

Navigation and deployment of the prosthetic valve during trans-catheter aortic valve implantation (TAVI) can be greatly facilitated with 3-D models showing detailed anatomical structures. Fast and robust automatic contrast detection at the aortic root on X-ray images is indispensable for automatically triggering a 2-D/3-D registration to align the 3-D model. Previously, we have proposed an automatic method for contrast detection at the aortic root on fluoroscopic and angiographic sequences [4]. In this paper, we extend that algorithm in several ways, making it more robust to handle more general and difficult cases. Specifically, the histogram likelihood ratio test is multiplied with the histogram portion computation to handle faint contrast cases. Histogram mapping corrects sudden changes in the global brightness, thus avoiding potential false positives. Respiration and heart beating check further reduces the false positive rate. In addition, a probe mask is introduced to enhance the contrast feature curve when the dark ultrasound probe partially occludes the aortic root. Lastly, a semi-global registration method for aligning the aorta shape model is implemented to improve the robustness of the algorithm with respect to the selection of region of interest (ROI) containing the aorta. The extended algorithm was evaluated on 100 sequences, and improved the detection accuracy from 94% to 100%, compared to the original method. Also, the robustness of the extended algorithm was tested with 20 different shifts of the ROI, and the error rate was as low as 0.2%, in comparison to 6.6% for the original method.

Free moving bodies in the heart pose a serious health risk as they may be released in the arteries causing blood flow disruption. These bodies may be the result of various medical conditions and trauma. The conventional approach to removing these objects involves open surgery with sternotomy, the use of cardiopulmonary bypass, and a wide resection of the heart muscle. We advocate a minimally invasive surgical approach using a flexible robotic end effector guided by 3D transesophageal echocardiography. In a phantom study, we track a moving body in a beating heart using a modified normalized cross-correlation method, with mean RMS errors of 2.3 mm. We previously found the foreign body motion to be fast and abrupt, rendering infeasible a retrieval method based on direct tracking. We proposed a strategy based on guiding a robot to the most spatially probable location of the fragment and securing it upon its reentry to said location. To improve efficacy in the context of a robotic retrieval system, we extend this approach by exploring multiple candidate capture locations. Salient locations are identified based on spatial probability, dwell time, and visit frequency; secondary locations are also examined. Aggregate results indicate that the location of highest spatial probability (50% occupancy) is distinct from the longest-dwelled location (0.84 seconds). Such metrics are vital in informing the design of a retrieval system and capture strategies, and they can be computed intraoperatively to select the best capture location based on constraints such as workspace, time, and device manipulability. Given the complex nature of fragment motion, the ability to analyze multiple capture locations is a desirable capability in an interventional system.

During interventional procedures, 3D imaging modalities like CT and MRI are not commonly used due to interference with the surgery and radiation exposure concerns. Therefore, real-time information is usually limited and building models of cardiac motion are difficult. In such case, vessel motion modeling based on 2-D angiography images become indispensable. Due to issues with existing vessel segmentation algorithms and the lack of contrast in occluded vessels, manual segmentation of certain branches is usually necessary. In addition, such occluded branches are the most important vessels during coronary interventions and obtaining motion models for these can greatly help in reducing the procedure time and radiation exposure. Segmenting different cardiac phases independently does not guarantee temporal consistency and is not efficient for occluded branches required manual segmentation. In this paper, we propose a coronary motion modeling system which extracts the coronary tree for every cardiac phase, maintaining the segmentation by tracking the coronary tree during the cardiac cycle. It is able to map every frame to the specific cardiac phase, thereby inferring the shape information of the coronary arteries using the model corresponding to its phase. Our experiments show that our motion modeling system can achieve promising results with real-time performance.

Otologic surgery is performed for a variety of reasons including treatment of recurrent ear infections, alleviation of dizziness, and restoration of hearing loss. A typical ear surgery consists of a tympanomastoidectomy in which both the middle ear is explored via a tympanic membrane flap and the bone behind the ear is removed via mastoidectomy to treat disease and/or provide additional access. The mastoid dissection is performed using a high-speed drill to excavate bone based on a pre-operative CT scan. Intraoperatively, the surface of the mastoid component of the temporal bone provides visual feedback allowing the surgeon to guide their dissection. Dissection begins in "safe areas" which, based on surface topography, are believed to be correlated with greatest distance from surface to vital anatomy thus decreasing the chance of injury to the brain, large blood vessels (e.g. the internal jugular vein and internal carotid artery), the inner ear, and the facial nerve. "Safe areas" have been identified based on surgical experience with no identifiable studies showing correlation of the surface with subsurface anatomy. The purpose of our study was to investigate whether such a correlation exists. Through a three-step registration process, we defined a correspondence between each of twenty five clinically-applicable temporal bone CT scans of patients and an atlas and explored displacement and angular differences of surface topography and depth of critical structures from the surface of the skull. The results of this study reflect current knowledge of osteogenesis and anatomy. Based on two features (distance and angular difference), two regions (suprahelical and posterior) of the temporal bone show the least variability between surface and subsurface anatomy.

Intraoperative brain deformation can significantly degrade the accuracy of image guidance using preoperative MR images (pMR). To compensate for brain deformation, biomechanical models have been used to assimilate intraoperative displacement data, compute whole-brain deformation field, and to produce updated MR images (uMR). Stereovision (SV) is an important technique to capture both geometry and texture information of exposed cortical surface at the craniotomy, from which surface displacement data (known as sparse data) can be extracted by registering with pMR to drive the computational model. Approaches that solely utilize geometrical information (e.g., closest point distance (CPD) and iterative closest point (ICP) method) do not seem to capture surface deformation accurately especially when significant lateral shift occurs. In this study, we have developed a texture intensity-based method to register cortical surface reconstructed from stereovision after dural opening with pMR to extract 3D sparse data. First, a texture map is created from pMR using surface geometry before dural opening. Second, a mutual information (MI)-based registration was performed between the texture map and the corresponding stereo image after dural opening to capture the global lateral shift. A block-matching algorithm was then executed to differentiate local displacements in smaller patches. The global and local displacements were finally combined and transformed in 3D following stereopsis. We demonstrate the application of the proposed method with a clinical patient case, and show that the accuracy of the technique is 1-2 mm in terms of model-data misfit with a computation time <10 min.

In deep brain stimulation surgeries, stimulating electrodes are placed at specific targets in the deep brain to treat neurological disorders. Reaching these targets safely requires avoiding critical structures in the brain. Meticulous planning is required to find a safe path from the cortical surface to the intended target. Choosing a trajectory automatically is difficult because there is little consensus among neurosurgeons on what is optimal. Our goals are to design a path planning system that is able to learn the preferences of individual surgeons and, eventually, to standardize the surgical approach using this learned information. In this work, we take the first step towards these goals, which is to develop a trajectory planning approach that is able to effectively mimic individual surgeons and is designed such that parameters, which potentially can be automatically learned, are used to describe an individual surgeon's preferences. To validate the approach, two neurosurgeons were asked to choose between their manual and a computed trajectory, blinded to their identity. The results of this experiment showed that the neurosurgeons preferred the computed trajectory over their own in 10 out of 40 cases. The computed trajectory was judged to be equivalent to the manual one or otherwise acceptable in 27 of the remaining cases. These results demonstrate the potential clinical utility of computer-assisted path planning.

Percutaneous cochlear implantation (PCI) is a minimally invasive image-guided cochlear implant approach, where access to the cochlea is achieved by drilling a linear channel from the outer skull to the cochlea. The PCI approach requires pre- and intra-operative planning. Segmentation of critical ear anatomy and computation of a safe drilling trajectory are performed in a pre-operative CT. The computed safe drilling trajectory must then be mapped to the intraoperative space. The mapping can be done using the transformation matrix that registers the pre- and intra-operative CTs. However, the difference in orientation between the pre- and intra-operative CTs is too extreme to be recovered by standard, gradient descent-based registration methods. Thus, we have so far relied on an expert to manually initialize the registration. In this work we present a method that aligns the scans automatically. We compared the performance of the automatic approach to the registration approach when an expert does the manual initialization on ten pairs of scans. There is a maximum difference of 0.19 mm between the entry and target points resulting from the automatic and manually initialized registration processes. This suggests that the automatic registration method is accurate enough to be used in a PCI surgery.

Recent studies have found correlation between the risk of rupture of saccular aneurysms and their morphological characteristics, such as volume, surface area, neck length, among others. For reliably exploiting these parameters in endovascular treatment planning, it is crucial that they are accurately quantified. In this paper, we present a novel framework to assist physicians in accurately assessing saccular aneurysms and efficiently planning for endovascular intervention. The approach consists of automatically segmenting the pathological vessel, followed by the construction of its surface representation. The aneurysm is then separated from the vessel surface through a graph-cut based algorithm that is driven by local geometry as well as strong prior information. The corresponding healthy vessel is subsequently reconstructed and measurements representing the patient-specific geometric parameters of pathological vessel are computed. To better support clinical decisions on stenting and device type selection, the placement of virtual stent is eventually carried out in conformity with the shape of the diseased vessel using the patient-specific measurements. We have implemented the proposed methodology as a fully functional system, and extensively tested it with phantom and real datasets.

Bronchoscopy-guidance systems assist physicians during bronchoscope navigation. However, these systems require an attending technician and fail to continuously track the bronchoscope. We propose a real-time technicianfree bronchoscopy-guidance system that employs continuous tracking. For guidance, our system presents directions on virtual views that are generated from the bronchoscope's tracked location. The system achieves bronchoscope tracking using a strategy that is based on a recently proposed method for sensor-based bronchoscope-motion tracking.1 Furthermore, a graphical indicator notifies the physician when he/she has maneuvered the bronchoscope to an incorrect branch. Our proposed system uses the sensor data to generate virtual views through multiple candidate routes and employs image matching in a Bayesian framework to determine the most probable bronchoscope pose. Tests based on laboratory phantoms validate the potential of the system.

Bronchoscopic biopsy of lymph nodes is an important step in staging lung cancer. Lymph nodes, however, lie behind the airway walls and are near large vascular structures - all of these structures are hidden from the bronchoscope's field of view. Previously, we had presented a computer-based virtual bronchoscopic navigation system that provides reliable guidance for bronchoscopic sampling. While this system offers a major improvement over standard practice, bronchoscopists told us that target localization- lining up the bronchoscope before deploying a needle into the target - can still be challenging. We therefore address target localization in two distinct ways: (1) automatic computation of an optimal diagnostic sampling pose for safe, effective biopsies, and (2) a novel visualization of the target and surrounding major vasculature. The planning determines the final pose for the bronchoscope such that the needle, when extended from the tip, maximizes the tissue extracted. This automatically calculated local pose orientation is conveyed in endoluminal renderings by a 3D arrow. Additional visual cues convey obstacle locations and target depths-of-sample from arbitrary instantaneous viewing orientations. With the system, a physician can freely navigate in the virtual bronchoscopic world perceiving the depth-of-sample and possible obstacle locations at any endoluminal pose, not just one pre-determined optimal pose. We validated the system using mediastinal lymph nodes in eleven patients. The system successfully planned for 20 separate targets in human MDCT scans. In particular, given the patient and bronchoscope constraints, our method found that safe, effective biopsies were feasible in 16 of the 20 targets; the four remaining targets required more aggressive safety margins than a "typical" target. In all cases, planning computation took only a few seconds, while the visualizations updated in real time during bronchoscopic navigation.

Localizing sub-palpable nodules in minimally invasive video-assisted thoracic surgery (VATS) presents a significant challenge. To overcome inherent problems of preoperative nodule tagging using CT fluoroscopic guidance, an intraoperative C-arm cone-beam CT (CBCT) image-guidance system has been developed for direct localization of subpalpable tumors in the OR, including real-time tracking of surgical tools (including thoracoscope), and video-CBCT registration for augmentation of the thoracoscopic scene. Acquisition protocols for nodule visibility in the inflated and deflated lung were delineated in phantom and animal/cadaver studies. Motion compensated reconstruction was implemented to account for motion induced by the ventilated contralateral lung. Experience in CBCT-guided targeting of simulated lung nodules included phantoms, porcine models, and cadavers. Phantom studies defined low-dose acquisition protocols providing contrast-to-noise ratio sufficient for lung nodule visualization, confirmed in porcine specimens with simulated nodules (3-6mm diameter PE spheres, ~100-150HU contrast, 2.1mGy). Nodule visibility in CBCT of the collapsed lung, with reduced contrast according to air volume retention, was more challenging, but initial studies confirmed visibility using scan protocols at slightly increased dose (~4.6-11.1mGy). Motion compensated reconstruction employing a 4D deformation map in the backprojection process reduced artifacts associated with motion blur. Augmentation of thoracoscopic video with renderings of the target and critical structures (e.g., pulmonary artery) showed geometric accuracy consistent with camera calibration and the tracking system (2.4mm registration error). Initial results suggest a potentially valuable role for CBCT guidance in VATS, improving precision in minimally invasive, lungconserving surgeries, avoid critical structures, obviate the burdens of preoperative localization, and improve patient safety.

CT-fluoroscopy (CTF) is an efficient imaging method for guiding percutaneous lung interventions such as biopsy. During CTF-guided biopsy procedure, four to ten axial sectional images are captured in a very short time period to provide nearly real-time feedback to physicians, so that they can adjust the needle as it is advanced toward the target lesion. Although popularly used in clinics, this traditional CTF-guided intervention procedure may require frequent scans and cause unnecessary radiation exposure to clinicians and patients. In addition, CTF only generates limited slices of images and provides limited anatomical information. It also has limited response to respiratory movements and has narrow local anatomical dynamics. To better utilize CTF guidance, we propose a fast CT-CTF registration algorithm with respiratory motion estimation for image-guided lung intervention using electromagnetic (EM) guidance. With the pre-procedural exhale and inhale CT scans, it would be possible to estimate a series of CT images of the same patient at different respiratory phases. Then, once a CTF image is captured during the intervention, our algorithm can pick the best respiratory phase-matched 3D CT image and performs a fast deformable registration to warp the 3D CT toward the CTF. The new 3D CT image can be used to guide the intervention by superimposing the EM-guided needle location on it. Compared to the traditional repetitive CTF guidance, the registered CT integrates both 3D volumetric patient data and nearly real-time local anatomy for more effective and efficient guidance. In this new system, CTF is used as a nearly real-time sensor to overcome the discrepancies between static pre-procedural CT and the patient's anatomy, so as to provide global guidance that may be supplemented with electromagnetic (EM) tracking and to reduce the number of CTF scans needed. In the experiments, the comparative results showed that our fast CT-CTF algorithm can achieve better registration accuracy.