This PDF file contains the front matter associated with SPIE Proceedings Volume 7964, including the Title Page, Copyright information, Table of Contents, Introduction, and the Conference Committee listing.

The alignment of image-space to physical-space lies at the heart of all image-guided procedures. In intracranial surgery, point-based registrations can be used with either skin-affixed or bone-implanted extrinsic objects called fiducial markers. The advantages of point-based registration techniques are that they are robust, fast, and have a well developed mathematical foundation for the assessment of registration quality. In abdominal image-guided procedures such techniques have not been successful. It is difficult to accurately locate sufficient homologous intrinsic points in imagespace and physical-space, and the implantation of extrinsic fiducial markers would constitute "surgery before the surgery." Image-space to physical-space registration for abdominal organs has therefore been dominated by surfacebased registration techniques which are iterative, prone to local minima, sensitive to initial pose, and sensitive to percentage coverage of the physical surface. In our work in image-guided kidney surgery we have developed a composite approach using "virtual fiducials." In an open kidney surgery, the perirenal fat is removed and the surface of the kidney is dotted using a surgical marker. A laser range scanner (LRS) is used to obtain a surface representation and matching high definition photograph. A surface to surface registration is performed using a modified iterative closest point (ICP) algorithm. The dots are extracted from the high definition image and assigned the three dimensional values from the LRS pixels over which they lie. As the surgery proceeds, we can then use point-based registrations to re-register the spaces and track deformations due to vascular clamping and surgical tractions.

Off-pump, intracardiac, beating heart surgery has the potential to improve patient outcomes by eliminating the need for cardiopulmonary bypass and aortic cross clamping but it requires extensive image guidance as well as the development of specialized instrumentation. Previously, developments in image guidance and instrumentation were validated on either a static phantom or in vivo through porcine models. This paper describes the design and development of a surgical phantom for simulating off-pump mitral valve replacement inside the closed beating heart. The phantom allows surgical access to the mitral annulus while mimicking the pressure inside the beating heart. An image guidance system using tracked ultrasound, magnetic instrument tracking and preoperative models previously developed for off-pump mitral valve replacement is applied to the phantom. Pressure measurements and ultrasound images confirm the phantom closely mimics conditions inside the beating heart.

The ability to perform fast, accurate, deformable registration with intraoperative images featuring surgical excisions was investigated for use in cone-beam CT (CBCT) guided head and neck surgery. Existing deformable registration methods generally fail to account for tissue excised between image acquisitions and typically simply "move" voxels within the images with no ability to account for tissue that is removed (or introduced) between scans. We have thus developed an approach in which an extra dimension is added during the registration process to act as a sink for voxels removed during the course of the procedure. A series of cadaveric images acquired using a prototype CBCT-capable C-arm were used to model tissue deformation and excision occurring during a surgical procedure, and the ability of deformable registration to correctly account for anatomical changes under these conditions was investigated. Using a previously developed version of the Demons deformable registration algorithm, we identify the difficulties that traditional registration algorithms encounter when faced with excised tissue and present a modified version of the algorithm better suited for use in intraoperative image-guided procedures. Studies were performed for different deformation and tissue excision tasks, and registration performance was quantified in terms of the ability to accurately account for tissue excision while avoiding spurious deformations arising around the excision.

Intraoperative surface imaging with navigated beta-probes has been shown to be a possibility to enable control of tumor resection borders. By employing ad hoc models of the detection physics the image quality can be improved. Our model computes the amount of radiation from a single point source that reaches the detector, with the solid angle subtended by the detector on the source, assuming perfect shielding. The sensitivity of the detector to the source due to the angle between the detector axis and the source-to-detector vector is also considered. A set of experiments was performed with three sources (two 10x10mm² and one 20x10mm² pieces of cellulose saturated with FDG) on a plate as phantom. Five sets of measurements were taken, three of them at a distance of 10mm from the plate und two at 30mm. At both distances one measurement set was taken in a random manner and the other ones systematically covering the whole area. The same experiments were simulated with our model and the GATE simulation framework. The resulting measurements from the experiments and simulations were then used to perform a reconstruction of the sources. The real measurements were compared to those simulated with our model and GATE, with a mean NCC of 80.64% for our model and 70.14% for GATE. In the reconstructions of the real measurements the sources were visually quite well separated, however the reconstructions of the measurements simulated by the model show that there is still room for further improvement.

Microwave ablation is a promising option in lung cancer therapy. However, it's rarely used in percutaneous lung cancer therapy compared to liver cancer, because the presence of a large amount of air within the lung creates significant back shadowing artifacts that preclude adequate delineation of anatomic details on sonography. To utilize microwave ablation in malignant lung tumor therapy, we developed a novel percutaneous intervention surgery navigation system (CAINS-I), which capitalizes on using computer assisted technology to help lung cancer patients whose condition are not amenable to surgical resection, sonographic guidance and intraoperative CT surgery. In these surgeries, preoperative CT images with patient respiration state are first acquired, which are then visualized using GPU-accelerated volume rendering. The optimal surgery trajectories are then planned based on 3D thermal field computation and surgery simulation in the surgery planning software. During the surgery, the patient breath is control by a portable volume ventilator system which could limit the movement and displacement of the tumor. Then the microwave probe is punctured into the tumor according to the dynamic respiratory state and the tumor is ablated by microwave energy. After the surgery, postoperative CT are acquired and compared to the preoperative CT, and the surgery is evaluated by compare preoperative and postoperative CT images. The development of this technique represented an advance from the traditional ways for lung cancer therapy and significantly extends the indications of microwave ablation.

While several mosaicking algorithms have been developed to compose endoscopic images of the internal urinary bladder wall into panoramic images, the quantitative evaluation of these output images in terms of geometrical distortions have often not been discussed. However, the visualization of the distortion level is highly desired for an objective image-based medical diagnosis. Thus, we present in this paper a method to create quality maps from the characteristics of transformation parameters, which were applied to the endoscopic images during the registration process of the mosaicking algorithm. For a global first view impression, the quality maps are laid over the panoramic image and highlight image regions in pseudo-colors according to their local distortions. This illustration supports then surgeons to identify geometrically distorted structures easily in the panoramic image, which allow more objective medical interpretations of tumor tissue in shape and size. Aside from introducing quality maps in 2-D, we also discuss a visualization method to map panoramic images onto a 3-D spherical bladder model. Reference points are manually selected by the surgeon in the panoramic image and the 3-D model. Then the panoramic image is mapped by the Hammer-Aitoff equal-area projection onto the 3-D surface using texture mapping. Finally the textured bladder model can be freely moved in a virtual environment for inspection. Using a two-hemisphere bladder representation, references between panoramic image regions and their corresponding space coordinates within the bladder model are reconstructed. This additional spatial 3-D information thus assists the surgeon in navigation, documentation, as well as surgical planning.

The localization of brachytherapy seeds in relation to the prostate is a key step in intraoperative treatment planning (ITP) for improving outcomes in prostate cancer patients treated with low dose rate prostate brachytherapy. Transrectal ultrasound (TRUS) has traditionally been the modality of choice to guide the prostate brachytherapy procedure due to its relatively low cost and apparent ease of use. However, TRUS is unable to visualize seeds well, precluding ITP and producing suboptimal results. While other modalities such as X-ray and magnetic resonance imaging have been investigated to localize seeds in relation to the prostate, photoacoustic imaging has become an emerging and promising modality to solve this challenge. Moreover, photoacoustic imaging may be more practical in the clinical setting compared to other methods since it adds little additional equipment to the ultrasound system already adopted in procedure today, reducing cost and simplifying engineering steps. In this paper, we demonstrate the latest efforts of localizing prostate brachytherapy seeds using photoacoustic imaging, including visualization of multiple seeds in actual prostate tissue. Although there are still several challenges to be met before photoacoustic imaging can be used in the operating room, we are pleased to present the current progress in this effort.

Planned in-situ radiosensitization may improve the therapeutic ratio of image guided ¹²⁵I prostate brachytherapy. Spacers used in permanent implants may be manufactured from a radiosensitizer-releasing polymer to deliver protracted localized sensitization of the prostate. Such devices will have a limited drug-loading capacity, and the drug release schedule that optimizes outcome, under such a constraint, is not known. This work determines the optimal elution schedules for ¹²⁵I prostate brachytherapy. The interaction between brachytherapy dose distributions and drug distribution around drug eluting spacers is modeled using a linear-quadratic (LQ) model of cell kill. Clinical brachytherapy plans were used to calculate the biologic effective dose (BED) for planned radiation dose distributions while adding the spatial distributions of radiosensitizer while varying the temporal release schedule subject to a constraint on the drug capacity of the eluting spacers. Results: The greatest increase in BED is achieved by schedules with the greatest sensitization early in the implant. Making brachytherapy spacers from radiosensitizer eluting polymer transforms inert parts of the implant process into a means of enhancing the effect of the brachytherapy radiation. Such an approach may increase the therapeutic ratio of prostate brachytherapy or offer a means of locally boosting the radiation effect without increasing the radiation dose to surrounding tissues.

To make Quantitative Radiology (QR) a reality in routine clinical practice, computerized automatic anatomy recognition (AAR) becomes essential. As part of this larger goal, we present in this paper a novel fuzzy strategy for building bodywide group-wise anatomic models. They have the potential to handle uncertainties and variability in anatomy naturally and to be integrated with the fuzzy connectedness framework for image segmentation. Our approach is to build a family of models, called the Virtual Quantitative Human, representing normal adult subjects at a chosen resolution of the population variables (gender, age). Models are represented hierarchically, the descendents representing organs contained in parent organs. Based on an index of fuzziness of the models, 32 thorax data sets, and 10 organs defined in them, we found that the hierarchical approach to modeling can effectively handle the non-linear relationships in position, scale, and orientation that exist among organs in different patients.

The problem of extrapolating cost-effective relevant information from distinctly finite or sparse data, while balancing the competing goals between workflow and engineering design, and between application and accuracy is the 'sparse data extrapolation problem'. Within the context of open abdominal image-guided liver surgery, one realization of this problem is compensating for non-rigid organ deformations while maintaining workflow for the surgeon. More specifically, rigid organ-based surface registration between CT-rendered liver surfaces and laser-range scanned intraoperative partial surface counterparts resulted in an average closest-point residual 6.1 ± 4.5 mm with maximumsigned distances ranging from -13.4 to 16.2 mm. Similar to the neurosurgical environment, there is a need to correct for soft tissue deformation to translate image-guided interventions to the abdomen (e.g. liver, kidney, pancreas, etc.). While intraoperative tomographic imaging is available, these approaches are less than optimal solutions to the sparse data extrapolation problem. In this paper, we compare and contrast three sparse data extrapolation methods to that of datarich interpolation for the correction of deformation within a liver phantom containing 43 subsurface targets. The findings indicate that the subtleties in the initial alignment pose following rigid registration can affect correction up to 5- 10%. The best deformation compensation achieved was approximately 54.5% (target registration error of 2.0 ± 1.6 mm) while the data-rich interpolative method was 77.8% (target registration error of 0.6 ± 0.5 mm).

Needle insertion planning for digital breast tomosynthesis (DBT) guided biopsy has the potential to improve patient comfort and intervention safety. However, a relevant planning should take into account breast tissue deformation and lesion displacement during the procedure. Deformable models, like finite elements, use the elastic characteristics of the breast to evaluate the deformation of tissue during needle insertion. This paper presents a novel approach to locally estimate the Young's modulus of the breast tissue directly from the DBT data. The method consists in computing the fibroglandular percentage in each of the acquired DBT projection images, then reconstructing the density volume. Finally, this density information is used to compute the mechanical parameters for each finite element of the deformable mesh, obtaining a heterogeneous DBT based breast model. Preliminary experiments were performed to evaluate the relevance of this method for needle path planning in DBT guided biopsy. The results show that the heterogeneous DBT based breast model improves needle insertion simulation accuracy in 71% of the cases, compared to a homogeneous model or a binary fat/fibroglandular tissue model.

1- or 2-directional MRI blood flow mapping sequences are an integral part of standard MR protocols for diagnosis and therapy control in heart diseases. Recent progress in rapid MRI has made it possible to acquire volumetric, 3-directional cine images in reasonable scan time. In addition to flow and velocity measurements relative to arbitrarily oriented image planes, the analysis of 3-dimensional trajectories enables the visualization of flow patterns, local features of flow trajectories or possible paths into specific regions. The anatomical and functional information allows for advanced hemodynamic analysis in different application areas like stroke risk assessment, congenital and acquired heart disease, aneurysms or abdominal collaterals and cranial blood flow. The complexity of the 4D MRI flow datasets and the flow related image analysis tasks makes the development of fast comprehensive data exploration software for advanced flow analysis a challenging task. Most existing tools address only individual aspects of the analysis pipeline such as pre-processing, quantification or visualization, or are difficult to use for clinicians. The goal of the presented work is to provide a software solution that supports the whole image analysis pipeline and enables data exploration with fast intuitive interaction and visualization methods. The implemented methods facilitate the segmentation and inspection of different vascular systems. Arbitrary 2- or 3-dimensional regions for quantitative analysis and particle tracing can be defined interactively. Synchronized views of animated 3D path lines, 2D velocity or flow overlays and flow curves offer a detailed insight into local hemodynamics. The application of the analysis pipeline is shown for 6 cases from clinical practice, illustrating the usefulness for different clinical questions. Initial user tests show that the software is intuitive to learn and even inexperienced users achieve good results within reasonable processing times.

There are commercial products which provide 3D rendered volumes, reconstructed from electro-anatomical mapping and/or pre-operative CT/MR images of a patient's heart with tools for highlighting target locations for cardiac ablation applications. However, it is not possible to update the three-dimensional (3D) volume intraoperatively to provide the interventional cardiologist with more up-to-date feedback at each instant of time. In this paper, we describe the system we have developed for real-time three-dimensional stereo visualization for cardiac ablation. A 4D ultrasound probe is used to acquire and update a 3D image volume. A magnetic tracking device is used to track the distal part of the ablation catheter in real time and a master-slave robot-assisted system is developed for actuation of a steerable catheter. Three-dimensional ultrasound image volumes go through some processing to make the heart tissue and the catheter more visible. The rendered volume is shown in a virtual environment. The catheter can also be added as a virtual tool to this environment to achieve a higher update rate on the catheter's position. The ultrasound probe is also equipped with an EM tracker which is used for online registration of the ultrasound images and the catheter tracking data. The whole augmented reality scene can be shown stereoscopically to enhance depth perception for the user. We have used transthoracic echocardiography (TTE) instead of the conventional transoesophageal (TEE) or intracardiac (ICE) echocardiogram. A beating heart model has been used to perform the experiments. This method can be used both for diagnostic and therapeutic applications as well as training interventional cardiologists.

We present a novel view on 2D/3D image registration by introducing a generic algorithmic framework that is based on supervised machine learning (SML). First and foremost, this class of algorithms, referred to as texture model registration (TMR), aims at making 2D/3D registration applicable for time-critical image guided medical procedures. TMR methods are two-stage. In a first offline pre-computational stage, a prediction rule is derived from a pre-interventional 3D image and according geometric constraints. This is achieved by computing digitally reconstructed radiographs, pre-processing them, extracting their texture, and applying SML methods. In a second online stage, the inferred rule is used for predicting the spatial rigid transformation of unseen intrainterventional 2D images. A first simple concrete TMR implementation, referred to as TMR-PCR, is introduced. This approach involves principal component regression (PCR) and simple intermediate pre-processing steps. Using TMR-PCR, first experimental results on five clinical IGRT 3D data sets and synthetic intra-interventional images are presented. The implementation showed an average registration rate of 48 Hz over 40000 registrations, and succeeded in the majority of cases with a mean target registration error smaller than 2 mm. Finally, the potential and characteristics of the proposed methodical framework are discussed.

A successful image-guided surgical intervention requires accurate measurement of coordinate systems. Uncertainty is introduced every time a pose is measured by the optical tracking system. When we transform a measured pose into a different coordinate system, the covariance (which encodes the uncertainty of the pose) must be propagated to this new coordinate system. In this paper, we describe a method for propagating covariances estimated from registration, tracking, and instrument calibration into the tip of the surgical tool. This is clinically important, since it is at the tool tip that the clinician cares about uncertainty. We demonstrate that the propagation method, which is computed in real time as the tool moves through space, reliably computes the propagated covariance by comparing our estimate to true covariances from Monte Carlo simulations.

Previous studies have shown that bronchoscopy guidance systems improve accuracy and reduce skill variation among physicians during bronchoscopy. In the past, we presented an image-based bronchoscopy guidance system that has been extensively validated in live bronchoscopic procedures. However, this system cannot actively recover from adverse events, such as patient coughing or dynamic airway collapses. After such events, the bronchoscope position is recovered only by moving back to a previously seen and easily identifiable bifurcation such as the main carina. Furthermore, the system requires an attending technician to closely follow the physician's movement of the bronchoscope to avoid misguidance. Also, when the physician is forced to advance the bronchoscope across multiple bifurcations, the system is not able to detect faulty maneuvers. We propose two system-level solutions. The first solution is a system-level guidance strategy that incorporates a global-registration algorithm to provide the physician with updated navigational and guidance information during bronchoscopy. The system can handle general navigation to a region of interest (ROI), as well as adverse events, and it requires minimal commands so that it can be directly controlled by the physician. The second solution visualizes the global picture of all the bifurcations and their relative orientations in advance and suggests the maneuvers needed by the bronchoscope to approach the ROI. Guided bronchoscopy results using human airway-tree phantoms demonstrate the potential of the two solutions.

Registration of endoscopic video to preoperative CT facilitates high-precision surgery of the head, neck, and skull-base. Conventional video-CT registration is limited by the accuracy of the tracker and does not use the underlying video or CT image data. A new image-based video registration method has been developed to overcome the limitations of conventional tracker-based registration. This method adds to a navigation system based on intraoperative C-arm cone-beam CT (CBCT), in turn providing high-accuracy registration of video to the surgical scene. The resulting registration enables visualization of the CBCT and planning data within the endoscopic video. The system incorporates a mobile C-arm, integrated with an optical tracking system, video endoscopy, deformable registration of preoperative CT with intraoperative CBCT, and 3D visualization. Similarly to tracker-based approach, the image-based video-CBCT registration the endoscope is localized with optical tracking system followed by a direct 3D image-based registration of the video to the CBCT. In this way, the system achieves video-CBCT registration that is both fast and accurate. Application in skull-base surgery demonstrates overlay of critical structures (e.g., carotid arteries) and surgical targets with sub-mm accuracy. Phantom and cadaver experiments show consistent improvement of target registration error (TRE) in video overlay over conventional tracker-based registration-e.g., 0.92mm versus 1.82mm for image-based and tracker-based registration, respectively. The proposed method represents a two-fold advance-first, through registration of video to up-to-date intraoperative CBCT, and second, through direct 3D image-based video-CBCT registration, which together provide more confident visualization of target and normal tissues within up-to-date images.

We propose a novel neuro-fuzzy hybrid transformation model for deformable image registration in intra-operative image guided procedures involving large soft tissue deformation. The hybrid model consists of two parts: a physics-based model and a mathematical approximation model. The physics-based model is based on elastic solid mechanics to model major deformation patterns of the central part of organs, and the mathematical approximation model depicts the deformation of the residual part along organ boundary. A neuro-fuzzy technique is employed to seamlessly integrate the two parts into a unified hybrid model. Its unique feature is to incorporate domain knowledge of soft tissue deformation patterns and significantly reduce the number of transformation parameters. We demonstrate the effectiveness of our hybrid model to register liver magnetic resonance (MR) images in human subject study. This technique has the potential to significantly improve intra-operative image guidance in abdominal and thoracic procedures.

Congenital heart defect (CHD) is the most common birth defect and a frequent cause of death for children. Tetralogy of Fallot (ToF) is the most often occurring CHD which affects in particular the pulmonary valve and trunk. Emerging interventional methods enable percutaneous pulmonary valve implantation, which constitute an alternative to open heart surgery. While minimal invasive methods become common practice, imaging and non-invasive assessment tools become crucial components in the clinical setting. Cardiac computed tomography (CT) and cardiac magnetic resonance imaging (cMRI) are techniques with complementary properties and ability to acquire multiple non-invasive and accurate scans required for advance evaluation and therapy planning. In contrary to CT which covers the full 4D information over the cardiac cycle, cMRI often acquires partial information, for example only one 3D scan of the whole heart in the end-diastolic phase and two 2D planes (long and short axes) over the whole cardiac cycle. The data acquired in this way is called sparse cMRI. In this paper, we propose a regression-based approach for the reconstruction of the full 4D pulmonary trunk model from sparse MRI. The reconstruction approach is based on learning a distance function between the sparse MRI which needs to be completed and the 4D CT data with the full information used as the training set. The distance is based on the intrinsic Random Forest similarity which is learnt for the corresponding regression problem of predicting coordinates of unseen mesh points. Extensive experiments performed on 80 cardiac CT and MR sequences demonstrated the average speed of 10 seconds and accuracy of 0.1053mm mean absolute error for the proposed approach. Using the case retrieval workflow and local nearest neighbour regression with the learnt distance function appears to be competitive with respect to "black box" regression with immediate prediction of coordinates, while providing transparency to the predictions made.

Bronchoscopy-guidance systems have been shown to improve the success rate of bronchoscopic procedures. A key technical cornerstone of bronchoscopy-guidance systems is the synchronization between the virtual world, derived from a patient's three-dimensional (3D) multidetector computed-tomography (MDCT) scan, and the real world, derived from the bronchoscope video during a live procedure. Two main approaches for synchronizing these worlds exist: electromagnetic navigation bronchoscopy (ENB) and image-based bronchoscopy. ENB systems require considerable extra hardware, and both approaches have drawbacks that hinder continuous robust guidance. In addition, they both require an attending technician to be present. We propose a technician-free strategy that enables real-time guidance of bronchoscopy. The approach uses measurements of the bronchoscope's movement to predict its position in 3D virtual space. To achieve this, a bronchoscope model, defining the device's shape in the airway tree to a given point p, provides an insertion depth to p. In real time, our strategy compares an observed bronchoscope insertion depth and roll angle, measured by an optical sensor, to precalculated insertion depths along a predefined route in the virtual airway tree. This leads to a prediction of the bronchoscope's location and orientation. To test the method, experiments involving a PVC-pipe phantom and a human airway-tree phantom verified the bronchoscope models and the entire method, respectively. The method has considerable potential for improving guidance robustness and simplicity over other bronchoscopy-guidance systems.

Airway wall remodeling in asthma and chronic obstructive pulmonary disease (COPD) is a well-known indicator of the pathology. In this context, current clinical studies aim for establishing the relationship between the airway morphological structure and its function. Multislice computed tomography (MSCT) allows morphometric assessment of airways, but requires dedicated segmentation tools for clinical exploitation. While most of the existing tools are limited to cross-section measurements, this paper develops a fully 3D approach for airway wall segmentation. Such approach relies on a deformable model which is built up as a patient-specific surface model at the level of the airway lumen and deformed to reach the outer surface of the airway wall. The deformation dynamics obey a force equilibrium in a Lagrangian framework constrained by a vector field which avoids model self-intersections. The segmentation result allows a dense quantitative investigation of the airway wall thickness with a deeper insight at bronchus subdivisions than classic cross-section methods. The developed approach has been assessed both by visual inspection of 2D cross-sections, performed by two experienced radiologists on clinical data obtained with various protocols, and by using a simulated ground truth (pulmonary CT image model). The results confirmed a robust segmentation in intra-pulmonary regions with an error in the range of the MSCT image resolution and underlined the interest of the volumetric approach versus purely 2D methods.

Transbronchial needle aspiration (TBNA) is a common procedure to collect tissue samples from the inside of the lung for diagnostic use. However, the main drawback of the procedure is that it has to be blindly performed because the biopsy target region is behind the bronchial wall and hence not within the field of view of the bronchoscope. Thus, the diagnostic yield rate is low. To increase success rate of TBNA biopsy an electromagnetic trackable TBNA needle has been introduced. Nevertheless, the introduced prototype TBNA instrument was evaluated in a rigid rubber phantom without taking respiratory motion into account. The purpose of this study is to present a new TBNA needle where the electromagnetic sensor is directly integrated into a TBNA needle and to access its performance in a regularly ventilated lung. Using our previously presented navigation system, seven TBNA interventions were performed in a porcine lung during regular respiration lung movement; respectively a control computer tomography scan was acquired. We evaluated tracking accuracy of the electromagnetically tracked needle during the entire respiratory cycle for each intervention. The newly developed TBNA needle successfully operated throughout all seven interventions. According to the results, our electromagnetic TBNA tracking system is a promising approach to increase the TBNA biopsy success rate.

This paper presents an improved bronchoscope tracking method for bronchoscopic navigation using scale invariant features and sequential Monte Carlo sampling. Although image-based methods are widely discussed in the community of bronchoscope tracking, they are still limited to characteristic information such as bronchial bifurcations or folds and cannot automatically resume the tracking procedure after failures, which result usually from problematic bronchoscopic video frames or airway deformation. To overcome these problems, we propose a new approach that integrates scale invariant feature-based camera motion estimation into sequential Monte Carlo sampling to achieve an accurate and robust tracking. In our approach, sequential Monte Carlo sampling is employed to recursively estimate the posterior probability densities of the bronchoscope camera motion parameters according to the observation model based on scale invariant feature-based camera motion recovery. We evaluate our proposed method on patient datasets. Experimental results illustrate that our proposed method can track a bronchoscope more accurate and robust than current state-of-the-art method, particularly increasing the tracking performance by 38.7% without using an additional position sensor.

Purpose: Ultrasound (US) elevation beamwidth causes a certain type of image artifact around the anechoic areas of the tissue. It is generally assumed that the US image is of zero thickness, which contradicts the fact that the acoustic beam can only be mechanically focused at a depth resulting in a finite, non-uniformed elevation beamwidth. We suspect that elevation beamwidth artifacts contribute to target reconstruction error in computer-assisted interventions. This paper introduces a method for characterization of the beamwidth for transrectal ultrasound (TRUS) used in prostate brachythyerapy. In particular, we measure how the US sectionthickness varies along the beam's axial depth. Method: We developed a beam-profiling device (a TRUS-bridge phantom) specifically tailored for standard brachytherapy ultrasound imaging systems to generate a complete section-thickness profile of a given TRUS transducer. The device was designed in CAD software and prototyped by a 3D printer. Result: The experimental results demonstrated that the TRUS beam in the elevation direction is focused closely to the transducer and theoretically the transducer would provide a better elevational resolution within that range. Conclusion: We presented a beam profiling phantom to measure the section-thickness of a transrectal ultrasound transducer for operating room use. However, there are some limitations which need to be addressed, for example, phantom sterilization and the speed of sound in the current medium of experiment which is not the same as that of biological tissues.

We apply the initial momentum shape representation of diffeomorphic metric mapping from a template region of interest (ROI) to a given ROI as a morphometic marker in Parkinson's disease. We used a three-step segmentation-registrationmomentum process to derive feature vectors from ROIs in a group of 42 subjects consisting of 19 Parkinson's Disease (PD) subjects and 23 normal control (NC) subjects. Significant group differences between PD and NC subjects were detected in four basal ganglia structures including the caudate, putamen, thalamus and globus pallidus. The magnitude of regionally significant between-group differences detected ranged between 34-75%. Visualization of the different structural deformation pattern between-groups revealed that some parts of basal ganglia structure actually hypertrophy, presumably as a compensatory response to more widespread atrophy. Our results of both hypertrophy and atrophy in the same structures further demonstrate the importance of morphological measures as opposed to overall volume in the assessment of neurodegenerative disease.

A number of groups have reported on the occurrence of intra-operative brain shift during deep brain stimulation (DBS) surgery. This has a number of implications for the procedure including an increased chance of intra-cranial bleeding and complications due to the need for more exploratory electrodes to account for the brain shift. It has been reported that the amount of pneumocephalus or air invasion into the cranial cavity due to the opening of the dura correlates with intraoperative brain shift. Therefore, pre-operatively predicting the amount of pneumocephalus expected during surgery is of interest toward accounting for brain shift. In this study, we used 64 DBS patients who received bilateral electrode implantations and had a post-operative CT scan acquired immediately after surgery (CT-PI). For each patient, the volumes of the pneumocephalus, left ventricle, right ventricle, third ventricle, white matter, grey matter, and cerebral spinal fluid were calculated. The pneumocephalus was calculated from the CT-PI utilizing a region growing technique that was initialized with an atlas-based image registration method. A multi-atlas-based image segmentation method was used to segment out the ventricles of each patient. The Statistical Parametric Mapping (SPM) software package was utilized to calculate the volumes of the cerebral spinal fluid (CSF), white matter and grey matter. The volume of individual structures had a moderate correlation with pneumocephalus. Utilizing a multi-linear regression between the volume of the pneumocephalus and the statistically relevant individual structures a Pearson's coefficient of r = 0.4123 (p = 0.0103) was found. This study shows preliminary results that could be used to develop a method to predict the amount of pneumocephalus ahead of the surgery.

Preoperative magnetic resonance images are typically used for neuronavigation in image-guided neurosurgery. However, intraoperative brain deformation (e.g., as a result of gravitation, loss of cerebrospinal fluid, retraction, resection, etc.) significantly degrades the accuracy in image guidance, and must be compensated for in order to maintain sufficient accuracy for navigation. Biomechanical finite element models are effective techniques that assimilate intraoperative data and compute whole-brain deformation from which to generate model-updated MR images (uMR) to improve accuracy in intraoperative guidance. To date, most studies have focused on early surgical stages (i.e., after craniotomy and durotomy), whereas simulation of more complex events at later surgical stages has remained to be a challenge using biomechanical models. We have developed a method to simulate partial or complete tumor resection that incorporates intraoperative volumetric ultrasound (US) and stereovision (SV), and the resulting whole-brain deformation was used to generate uMR. The 3D ultrasound and stereovision systems are complimentary to each other because they capture features deeper in the brain beneath the craniotomy and at the exposed cortical surface, respectively. In this paper, we illustrate the application of the proposed method to simulate brain tumor resection at three temporally distinct surgical stages throughout a clinical surgery case using sparse displacement data obtained from both the US and SV systems. We demonstrate that our technique is feasible to produce uMR that agrees well with intraoperative US and SV images after dural opening, after partial tumor resection, and after complete tumor resection. Currently, the computational cost to simulate tumor resection can be up to 30 min because of the need for re-meshing and the trial-and-error approach to refine the amount of tissue resection. However, this approach introduces minimal interruption to the surgical workflow, which suggests the potential for its clinical application with further improvement in computational efficiency.

Compensating for brain shift as surgery progresses is important to ensure sufficient accuracy in patient-to-image registration in the operating room (OR) for reliable neuronavigation. Ultrasound has emerged as an important and practical imaging technique for brain shift compensation either by itself or through computational modeling that estimates whole-brain deformation. Using volumetric true 3D ultrasound (3DUS), it is possible to nonrigidly (e.g., based on B-splines) register two temporally different 3DUS images directly to generate feature displacement maps for data assimilation in the biomechanical model. Because of a large amount of data and number of degrees-of-freedom (DOFs) involved, however, a significant computational cost may be required that can adversely influence the clinical feasibility of the technique for efficiently generating model-updated MR (uMR) in the OR. This paper parametrically investigates three B-splines registration parameters and their influence on the computational cost and registration accuracy: number of grid nodes along each direction, floating image volume down-sampling rate, and number of iterations. A simulated rigid body displacement field was employed as a ground-truth against which the accuracy of displacements generated from the B-splines nonrigid registration was compared. A set of optimal parameters was then determined empirically that result in a registration computational cost of less than 1 min and a sub-millimetric accuracy in displacement measurement. These resulting parameters were further applied to a clinical surgery case to demonstrate their practical use. Our results indicate that the optimal set of parameters result in sufficient accuracy and computational efficiency in model computation, which is important for future application of the overall biomechanical modeling to generate uMR for image-guidance in the OR.

Recent work from our group has shown the primacy of the bifurcation area ratio and tortuosity in determining the amount of disturbed flow at the carotid bifurcation, believed to be a local risk factor for the carotid atherosclerosis. We have also presented fast and reliable methods of extraction of geometry from routine 3D contrast-enhanced magnetic resonance angiography, as the necessary step along the way for large-scale trials of such local risk factors. In the present study, we refine our original geometric variables to better reflect the underlying fluid mechanical principles. Flaring of the bifurcation, leading to flow separation, is defined by the maximum relative expansion of the common carotid artery (CCA), proximal to the bifurcation apex. The beneficial effect of curvature on flow inertia, via its suppression of flow separation, is now characterized by the tortuosity of CCA as it enters the flare region. Based on data from 50 normal carotid bifurcations, multiple linear regressions of these new independent geometric predictors against the dependent disturbed flow burden reveals adjusted R² values approaching 0.5, better than the values closer to 0.3 achieved using the original variables. The excellent scan-rescan reproducibility demonstrated for our earlier geometric variables is shown to be preserved for the new definitions. Improved prediction of disturbed flow by robust and reproducible vascular geometry offers a practical pathway to large-scale studies of local risk factors in atherosclerosis.

Diagnosis and treatment decisions of cerebrovascular diseases are currently based on structural information like the endovascular lumen. In future, clinical diagnosis will increasingly be based on functional information which gives direct information about the physiological parameters and, hence, is a direct measure for the severity of the pathology. In this context, an important functional quantity is the volumetric blood flow over time. The proposed flow quantification method uses contrasted X-ray images from cerebrovascular interventions and a model of contrast agent dispersion to estimate the flow parameters from the spatial and temporal development of the contrast agent concentration through the vascular system. To evaluate the model-based blood flow quantification under realistic circumstances, dedicated cerebrovascular data has been acquired during clinical interventions. To this aim, a clinical protocol for this novel procedure has been defined and optimized. For the verification of the measured flow results ultrasound Doppler measurements have been performed acting as reference measurements. The clinical data available so far indicates the ability of the proposed flow model to explain the in-vivo transport of contrast agent in blood. The flow quantification results show good correspondence of flow waveform and mean volumetric flow rate with the accomplished ultrasound measurements before or after angiography.

We propose a system to estimate blood flow velocity in angiographic image data for patient-specific blood flow simulations. Angiographies are acquired routinely for diagnosis and before treatment of vascular diseases. Projective blood flow is measured in digital subtraction X-ray angiography (2D-DSA) images by tracking contrast agent propagation. Spatial information is added by re-projecting 2D centerline pixels to the reconstructed 3D X-ray rotation angiography (3D-RA) data of the same subject. Ambiguities caused by occluding vessels from the virtual viewpoint of the acquired 2D-DSA image are resolved by a graph-based approach. The blood flow velocity can be used as boundary condition for exact blood flow simulations that can help physicians to understand hemodynamics of the vasculature. Our focus is to analyze cerebral angiographic data. We performed several experiments with phantom and patient data that proved the accuracy and the functionality of our method. We evaluated experimentally the projective flow estimation method and the re-projection method. We measured mean deviations to the ground truth between 11 % and 15.7 % for phantom data. We also showed the ability of our method to produce plausible results with patient-data.

Presentation of detailed anatomical structures via 3-D models helps navigation and deployment of the prosthetic valve in TAVI procedures. Fast and automatic contrast detection in the aortic root on X-ray images facilitates a seamless workflow to utilize the 3-D models by triggering 2-D/3-D registration automatically when motion compensation is needed. In this paper, we propose a novel method for automatic detection of contrast injection in the aortic root on fluoroscopic and angiographic sequences. The proposed method is based on histogram analysis and likelihood ratio test, and is robust to variations in the background, the density and volume of the injected contrast, and the size of the aorta. The performance of the proposed algorithm was evaluated on 26 sequences from 5 patients and 3 clinical sites, with 16 out of 17 contrast injections correctly detected and zero false detections. The proposed method is of general form and can be extended for detection of contrast injection in other organs and/or applications.

Cardiac magnetic resonance perfusion imaging (CMR) and computed tomography angiography (CTA) are widely used to assess heart disease. CMR is used to measure the global and regional myocardial function and to evaluate the presence of ischemia; CTA is used for diagnosing coronary artery disease, such as coronary stenoses. Nowadays, the hemodynamic significance of coronary artery stenoses is determined subjectively by combining information on myocardial function with assumptions on coronary artery territories. As the anatomy of coronary arteries varies greatly between individuals, we developed a patient-specific tool for relating CTA and perfusion CMR data. The anatomical and functional information extracted from CTA and CMR data are combined into a single frame of reference. Our graphical user interface provides various options for visualization. In addition to the standard perfusion Bull's Eye Plot (BEP), it is possible to overlay a 2D projection of the coronary tree on the BEP, to add a 3D coronary tree model and to add a 3D heart model. The perfusion BEP, the 3D-models and the CTA data are also interactively linked. Using the CMR and CTA data of 14 patients, our tool directly established a spatial correspondence between diseased coronary artery segments and myocardial regions with abnormal perfusion. The location of coronary stenoses and perfusion abnormalities were visualized jointly in 3D, thereby facilitating the study of the relationship between the anatomic causes of a blocked artery and the physiological effects on the myocardial perfusion. This tool is expected to improve diagnosis and therapy planning of early-stage coronary artery disease.

Atrial fibrillation is a common cardiac arrhythmia in which aberrant electrical activity cause the atria to quiver which results in irregular beating of the heart. Catheter ablation therapy is becoming increasingly popular in treating atrial fibrillation, a procedure in which an electrophysiologist guides a catheter into the left atrium and creates radiofrequency lesions to stop the arrhythmia. Typical visualization tools include bi-plane fluoroscopy, 2-D ultrasound, and electroanatomic maps, however, recently there has been increased interest in incorporating preoperative surface models into the procedure. Typical strategies for registration include landmark-based and surface-based methods. Drawbacks of these approaches include difficulty in accurately locating corresponding landmark pairs and the time required to sample surface points with a catheter. In this paper, we describe a new approach which models the catheter tip as a Gaussian kernel and eliminates the need to collect surface points by instead using the stream of continuosly tracked catheter points. We demonstrate the feasibility of this technique with a left atrial phantom model and compare the results with a standard surface based approach.

During coronary artery angiography, a catheter is used to inject a contrast dye into the coronary arteries. Due to the anatomical variation of the aorta and the coronary arteries in different humans, one common catheter cannot be used for all patients. The cardiologists test different catheters for a patient and select the best catheter according to the patient's anatomy. This procedure is time consuming and there is a slight chance of cancer from excessive exposure to radiation. To overcome these problems, we propose a computer aided catheter selection procedure. In this paper we present our approach for the angiography of the Right Coronary Artery (RCA). Our approach involves segmentation of the aorta and coronary arteries, finding the centerline and computing the Curve Angle (CA) and Curve Length (CL) between the aorta and the coronary arteries. We then compute CA and CL of catheters and suggest a catheter with the closest CA and CL with respect to the aorta's and coronary arteries' CA and CL. This solution avoids testing of many catheters during catheterization. The cardiologist already gets the recommendation about the optimal catheter for the patient prior to the intervention.

X-ray image guided angioplasty is a minimally invasive procedure that involves the insertion of a catheter into a blood vessel to remove blockages to blood flow. There are several issues associated with conventional angioplasty which cause risks for the patient (damage to blood vessels, dislodging plaques, etc.) and difficulties for the clinician (X-ray exposure, fatigue, etc.). Autonomous or semi-autonomous robot-assisted catheter insertion is a solution that can reduce these problems substantially. To perform autonomous catheter insertion, closed-loop position control of the distal tip of the catheter is required during insertion. Therefore accurate real-time position feedback is needed for this purpose. We have developed a real-time image processing algorithm for catheter tip position tracking which has an acceptable performance but is sensitive to X-ray image artifacts caused by bones and dense tissues. A magnetic tracking system (MTS) is another modality that has also been used for catheter tip position tracking, but it is sensitive to external electromagnetic interferences and ferromagnetic material. Combining the measurement data provided by both imaging and magnetic sensors can compensate for the deficiencies of each and can also improve the robustness of catheter tip position tracking. We have developed a Kalman filter based sensor fusion scheme to overcome deficiencies of both of these methods and create a reliable real-time tracking of a catheter tip. Experiments have been performed by inserting a guide catheter into a model of the vasculature. The method has been tested in presence of occlusion in the images and also electromagnetic interference.

Minimally invasive surgery is a medically complex discipline that can heavily benefit from computer assistance. One way to assist the surgeon is to blend in useful information about the intervention into the surgical view using Augmented Reality. This information can be obtained during preoperative planning and integrated into a patient-tailored model of the intervention. Due to soft tissue deformation, intraoperative sensor data such as endoscopic images has to be acquired and non-rigidly registered with the preoperative model to adapt it to local changes. Here, we focus on a procedure that reconstructs the organ surface from stereo endoscopic images with millimeter accuracy in real-time. It deals with stereo camera calibration, pixel-based correspondence analysis, 3D reconstruction and point cloud meshing. Accuracy, robustness and speed are evaluated with images from a test setting as well as intraoperative images. We also present a workflow where the reconstructed surface model is registered with a preoperative model using an optical tracking system. As preliminary result, we show an initial overlay between an intraoperative and a preoperative surface model that leads to a successful rigid registration between these two models.

One of the main challenges related to computer-assisted laparoscopic surgery is the accurate registration of pre-operative planning images with patient's anatomy. One popular approach for achieving this involves intraoperative 3D reconstruction of the target organ's surface with methods based on multiple view geometry. The latter, however, require robust and fast algorithms for establishing correspondences between multiple images of the same scene. Recently, the first endoscope based on Time-of-Flight (ToF) camera technique was introduced. It generates dense range images with high update rates by continuously measuring the run-time of intensity modulated light. While this approach yielded promising results in initial experiments, the endoscopic ToF camera has not yet been evaluated in the context of related work. The aim of this paper was therefore to compare its performance with different state-of-the-art surface reconstruction methods on identical objects. For this purpose, surface data from a set of porcine organs as well as organ phantoms was acquired with four different cameras: a novel Time-of-Flight (ToF) endoscope, a standard ToF camera, a stereoscope, and a High Definition Television (HDTV) endoscope. The resulting reconstructed partial organ surfaces were then compared to corresponding ground truth shapes extracted from computed tomography (CT) data using a set of local and global distance metrics. The evaluation suggests that the ToF technique has high potential as means for intraoperative endoscopic surface registration.

The purpose of this paper is to present a detailed description of our real-time navigation system for computer assisted surgery. The system was developed with laparoscopic partial nephrectomies as a first application scenario. The main goal of the application is to enable tracking of the tumor position and orientation during a surgery. Our system is based on ultrasound to CT registration and electromagnetic tracking. The basic idea is to process tracking information to generate an augmented reality (AR) visualization of a tumor model in the camera image of a laparoscopic camera. Thereby it enhances the surgeon's view on the current scene and therefore facilitates higher safety during the surgery. So far we have applied our system in vitro during two phantom trials with a surgeon which yielded promising results.

The high recurrence rate of bladder cancer requires patients to undergo frequent surveillance screenings over their lifetime following initial diagnosis and resection. Our laboratory is developing panoramic stitching software that would compile several minutes of cystoscopic video into a single panoramic image, covering the entire bladder, for review by an urolgist at a later time or remote location. Global alignment of video frames is achieved by using a bundle adjuster that simultaneously recovers both the 3D structure of the bladder as well as the scope motion using only the video frames as input. The result of the algorithm is a complete 360° spherical panorama of the outer surface. The details of the software algorithms are presented here along with results from both a virtual cystoscopy as well from real endoscopic imaging of a bladder phantom. The software successfully stitched several hundred video frames into a single panoramic with subpixel accuracy and with no knowledge of the intrinsic camera properties, such as focal length and radial distortion. In the discussion, we outline future work in development of the software as well as identifying factors pertinent to clinical translation of this technology.

This paper describes a navigation system for flexible endoscopes equipped with ultrasound scan heads. For navigation and needle biopsy procedures it provides additional oblique slices from preoperative computed tomography (CT) volumes which are displayed with the corresponding endoscopic ultrasound (US) image. In contrast to similar systems an additional abdominal 3D ultrasound image is used to achieve the required registration. Two different approaches are compared: one method is based on direct inter-modal registration between abdominal 3D ultrasound and CT volume. The second method uses another 3D US scan taken preoperatively before the CT scan. Here, the CT is calibrated by means of an optical tracking system and the transformation between CT and the calibrated 3D US can be calculated without image registration. Before intervention, a pre-interventional 3D US is registered intra-modal to the preoperative US. This second method invoked to be the more robust and accurate procedure. For experimental studies a phantom has been developed which consists of a plastic tube inside a water tank. For error evaluation small plastic spheres have been fixed around the tube at different distances. First results give an overall error of 3.9 mm for the first method while the overall error for the intramodal method amounted to 3.1 mm.

Virtual colonoscopy provides techniques not available in optical colonoscopy, an exciting one being the ability to perform an electronic biopsy. An electronic biopsy image is created using ray-casting volume rendering of the CT data with a translucent transfer function mapping higher densities to red and lower densities to blue. The resulting image allows the physician to gain insight into the internal structure of polyps. Benign tissue and adenomas can be differentiated; the former will appear as homogeneously blue and the latter as irregular red structures. Although this technique is now common, is included with clinical systems, and has been used successfully for computer aided detection, there has so far been no study to evaluate the effectiveness of a physician using electronic biopsy in determining the pathological state of a polyp. We present here such a study, wherein an experienced radiologist ranked polyps based on electronic biopsy alone per scan (supine and prone), as well as both combined. Our results show a correct identification 77% of the time using prone or supine images alone, and 80% accuracy using both. Using ROC analysis based on this study with one reader and a modest sample size, the combined score is not significantly higher than using a single electronic biopsy image alone. However, our analysis indicates a trend of superiority for the combined ranking that deserves a follow-up confirmatory study with a larger sample and more readers. This study yields hope that an improved electronic biopsy technique could become a primary clinical diagnosis method.

For trauma and orthopedic surgery, maneuvering a mobile C-arm X-ray device into a desired position in order to acquire the right picture is a routine task. The precision and ease of use of the C-arm positioning becomes even more important for more advanced imaging techniques as parallax-free X-ray image stitching, for example. Standard mobile C-arms have only five degrees of freedom (DOF), which definitely restricts their motions that have six DOF in 3D Cartesian space. We have proposed a method to model the kinematics of the mobile Carm and operating table as an integrated 6DOF C-arm X-ray imaging system.1 This enables mobile C-arms to be positioned relative to the patient's table with six DOF in 3D Cartesian space. Moving mobile C-arms to a desired position and orientation requires finding the necessary joint values, which is an inverse kinematics problem. In this paper, we present closed-form solutions, i.e. analytic expressions, obtained in an algebraic way for the inverse kinematics problem of the 6DOF C-arm model. In addition, we implement a 6DOF C-arm system for interactively radiation-free C-arm positioning based on a continuous guidance from C-arm pose estimation. For this we employ a visual marker pattern attached under the operating table and a mobile C-arm system augmented by a video camera and mirror construction. In our experiment, repositioning C-arm to a pre-defined pose in a phantom study demonstrates the practicality and accuracy of our developed 6DOF C-arm system.

We present a spectral-based method for the 2D/3D rigid registration of X-ray images to a CT scan. The method uses a Fourier-based representation to decompose the six rigid transformation parameters problem into a twoparameter out-of-plane rotation and a four-parameter in-plane transformation problems. Preoperatively, a set of Digitally Reconstructed Radiographs (DRRs) are generated offline from the CT in the expected in-plane location ranges of the fluoroscopic X-ray imaging devices. Each DRR is transformed into a imaging device in-plane invariant features space. Intraoperatively, a few 2D projections of the patient anatomy are acquired with an X-ray imaging device. Each projection is transformed into its in-plane invariant representation. The out-of-plane parameters are first computed by maximization of the Normalized Cross-Correlation between the invariant representations of the DRRs and the X-ray images. Then, the in-plane parameters are computed with the phase correlation method based on the Fourier-Mellin transform. Experimental results on publicly available data sets show that our method can robustly estimate the out-of-plane parameters with accuracy of 1.5° in less than 1sec for out-of-plane rotations of 10° or more, and perform the entire registration in less than 10secs.

Approaches through the temporal bone require surgeons to drill away bone to expose a target skull base lesion while evading vital structures contained within it, such as the sigmoid sinus, jugular bulb, and facial nerve. We hypothesize that an augmented neuronavigation system that continuously calculates the distance to these structures and warns if the surgeon drills too close, will aid in making safe surgical approaches. Contemporary image guidance systems are lacking an automated method to segment the inhomogeneous and complexly curved facial nerve. Therefore, we developed a segmentation method to delineate the intra-temporal facial nerve centerline from clinically available temporal bone CT images semi-automatically. Our method requires the user to provide the start- and end-point of the facial nerve in a patient's CT scan, after which it iteratively matches an active appearance model based on the shape and texture of forty facial nerves. Its performance was evaluated on 20 patients by comparison to our gold standard: manually segmented facial nerve centerlines. Our segmentation method delineates facial nerve centerlines with a maximum error along its whole trajectory of 0.40±0.20 mm (mean±standard deviation). These results demonstrate that our model-based segmentation method can robustly segment facial nerve centerlines. Next, we can investigate whether integration of this automated facial nerve delineation with a distance calculating neuronavigation interface results in a system that can adequately warn surgeons during temporal bone drilling, and effectively diminishes risks of iatrogenic facial nerve palsy.

Percutaneous femoroplasty [1], or femoral bone augmentation, is a prospective alternative treatment for reducing the risk of fracture in patients with severe osteoporosis. We are developing a surgical robotics system that will assist orthopaedic surgeons in planning and performing a patient-specific, augmentation of the femur with bone cement. This collaborative project, sponsored by the National Institutes of Health (NIH), has been the topic of previous publications [2],[3] from our group. This paper presents modifications to the pose recovery of a fluoroscope tracking (FTRAC) fiducial during our process of 2D/3D registration of X-ray intraoperative images to preoperative CT data. We show improved automata of the initial pose estimation as well as lower projection errors with the advent of a multiimage pose optimization step.

Cochlear implantation is a surgical procedure for treating patients with hearing loss in which an electrode array is inserted into the cochlea. The traditional surgical approach requires drilling away a large portion of the bone behind the ear to provide anatomical reference and access to the cochlea. A minimally-invasive technique, called percutaneous cochlear implantation (PCI), has been proposed that involves drilling a linear path from the lateral skull to the cochlea avoiding vital structures and inserting the implant using that drilled path. The steps required to achieve PCI safely include: placing three bone-implanted markers surrounding the ear, obtaining a CT scan, planning a surgical path to the cochlea avoiding vital anatomy, designing and constructing a microstereotactic frame that mounts on the markers and constrains the drill to the planned path, affixing the frame on the markers, using it to drill to the cochlea, and inserting the electrode through the drilled path. We present in this paper a cadaveric study demonstrating the PCI technique on three temporal bone cadaveric specimens for inserting electrode array into the cochlea. A custom fixture, called a Microtable, which is a type of microstereotactic frame that can be constructed in less than five minutes, was fabricated for each specimen and used to reach the cochlea. The insertion was successfully performed on all three specimens. Postinsertion CT scans confirm the correct placement of the electrodes inside the cochlea without any damage to the facial nerve.

In ultrasound-guided needle insertion procedures, tracking of the needle relative to the ultrasound image is beneficial for needle trajectory planning and guidance. A single camera closed-form method is proposed for automatic real-time trajectory tracking with a low-cost camera mounted directly on the ultrasound transducer. The camera is calibrated to the ultrasound image coordinates. By mounting the camera on the transducer, issues of visual obstruction are reduced and accuracy of tracking is increased compared to camera-tracking systems with a fixed case. Compared to previous work with stereo cameras, a single camera further reduces cost, complexity and size, but requires a needle with known markings. The proposed solution uses the depth markings etched on many common needles (e.g. epidural needle). A fully automatic image processing method has been developed for real-time identification of the needle trajectory using a novel closed-form solution based on three identified markings and the camera's intrinsic calibration parameters. The trajectory of the needle relative to the ultrasound image is calculated and displayed. Validation compares the calculated intersection of the needle trajectory to the ultrasound image with the depiction of the actual needle intersection in the image. The overall error is 3.0 ± 2.6 mm for a low-cost 640×480 pixel USB camera.

A feature-based registration was developed to align biplane and tracked ultrasound images of the aortic root with a preoperative CT volume. In transcatheter aortic valve replacement, a prosthetic valve is inserted into the aortic annulus via a catheter. Poor anatomical visualization of the aortic root region can result in incorrect positioning, leading to significant morbidity and mortality. Registration of pre-operative CT to transesophageal ultrasound and fluoroscopy images is a major step towards providing augmented image guidance for this procedure. The proposed registration approach uses an iterative closest point algorithm to register a surface mesh generated from CT to 3D US points reconstructed from a single biplane US acquisition, or multiple tracked US images. The use of a single simultaneous acquisition biplane image eliminates reconstruction error introduced by cardiac gating and TEE probe tracking, creating potential for real-time intra-operative registration. A simple initialization procedure is used to minimize changes to operating room workflow. The algorithm is tested on images acquired from excised porcine hearts. Results demonstrate a clinically acceptable accuracy of 2.6mm and 5mm for tracked US to CT and biplane US to CT registration respectively.

PURPOSE: Thermal ablation is a popular method in local cancer management; however it is extremely challenging to predict thermal changes in vivo. Ultrasound could be a convenient and inexpensive imaging modality for real-time monitoring of the ablation, but the required advanced image processing algorithms need extensive validation. Our goal is to design and develop a reliable test-bed for validation of these monitoring algorithms. METHOD: We previously developed a test-bed, consisting of ablated tissue sample and fiducial lines embedded in tissue-mimicking gel.1 The gel block is imaged by ultrasound and sliced to acquire pathology images. Following fiducial localization in both image modalities, the pathology and US data were registered. Ground truth ablated region is retrieved from pathology images and compared to the result of the ultrasound-based processing in 3D space. We improved on this platform to resolve limitations that hindered its usage in a larger-scale validation study. A simulator for evaluating and optimizing different line fiducial structures was implemented, and a new fiducial line structure was proposed. RESULTS: The new proposed fiducial configuration outperforms the previous in terms of accuracy, fiducial visibility, and use of larger tissue samples. Simulation results show improvement in pose recovery accuracy using our proposed fiducial structure, reducing target registration error (TRE) by 34%. Inaccurate pixel spacing information and fiducial localization noise are the main sources of error in slice pose recovery. CONCLUSION: A new generation of test-bed was developed, with software that does not require lengthy manual data processing, and is easier to maintain and extend. Further experimental work is required to optimize phantom preparation and precise pixel spacing computation.

Needle tip dexterity is advantageous for transthoracic lung biopsies, which are typically performed with rigid, straight biopsy needles. By providing intraoperative compensation for trajectory error and lesion motion, tendon-driven biopsy needles may reach smaller or deeper nodules in fewer attempts, thereby reducing trauma. An image-guided robotic system that uses these needles also has the potential to reduce radiation exposure to the patient and physician. In this paper, we discuss the design, workflow, kinematic modeling, and control of both the needle and a compact and inexpensive robotic prototype that can actuate the tendon-driven needle for transthoracic lung biopsy. The system is designed to insert and steer the needle under Computed Tomography (CT) guidance. In a free-space targeting experiment using a discrete proportional control law with digital camera feedback, we show a position error of less than 1 mm achieved using an average of 8.3 images (n=3).

Liver tumor, one of the most wide-spread diseases, has a very high mortality in China. To improve success rates of liver surgeries and life qualities of such patients, we implement an interactive liver surgery planning system based on contrastenhanced liver CT images. The system consists of five modules: pre-processing, segmentation, modeling, quantitative analysis and surgery simulation. The Graph Cuts method is utilized to automatically segment the liver based on an anatomical prior knowledge that liver is the biggest organ and has almost homogeneous gray value. The system supports users to build patient-specific liver segment and sub-segment models using interactive portal vein branch labeling, and to perform anatomical resection simulation. It also provides several tools to simulate atypical resection, including resection plane, sphere and curved surface. To match actual surgery resections well and simulate the process flexibly, we extend our work to develop a virtual scalpel model and simulate the scalpel movement in the hepatic tissue using multi-plane continuous resection. In addition, the quantitative analysis module makes it possible to assess the risk of a liver surgery. The preliminary results show that the system has the potential to offer an accurate 3D delineation of the liver anatomy, as well as the tumors' location in relation to vessels, and to facilitate liver resection surgeries. Furthermore, we are testing the system in a full-scale clinical trial.