This PDF file contains the front matter associated with SPIE Proceedings Volume 10135, including the Title Page, Copyright information, Table of Contents, and Conference Committee listing.

Biomechanical breast models have been employed for applications in image registration and analysis, breast augmentation simulation, and for surgical and biopsy guidance. Accurate applications of stress-strain relationships of tissue within the breast can improve the accuracy of biomechanical models that attempt to simulate breast movements. Reported stiffness values for adipose, glandular, and cancerous tissue types vary greatly. Variations in reported stiffness properties are mainly due to differences in testing methodologies and assumptions, measurement errors, and natural inter patient differences in tissue elasticity. Therefore, patient specific, in vivo determination of breast tissue properties is ideal for these procedural applications. Many in vivo elastography methods are not quantitative and/or do not measure material properties under deformation conditions that are representative of the procedure being simulated in the model. In this study, we developed an elasticity estimation method that is performed using deformations representative of supine therapeutic procedures. Reconstruction of material properties was performed by iteratively fitting two anatomical images before and after tissue stimulation. The method proposed is work flow friendly, quantitative, and uses a non-contact, gravity-induced deformation source. We tested this material property optimization procedure in a healthy volunteer and in simulation. In simulation, we show that the algorithm can reconstruct properties with errors below 1% for adipose and 5.6% for glandular tissue regardless of the starting stiffness values used as initial guesses. In clinical data, reconstruction errors are higher (3.6% and 24.2%) due to increased noise in the system. In a clinical context, the elastography method was shown to be promising for use in biomechanical model assisted supine procedures.

The quality of brain tumor resection surgery is dependent on the spatial agreement between preoperative image and intraoperative anatomy. However, brain shift compromises the aforementioned alignment. Currently, the clinical standard to monitor brain shift is intraoperative magnetic resonance (iMR). While iMR provides better understanding of brain shift, its cost and encumbrance is a consideration for medical centers. Hence, we are developing a model-based method that can be a complementary technology to address brain shift in standard resections, with resource-intensive cases as referrals for iMR facilities. Our strategy constructs a deformation ‘atlas’ containing potential deformation solutions derived from a biomechanical model that account for variables such as cerebrospinal fluid drainage and mannitol effects. Volumetric deformation is estimated with an inverse approach that determines the optimal combinatory ‘atlas’ solution fit to best match measured surface deformation. Accordingly, preoperative image is updated based on the computed deformation field. This study is the latest development to validate our methodology with iMR. Briefly, preoperative and intraoperative MR images of 2 patients were acquired. Homologous surface points were selected on preoperative and intraoperative scans as measurement of surface deformation and used to drive the inverse problem. To assess the model accuracy, subsurface shift of targets between preoperative and intraoperative states was measured and compared to model prediction. Considering subsurface shift above 3 mm, the proposed strategy provides an average shift correction of 59% across 2 cases. While further improvements in both the model and ability to validate with iMR are desired, the results reported are encouraging.

Patient-specific finite element (FE) models of the breast have received increasing attention due to the potential capability of fusing images from different modalities. During the Magnetic Resonance Imaging (MRI) to X-ray mammography registration procedure, the FE model is compressed mimicking the mammographic acquisition. Subsequently, suspicious lesions in the MRI volume can be projected into the 2D mammographic space. However, most registration algorithms do not provide the reverse information, avoiding to obtain the 3D geometrical information from the lesions localized in the mammograms. In this work we introduce a fast method to localize the 3D position of the lesion within the MRI, using both cranio-caudal (CC) and medio-lateral oblique (MLO) mammographic projections, indexing the tetrahedral elements of the biomechanical model by means of an uniform grid. For each marked lesion in the Full-Field Digital Mammogram (FFDM), the X-ray path from source to the marker is calculated. Barycentric coordinates are computed in the tetrahedrons traversed by the ray. The list of elements and coordinates allows to localize two curves within the MRI and the closest point between both curves is taken as the 3D position of the lesion. The registration errors obtained in the mammographic space are 9.89 ± 3.72 mm in CC- and 8.04 ± 4.68 mm in MLO-projection and the error in the 3D MRI space is equal to 10.29 ± 3.99 mm. Regarding the uniform grid, it is computed spending between 0.1 and 0.7 seconds. The average time spent to compute the 3D location of a lesion is about 8 ms.

Lung cancer is one of the leading causes of cancer death in men and women. External Beam Radiation Therapy (EBRT) is a commonly used primary treatment for the condition. A major challenge with such treatments is the delivery of sufficient radiation dose to the lung tumor while ensuring that surrounding healthy lung parenchyma receives only minimal dose. This can be achieved by coupling EBRT with respiratory computer models which can predict the tumour location as a function of phase during the breathing cycle¹. The diaphragm muscle contraction is mainly responsible for a large portion of the lung tumor motion during normal breathing, especially when tumours are in the lower lobes, therefore the importance of accurately modelling the diaphragm is paramount in lung tumour motion prediction. The goal of this research is to develop a biomechanical model of the diaphragm, including its active and passive response, using detailed geometric, biomechanical and anatomical information that mimics the diaphragmatic behaviour in a patient specific manner. For this purpose, a Finite Element Model (FEM) of the diaphragm was developed in order to predict the in vivo motion of the diaphragm, paving the way for computer assisted lung cancer tumor tracking in EBRT. Preliminary results obtained from the proposed model are promising and they indicate that it can be used as a plausible tool for effective lung cancer EBRT to improve patient care.

External beam radiation therapy (EBRT) treats cancer by delivering daily fractions of radiation to a target volume. For prostate cancer, the target undergoes day-to-day variations in position, volume, and shape. For stereotactic photon and for proton EBRT, endorectal balloons (ERBs) can be used to limit variations. To date, patterns of non-rigid variations for patients with ERB have not been modeled. We extracted and modeled the patient-specific patterns of variations, using regularly acquired CT-images, non-rigid point cloud registration, and principal component analysis (PCA). For each patient, a non-rigid point-set registration method, called Coherent Point Drift, (CPD) was used to automatically generate landmark correspondences between all target shapes. To ensure accurate registrations, we tested and validated CPD by identifying parameter values leading to the smallest registration errors (surface matching error 0.13±0.09 mm). PCA demonstrated that 88±3.2% of the target motion could be explained using only 4 principal modes. The most dominant component of target motion is a squeezing and stretching in the anterior-posterior and superior-inferior directions. A PCA model of daily landmark displacements, generated using 6 to 10 CT-scans, could explain well the target motion for the CT-scans not included in the model (modeling error decreased from 1.83±0.8 mm for 6 CT-scans to 1.6±0.7 mm for 10 CT-scans). PCA modeling error was smaller than the naive approximation by the mean shape (approximation error 2.66±0.59 mm). Future work will investigate the use of the PCA-model to improve the accuracy of EBRT techniques that are highly susceptible to anatomical variations such as, proton therapy

Image-to-physical registration based on volumetric data like computed tomography on the one side and intraoperative endoscopic images on the other side is an important method for various surgical applications. In this contribution, we present methods to generate panoramic views from endoscopic recordings for image-to-physical registration of narrow drill holes inside spongy bone. One core application is the registration of drill poses inside the mastoid during minimally invasive cochlear implantations. Besides the development of image processing software for registration, investigations are performed on a miniaturized optical system, achieving 360° radial imaging with one shot by extending a conventional, small, rigid, rod lens endoscope. A reflective cone geometry is used to deflect radially incoming light rays into the endoscope optics. Therefore, a cone mirror is mounted in front of a conventional 0° endoscope. Furthermore, panoramic images of inner drill hole surfaces in artificial bone material are created. Prior to drilling, cone beam computed tomography data is acquired from this artificial bone and simulated endoscopic views are generated from this data. A qualitative and quantitative image comparison of resulting views in terms of image-to-image registration is performed. First results show that downsizing of panoramic optics to a diameter of 3mm is possible. Conventional rigid rod lens endoscopes can be extended to produce suitable panoramic one-shot image data. Using unrolling and stitching methods, images of the inner drill hole surface similar to computed tomography image data of the same surface were created. Registration is performed on ten perturbations of the search space and results in target registration errors of (0:487 ± 0:438)mm at the entry point and (0:957 ± 0:948)mm at the exit as well as an angular error of (1:763 ± 1:536)°. The results show suitability of this image data for image-to-image registration. Analysis of the error components in different directions reveals a strong influence of the pattern structure, meaning higher diversity results into smaller errors.

Purpose: In image-guided procedures, image acquisition is often performed primarily for the task of geometrically registering information from another image dataset, rather than detection / visualization of a particular feature. While the ability to detect a particular feature in an image has been studied extensively with respect to image quality characteristics (noise, resolution) and is an ongoing, active area of research, comparatively little has been accomplished to relate such image quality characteristics to registration performance.

Methods: To establish such a framework, we derived Cramer-Rao lower bounds (CRLB) for registration accuracy, revealing the underlying dependencies on image variance and gradient strength. The CRLB was analyzed as a function of image quality factors (in particular, dose) for various similarity metrics and compared to registration accuracy using CT images of an anthropomorphic head phantom at various simulated dose levels. Performance was evaluated in terms of root mean square error (RMSE) of the registration parameters.

Results: Analysis of the CRLB shows two primary dependencies: 1) noise variance (related to dose); and 2) sum of squared image gradients (related to spatial resolution and image content). Comparison of the measured RMSE to the CRLB showed that the best registration method, RMSE achieved the CRLB to within an efficiency factor of 0.21, and optimal estimators followed the predicted inverse proportionality between registration performance and radiation dose.

Conclusions: Analysis of the CRLB for image registration is an important step toward understanding and evaluating an intraoperative imaging system with respect to a registration task. While the CRLB is optimistic in absolute performance, it reveals a basis for relating the performance of registration estimators as a function of noise content and may be used to guide acquisition parameter selection (e.g., dose) for purposes of intraoperative registration.

Based on the Iterative Closest Point (ICP) framework, we present a generalized solution for the registration between homologous points and lines. The transformation we seek comprises an anisotropic scaling, followed by rotation and translation. This algorithm is demonstrated using the Perspective-n-Point (PnP) problem where lines form a bundle, and the Non-Perspective-n-Point (NPnP) problem where each line potentially has its own origin. We also prove that one existing NPnP solution is, in fact, equivalent to ICP, and that a second PnP solution differs from ICP only in the iteratively estimated translation. Applications for these types of registration include ultrasound calibration, kinematics tracking under fluoroscopic video, and camera pose estimation. Simulation results suggest this ICP algorithm compares favorably to existing PnP and NPnP algorithms, and has an extremely compact formulation.

Pelvic Kirschner wire (K-wire) insertion is a challenging surgical task requiring interpretation of complex 3D anatomical shape from 2D projections (fluoroscopy) and delivery of device trajectories within fairly narrow bone corridors in proximity to adjacent nerves and vessels. Over long trajectories (~10-25 cm), K-wires tend to curve (deform), making conventional rigid navigation inaccurate at the tip location. A system is presented that provides accurate 3D localization and guidance of rigid or deformable surgical devices (“components” – e.g., K-wires) based on 3D-2D registration. The patient is registered to a preoperative CT image by virtually projecting digitally reconstructed radiographs (DRRs) and matching to two or more intraoperative x-ray projections. The K-wire is localized using an analogous procedure matching DRRs of a deformably parametrized model for the device component (deformable known-component registration, or dKC-Reg). A cadaver study was performed in which a K-wire trajectory was delivered in the pelvis. The system demonstrated target registration error (TRE) of 2.1 ± 0.3 mm in location of the K-wire tip (median ± interquartile range, IQR) and 0.8 ± 1.4º in orientation at the tip (median ± IQR), providing functionality analogous to surgical tracking / navigation using imaging systems already in the surgical arsenal without reliance on a surgical tracker. The method offers quantitative 3D guidance using images (e.g., inlet / outlet views) already acquired in the standard of care, potentially extending the advantages of navigation to broader utilization in trauma surgery to improve surgical precision and safety.

A 3D kinematic measurement of joint movement is crucial for orthopedic surgery assessment and diagnosis. This is usually obtained through a frame-by-frame registration of the 3D bone volume to a fluoroscopy video of the joint movement. The high cost of a high-quality fluoroscopy imaging system has hindered the access of many labs to this application. This is while the more affordable and low-dosage version, the mini C-arm, is not commonly used for this application due to low image quality. In this paper, we introduce a novel method for kinematic analysis of joint movement using the mini C-arm. In this method the bone of interest is recovered and isolated from the rest of the image using a non-rigid registration of an atlas to each frame. The 3D/2D registration is then performed using the weighted histogram of image gradients as an image feature. In our experiments, the registration error was 0.89 mm and 2.36° for human C2 vertebra. While the precision is still lacking behind a high quality fluoroscopy machine, it is a good starting point facilitating the use of mini C-arms for motion analysis making this application available to lower-budget environments. Moreover, the registration was highly resistant to the initial distance from the true registration, converging to the answer from anywhere within ±90° of it.

Image registration of preprocedural contrast-enhanced CTs to intraprocedual cone-beam computed tomography (CBCT) can provide additional information for interventional liver oncology procedures such as transcatheter arterial chemoembolisation (TACE). In this paper, a novel similarity metric for gradient-based image registration is proposed. The metric relies on the patch-based computation of histograms of oriented gradients (HOG) building the basis for a feature descriptor. The metric was implemented in a framework for rigid 3D-3D-registration of pre-interventional CT with intra-interventional CBCT data obtained during the workflow of a TACE. To evaluate the performance of the new metric, the capture range was estimated based on the calculation of the mean target registration error and compared to the results obtained with a normalized cross correlation metric. The results show that 3D HOG feature descriptors are suitable as image-similarity metric and that the novel metric can compete with established methods in terms of registration accuracy

For patients with malignant brain tumors (glioblastomas), a safe maximal resection of tumor is critical for an increased survival rate. However, complete resection of the cancer is hard to achieve due to the invasive nature of these tumors, where the margins of the tumors become blurred from frank tumor to more normal brain tissue, but in which single cells or clusters of malignant cells may have invaded. Recent developments in fluorescence imaging techniques have shown great potential for improved surgical outcomes by providing surgeons intraoperative contrast-enhanced visual information of tumor in neurosurgery. The current near-infrared (NIR) fluorophores, such as indocyanine green (ICG), cyanine5.5 (Cy5.5), 5-aminolevulinic acid (5-ALA)-induced protoporphyrin IX (PpIX), are showing clinical potential to be useful in targeting and guiding resections of such tumors. Real-time tumor margin identification in NIR imaging could be helpful to both surgeons and patients by reducing the operation time and space required by other imaging modalities such as intraoperative MRI, and has the potential to integrate with robotically assisted surgery. In this paper, a segmentation method based on the Chan-Vese model was developed for identifying the tumor boundaries in an ex-vivo mouse brain from relatively noisy fluorescence images acquired by a multimodal scanning fiber endoscope (mmSFE). Tumor contours were achieved iteratively by minimizing an energy function formed by a level set function and the segmentation model. Quantitative segmentation metrics based on tumor-to-background (T/B) ratio were evaluated. Results demonstrated feasibility in detecting the brain tumor margins at quasi-real-time and has the potential to yield improved precision brain tumor resection techniques or even robotic interventions in the future.

Purpose. One of the goals of new navigation systems in the operating room and in outpatient clinics is to support the surgeon's decision making while minimizing the additional load on surrounding health personnel. To do so, the system needs to rely on context-awareness providing the surgeon with the most relevant visualization at all times. Such a system could also provide support for surgical training. The objective of this work is to assess the feasibility of an automatic surgical phase recognition using tracking data from a novel instrument for injections and biopsies. Methods. An injection into the sphenopalatine ganglion planned with MRI and CT images is carried out using optical tracking of the instrument. In the context of a feasibility study, the intervention was performed by 5 operators, each 5 times, on a specially designed phantom. The coordinate information is processed into 7 features characterizing the intervention. Three classifiers, Hidden Markov Model (HMM), a Support Vector Machine (SVM), and a combination of these (SVM+HMM) are trained on manually annotated data and cross-validated for intra- and inter-operator variability. Standard test metrics are used to compare the performance of each classifier. Results. HMM alone and SVM alone are comparable classifiers, but feeding the output of the SVM into an HMM results in significantly better classifications: accuracy of 97.8%, sensitivity of 93.1% and specificity of 98.4%. Conclusion. The use of trajectory information can provide a robust real-time phase recognition of surgical phases for needle-based interventions.

Intracerebral hemorrhages (ICHs) occur in 24 out of 100,000 people annually and have high morbidity and mortality rates. The standard treatment is conservative. We hypothesize that a patient-specific, mechanical model coupled with a robotic steerable needle, used to aspirate a hematoma, would result in a minimally invasive approach to ICH management that will improve outcomes. As a preliminary study, three realizations of a tissue aspiration framework are explored within the context of a biphasic finite element model based on Biot's consolidation theory. Short-term transient effects were neglected in favor of steady state formulation. The Galerkin Method of Weighted Residuals was used to solve coupled partial differential equations using linear basis functions, and assumptions of plane strain and homogeneous isotropic properties. All aspiration models began with the application of aspiration pressure sink(s), calculated pressures and displacements, and the use of von Mises stresses within a tissue failure criterion. With respect to aspiration strategies, one model employs an element-deletion strategy followed by aspiration redeployment on the remaining grid, while the other approaches use principles of superposition on a fixed grid. While the element-deletion approach had some intuitive appeal, without incorporating a dynamic grid strategy, it evolved into a less realistic result. The superposition strategy overcame this, but would require empirical investigations to determine the optimum distribution of aspiration sinks to match material removal. While each modeling framework demonstrated some promise, the superposition method's ease of computation, ability to incorporate the surgical plan, and better similarity to existing empirical observational data, makes it favorable.

Several neurosurgical procedures, such as Artero Venous Malformations (AVMs), aneurysm embolizations and StereoElectroEncephaloGraphy (SEEG) require accurate reconstruction of the cerebral vascular tree, as well as the classification of arteries and veins, in order to increase the safety of the intervention. Segmentation of arteries and veins from 4D CT perfusion scans has already been proposed in different studies. Nonetheless, such procedures require long acquisition protocols and the radiation dose given to the patient is not negligible. Hence, space is open to approaches attempting to recover the dynamic information from standard Contrast Enhanced Cone Beam Computed Tomography (CE-CBCT) scans. The algorithm proposed by our team is called ART 3.5 D. It is a novel algorithm based on the postprocessing of both the angiogram and the raw data of a standard Digital Subtraction Angiography from a CBCT (DSACBCT) allowing arteries and veins segmentation and labeling without requiring any additional radiation exposure for the patient and neither lowering the resolution. In addition, while in previous versions of the algorithm just the distinction of arteries and veins was considered, here the capillary phase simulation and identification is introduced, in order to increase further information useful for more precise vasculature segmentation.

Intraoperative microelectrode recordings (MER) have been used for several decades to guide neurosurgeons during the implantation of Deep Brain Stimulation (DBS) electrodes, especially when targeting the subthalamic nucleus (STN) to suppress the symptoms of Parkinson’s Disease. The standard approach is to use an array of up to five MER electrodes in a fixed configuration. Interpretation of the recorded signals yields a spatially very sparse set of information about the morphology of the respective brain structures in the targeted area. However, no aid is currently available for surgeons to intraoperatively integrate this information with other data available on the patient’s individual morphology (e.g. MR imaging data used for surgical planning). This integration might allow surgeons to better determine the most probable position of the electrodes within the target structure during surgery. This paper suggests a method for reconstructing a surface patch from the sparse MER dataset utilizing additional a priori knowledge about the geometrical configuration of the measurement electrodes. The conventional representation of MER measurements as intervals of target region/non-target region is therefore transformed into an equivalent boundary set representation, allowing ecient point-based calculations. Subsequently, the problem is to integrate the resulting patch with a preoperative model of the target structure, which can be formulated as registration problem minimizing a distance measure between the two surfaces. When restricting this registration procedure to translations, which is reasonable given certain geometric considerations, the problem can be solved globally by employing an exhaustive search with arbitrary precision in polynomial time. The proposed method is demonstrated using bilateral STN/Substantia Nigra segmentation data from preoperative MRIs of 17 Patients with simulated MER electrode placement. When using simulated data of heavily perturbed electrodes and subsequent MER measurements, our optimization resulted in an improvement of the electrode position within 1 mm of the ground truth in 80.29% of the cases.

PURPOSE: In scoliosis monitoring, tracked ultrasound has been explored as a safer imaging alternative to traditional radiography. The use of ultrasound in spinal curvature measurement requires identification of vertebral landmarks such as transverse processes, but as bones have reduced visibility in ultrasound imaging, skeletal landmarks are typically segmented manually, which is an exceedingly laborious and long process. We propose an automatic algorithm to segment and localize the surface of bony areas in the transverse process for scoliosis in ultrasound.

METHODS: The algorithm uses cascade of filters to remove low intensity pixels, smooth the image and detect bony edges. By applying first differentiation, candidate bony areas are classified. The average intensity under each area has a correlation with the possibility of a shadow, and areas with strong shadow are kept for bone segmentation. The segmented images are used to reconstruct a 3-D volume to represent the whole spinal structure around the transverse processes. RESULTS: A comparison between the manual ground truth segmentation and the automatic algorithm in 50 images showed 0.17 mm average difference. The time to process all 1,938 images was about 37 Sec. (0.0191 Sec. / Image), including reading the original sequence file.

CONCLUSION: Initial experiments showed the algorithm to be sufficiently accurate and fast for segmentation transverse processes in ultrasound for spinal curvature measurement. An extensive evaluation of the method is currently underway on images from a larger patient cohort and using multiple observers in producing ground truth segmentation.

Lumbar punctures (LPs) are interventional procedures used to collect cerebrospinal fluid (CSF), a bodily fluid needed to diagnose central nervous system disorders. Most lumbar punctures are performed blindly without imaging guidance. Because the target window is small, physicians can only accurately palpate the appropriate space about 30% of the time and perform a successful procedure after an average of three attempts. Although various forms of imaging based guidance systems have been developed to aid in this procedure, these systems complicate the procedure by including independent image modalities and requiring image-to-needle registration to guide the needle insertion. Here, we propose a simple and direct needle insertion platform utilizing a single ultrasound element within the needle through dynamic sensing and imaging. The needle-shaped ultrasound transducer can not only sense the distance between the tip and a potential obstacle such as bone, but also visually locate structures by combining transducer location tracking and back projection based tracked synthetic aperture beam-forming algorithm. The concept of the system was validated through simulation first, which revealed the tolerance to realistic error. Then, the initial prototype of the single element transducer was built into a 14G needle, and was mounted on a holster equipped with a rotation tracking encoder. We experimentally evaluated the system using a metal wire phantom mimicking high reflection bone structures and an actual spine bone phantom with both the controlled motion and freehand scanning. An ultrasound image corresponding to the model phantom structure was reconstructed using the beam-forming algorithm, and the resolution was improved compared to without beam-forming. These results demonstrated the proposed system has the potential to be used as an ultrasound imaging system for lumbar puncture procedures.

Percutaneous needle insertion procedures on the spine often require proper identification of the vertebral level in order to effectively deliver anesthetics and analgesic agents to achieve adequate block. For example, in obstetric epidurals, the target is at the L3-L4 intervertebral space. The current clinical method involves “blind” identification of the vertebral level through manual palpation of the spine, which has only 30% accuracy. This implies the need for better anatomical identification prior to needle insertion. A system is proposed to identify the vertebrae, assigning them to their respective levels, and track them in a standard sequence of ultrasound images, when imaged in the paramedian plane. Machine learning techniques are developed to identify discriminative features of the laminae. In particular, a deep network is trained to automatically learn the anatomical features of the lamina peaks, and classify image patches, for pixel-level classification. The chosen network utilizes multiple connected auto-encoders to learn the anatomy. Pre-processing with ultrasound bone enhancement techniques is done to aid the pixel-level classification performance. Once the lamina are identified, vertebrae are assigned levels and tracked in sequential frames. Experimental results were evaluated against an expert sonographer. Based on data acquired from 15 subjects, vertebrae identification with sensitivity of 95% and precision of 95% was achieved within each frame. Between pairs of subsequently analyzed frames, matches of predicted vertebral level labels were correct in 94% of cases, when compared to matches of manually selected labels

PURPOSE: Ultrasound imaging is an attractive alternative to X-ray for scoliosis diagnosis and monitoring due to its safety and inexpensiveness. The transverse processes as skeletal landmarks are accessible by means of ultrasound and are sufficient for quantifying scoliosis, but do not provide an informative visualization of the spine. METHODS: We created a method for visualization of the scoliotic spine using a 3D transform field, resulting from thin-spline interpolation of a landmark-based registration between the transverse processes that we localized in both the patient’s ultrasound and an average healthy spine model. Additional anchor points were computationally generated to control the thin-spline interpolation, in order to gain a transform field that accurately represents the deformation of the patient’s spine. The transform field is applied to the average spine model, resulting in a 3D surface model depicting the patient’s spine. We applied ground truth CT from pediatric scoliosis patients in which we reconstructed the bone surface and localized the transverse processes. We warped the average spine model and analyzed the match between the patient’s bone surface and the warped spine. RESULTS: Visual inspection revealed accurate rendering of the scoliotic spine. Notable misalignments occurred mainly in the anterior-posterior direction, and at the first and last vertebrae, which is immaterial for scoliosis quantification. The average Hausdorff distance computed for 4 patients was 2.6 mm. CONCLUSIONS: We achieved qualitatively accurate and intuitive visualization to depict the 3D deformation of the patient’s spine when compared to ground truth CT.

Cochlear implants (CIs) are considered standard treatment for patients who experience sensorineural hearing loss. Although these devices have been remarkably successful at restoring hearing, it is rare to achieve natural fidelity, and many patients experience poor outcomes. Our group has developed the first image-guided CI programming (IGCIP) technique where the positions of the electrodes are found in CT images and used to estimate neural activation patterns, which is unique information that audiologists can use to define patient-specific processor settings. In our current system, neural activation is estimated using only the distance from each electrode to the neural activation sites. This approach might be less accurate than using a high-resolution electro-anatomical model (EAM) of the electrically stimulated cochlea to perform physics-based estimation of neural activation. In this work, we propose a patientcustomized EAM approach where the EAM is spatially and electrically adapted to a patient-specific configuration. Spatial adaptation is done through non-rigid registration of the model with the patient CT image. Electrical adaptation is done by adjusting tissue resistivity parameters so that the intra-cochlear voltage distributions predicted by the model best match those directly measured for the patient via their implant. We demonstrated our approach for N=7 patients. We found that our approach results in mean percent differences between direct and simulated measurements of voltage distributions of 11%. In addition, visual comparison shows the simulated and measured voltage distributions are qualitatively in good agreement. This represents a crucial step toward developing and validating the first in vivo patient-specific cochlea EAMs.

Cochlear Implants (CIs) are surgically implantable neural prosthetic devices used to treat profound hearing loss. Recent literature indicates that there is a correlation between the positioning of the electrode array within the cochlea and the ultimate hearing outcome of the patient, indicating that further studies aimed at better understanding the relationship between electrode position and outcomes could have significant implications for future surgical techniques, array design, and processor programming methods. Post-implantation high resolution CT imaging is the best modality for localizing electrodes and provides the resolution necessary to visually identify electrode position, albeit with an unknown degree of accuracy depending on image acquisition parameters, like the HU range of reconstruction, radiation dose, and resolution of the image. In this paper, we report on the development of a phantom that will both permit studying which CT acquisition parameters are best for accurately identifying electrode position and serve as a ground truth for evaluating how different electrode localization methods perform when using different CT scanners and acquisition parameters. We conclude based on our tests that image resolution and HU range of reconstruction strongly affect how accurately the true position of the electrode array can be found by both experts and automatic analysis techniques. The results presented in this paper demonstrate that our phantom is a versatile tool for assessing how CT acquisition parameters affect the localization of CIs.

Cochlear implants must be inserted carefully to avoid damaging the delicate anatomical structures of the inner ear. This has motivated several approaches to improve the safety and efficacy of electrode array insertion by automating the process with specialized robotic or manual insertion tools. When such tools are used, they must be positioned at the entry point to the cochlea and aligned with the desired entry vector. This paper presents an image guidance system capable of accurately positioning a cochlear implant insertion tool. An optical tracking system localizes the insertion tool in physical space while a graphical user interface incorporates this with patient- specific anatomical data to provide error information to the surgeon in real-time. Guided by this interface, novice users successfully aligned the tool with an mean accuracy of 0.31 mm.

A new method for template-guided cochlear implantation surgery is proposed which has been developed to create a minimally invasive access to the inner ear. A first design of the surgical template was drafted, built, and finally tested regarding its accuracy. For individual finalization of the micro-stereotactic frame bone cement is utilized as this well-known and well-established material suggests ease of use as well as high clinical acceptance and enables both sterile and rapid handling. The new concept includes an alignment device, based on a passive hexapod with manually adjustable legs for temporary fixation of the separate parts in the patient-specific pose until the bone cement is spread and finally cured. Additionally, a corresponding evaluation method was developed to determine the accuracy of the microstereotactic frame in some initial experiments. In total 18 samples of the surgical template were fabricated based on previously planned trajectories. The mean positioning error at the target point was 0.30 mm with a standard deviation of 0.25 mm.

Cochlear implants (CIs) are used to treat patients with severe-to-profound hearing loss. In surgery, an electrode array is implanted in the cochlea. After implantation, the CI processor is programmed by an audiologist. One factor that negatively impacts outcomes and can be addressed by programming is cross-electrode neural stimulation overlap (NSO). In the recent past, we have proposed a system to assist the audiologist in programming the CI that we call Image-Guided CI Programming (IGCIP). IGCIP permits using CT images to detect NSO and recommend which subset of electrodes should be active to avoid NSO. In an ongoing clinical study, we have shown that IGCIP leads to significant improvement in hearing outcomes. Most of the IGCIP steps are robustly automated but electrode configuration selection still sometimes requires expert intervention. With expertise, Distance-Vs-Frequency (DVF) curves, which are a way to visualize the spatial relationship learned from CT between the electrodes and the nerves they stimulate, can be used to select the electrode configuration. In this work, we propose an automated technique for electrode configuration selection. It relies on matching new patients’ DVF curves to a library of DVF curves for which electrode configurations are known. We compare this approach to one we have previously proposed. We show that, generally, our new method produces results that are as good as those obtained with our previous one while being generic and requiring fewer parameters.

While clinical neurosurgery has benefited from the advent of frameless image guidance for over three decades, the translation of image guided technologies to abdominal surgery, and more specifically liver resection, has been far more limited. Fundamentally, the workflow, complexity, and presentation have confounded development. With the first real efforts in translation beginning at the turn of the millennia, the work in developing novel augmented technologies to enhance screening, planning, and surgery has come to realization for the field. In this paper, we will review several examples from our own work that demonstrate the impact of image-guided procedure methods in eight clinical studies that speak to: (1) the accuracy in planning for liver resection, (2) enhanced surgical planning with portal vein embolization impact, (3) linking splenic volume changes to post-hepatectomy complications, (4) enhanced intraoperative localization in surgically occult lesions, (5) validation of deformation correction, and a (6) a novel blinded study focused at the value of deformation correction. All six of these studies were achieved in human systems and show the potential impact image guided methodologies could make on liver tissue resection procedures.

Purpose: This study presents development and integration of hardware and software that enables ultrasound (US) and computer tomography (CT) fusion for a FDA-approved CT-guided needle placement robot. Having real-time US image registered to a priori-taken intraoperative CT image provides more anatomic information during needle insertion, in order to target hard-to-see lesions or avoid critical structures invisible to CT, track target motion, and to better monitor ablation treatment zone in relation to the tumor location. Method: A passive encoded mechanical arm is developed for the robot in order to hold and track an abdominal US transducer. This 4 degrees of freedom (DOF) arm is designed to attach to the robot end-effector. The arm is locked by default and is released by a press of button. The arm is designed such that the needle is always in plane with US image. The articulated arm is calibrated to improve its accuracy. Custom designed software (OncoNav, NIH) was developed to fuse real-time US image to a priori-taken CT. Results: The accuracy of the end effector before and after passive arm calibration was 7.07mm ± 4.14mm and 1.74mm ±1.60mm, respectively. The accuracy of the US image to the arm calibration was 5mm. The feasibility of US-CT fusion using the proposed hardware and software was demonstrated in an abdominal commercial phantom. Conclusions: Calibration significantly improved the accuracy of the arm in US image tracking. Fusion of US to CT using the proposed hardware and software was feasible.

PURPOSE: The open-source Perk Tutor training platform has been shown to improve trainee performance in interventions that require ultrasound guidance. Our goal was to determine if needle coordination of medical trainees can be improved by training with Perk Tutor compared to training with ultrasound only. METHODS: Twenty participants with no previous experience were randomized into two groups; the Perk Tutor group and the Control group. The Perk Tutor group had access to the 3D visualization while the Control group used ultrasound only during their training. Performance was analyzed, measured and compared by Perk Tutor with regards to four needle coordination metrics. None of the groups had access to 3D visualization during performance testing. RESULTS: The needle tracking measurements showed, for the Perk Tutor group, lower average distance between the needle tip and ultrasound (1.2 [0.9 – 2.8] mm vs 2.7 [2.3 – 4.0] mm, respectively; P = 0.023) and lower maximum distance between the needle tip and ultrasound (2.2 [1.9 – 3.2] mm vs 4.6 [3.9 – 6.2] mm, respectively; P = 0.013). There was no significant difference in average needle to ultrasound plane angle and maximum needle to ultrasound plane distance. All participants were successful in the procedure. CONCLUSION: The Perk Tutor group had significantly reduced distance from the needle tip to the ultrasound plane. Training with Perk Tutor can improve trainees’ needle and ultrasound coordination.

MRI-guided needle intervention for cryoablation is a promising way to relieve the pain and treat the cancer. However, the limited size of MRI bore makes it impossible for clinicians to perform the operation in the bore. The patients had to be moved into the bore for scanning to verify the position of the needle’s tip and out for adjusting the needle’s trajectory. Real-time needle tracking and shown in MR images is of importance for clinicians to perform the operation more efficiently. In this paper, we have instrumented the cryotherapy needle with a MRI-safe electromagnetic (EM) sensor and optical sensor to measure the needle’s position and orientation. To overcome the limitation of line-of-sight for optical sensor and the poor dynamic performance of the EM sensor, Kalman filter based data fusion is developed. Further, we developed a navigation system in open-source software, 3D Slicer, to provide accurate visualization of the needle and the surrounding anatomy. Experiment of simulation the needle intervention at the entrance was performed with a realistic spine phantom to quantify the accuracy of the navigation using the retrospective analysis method. Eleven trials of needle insertion were performed independently. The target accuracy with the navigation using only EM sensor, only optical sensor and data fusion are 2.27 ±1.60 mm, 4.11 ± 1.77 mm and 1.91 − 1.10 mm, respectively.

State-of-the-art treatment for joint diseases like osteoarthritis focus on articular cartilage repair/regeneration by stem cell implantation therapy. However, the technique is limited by a lack of precision in the physician’s imaging and cell deposition toolkit. We describe a novel combination of high-resolution, rapid scan-rate optical coherence tomography (OCT) alongside a short-pulsed nanosecond thulium (Tm) laser for precise cell seeding in cartilage. The superior beam quality of thulium lasers and wavelength of operation 1940 nm offers high volumetric tissue removal rates and minimizes the residual thermal footprint. OCT imaging enables targeted micro-well placement, precise cell deposition, and feature contrast. A bench-top system is constructed using a 15 W, 1940 nm, nanosecond-pulsed Tm fiber laser (500 μJ pulse energy, 100 ns pulse duration, 30kHz repetition rate) for removing tissue, and a swept source laser (1310 ± 70 nm, 100 kHz sweep rate) for OCT imaging, forming a combined Tm/OCT system – a “smart laser knife”. OCT assists the smart laser knife user in characterizing cartilage to inform micro-well placement. The Tm laser creates micro-wells (2.35 mm diameter length, 1.5 mm width, 300 μm deep) and micro-incisions (1 mm wide, 200 μm deep) while OCT image-guidance assists and demonstrates this precision cutting and cell deposition with real-time feedback. To test micro-well creation and cell deposition protocol, gelatin phantoms are constructed mimicking cartilage optical properties and physiological structure. Cell viability is then assessed to illustrate the efficacy of the hydrogel deposition. Automated OCT feedback is demonstrated for cutting procedures to avoid important surface/subsurface structures. This bench-top smart laser knife system described here offers a new image-guided approach to precise stem cell seeding that can enhance the efficacy of articular cartilage repair.

A common representation of volumetric medical image data is the triplanar view (TV), in which the surgeon manually selects slices showing the anatomical structure of interest. In addition to common medical imaging such as MRI or computed tomography, recent advances in the field of optical coherence tomography (OCT) have enabled live processing and volumetric rendering of four-dimensional images of the human body. Due to the region of interest undergoing motion, it is challenging for the surgeon to simultaneously keep track of an object by continuously adjusting the TV to desired slices. To select these slices in subsequent frames automatically, it is necessary to track movements of the volume of interest (VOI). This has not been addressed with respect to 4DOCT images yet. Therefore, this paper evaluates motion tracking by applying state-of-the-art tracking schemes on maximum intensity projections (MIP) of 4D-OCT images. Estimated VOI location is used to conveniently show corresponding slices and to improve the MIPs by calculating thin-slab MIPs. Tracking performances are evaluated on an in-vivo sequence of human skin, captured at 26 volumes per second. Among investigated tracking schemes, our recently presented tracking scheme for soft tissue motion provides highest accuracy with an error of under 2.2 voxels for the first 80 volumes. Object tracking on 4D-OCT images enables its use for sub-epithelial tracking of microvessels for image-guidance.

Miniature infrared cameras have recently come to market in a form factor that facilitates packaging in endoscopic or other minimally invasive surgical instruments. If absolute temperature measurements can be made with these cameras, they may be useful for non-contact monitoring of electrocautery-based vessel sealing, or other thermal surgical processes like thermal ablation of tumors. As a first step in evaluating the feasibility of optical medical thermometry with these new cameras, in this paper we explore how well thermal measurements can be made with them. These cameras measure the raw flux of incoming IR radiation, and we perform a calibration procedure to map their readings to absolute temperature values in the range between 40 and 150 °C. Furthermore, we propose and validate a method to estimate the spatial extent of heat spread created by a cautery tool based on the thermal images.

Complex intravascular lesions, such as chronic total occlusions (CTOs), require forward-looking imaging. We propose to use a 25 MHz single-element transducer and an optical shape sensing (OSS) fiber integrated into a steerable catheter to achieve intravascular imaging in a forward-looking approach. A tissue-mimicking phantom with three hollow channels (3, 2 and 1 mm in diameter) and two steel spheres (1.5 mm in diameter) is used as imaging target. Ultrasound data and OSS data are simultaneously acquired while steering and rotating a 8.5 F catheter with bidirectional tip flexion. The obtained ultrasound data are reconstructed in 3D space using the position and direction information from the OSS data. Afterwards, the sparsely sampled ultrasound data are projected on a 2D plane and interpolated using normalized convolution (NC), which has been shown previously to perform well on irregularly sampled data [1]. The front surface of the phantom together with the location of two of the three channels and the two steel spheres are successfully reconstructed. The ability to reconstruct different components and their location in space is very important during CTOs crossing. This type of information can aid the crossing procedure providing insights about the best entry point, such as channels location, and helping in avoiding highly calcified areas, which usually are displayed in ultrasound imaging as highly scattering regions.

Ultrasound (US) imaging remains one of the most commonly used imaging modalities in medical practice. However, due to the physical effort required to perform US imaging tasks, 63-91% of ultrasonographers develop musculoskeletal disorders throughout their careers. The goal of this work is to provide ultrasonographers with a system that facilitates and reduces strain in US image acquisition. To this end, we propose a system for admittance force robot control that uses the six-degree-of-freedom UR5 industrial robot. A six-axis force sensor is used to measure the forces and torques applied by the sonographer on the probe. As the sonographer pushes against the US probe, the robot complies with these forces, following the user's desired path. A one-axis load cell is used to measure contact forces between the patient and the probe in real time. When imaging, the robot augments the axial forces applied by the user, lessening the physical effort required. User studies showed an overall decrease in hand tremor while imaging at high forces, improvements in image stability, and a decrease in difficulty and strenuousness.

We present a novel continuum robot design concept, Concentric Agonist-Antagonist Robots (CAAR), that uses push-pull, agonist-antagonist action of a pair of concentric tubes. The CAAR tubes are designed to have noncentral, offset neutral axes, and they are fixed together at their distal ends. Axial base translations then induce bending in the device. A CAAR segment can be created by selectively cutting asymmetric notches into the profile of two stock tubes, which relocates the neutral bending plane away from the center of the inner lumen. Like conventional concentric-tube robots (CTRs) based on counter-rotating precurved tubes, a CAAR can be made at very small scales and contain a large, open lumen. In contrast with CTRs, the CAAR concept has no elastic stability issues, offers a larger range of motion, and has lower overall stiffness. Furthermore, by varying the position of the neutral axes along the length of each tube, arbitrary, variable curvature actuation modes can be achieved. Precurving the tubes can additionally increase the workspace of a single segment. A single two-tube assembly can be used to create 3 degree-of-freedom (DOF) robot segments, and multiple segments can be deployed concentrically. Both additive manufacturing and traditional machining of stock tubes can create and customize the geometry and performance of the CAAR. In this paper, we explore the CAAR concept, provide kinematic and static models, and experimentally evaluate the model with a both a straight and a precurved CAAR. We conclude with a discussion of the significance and our plans for future work.

Ureteroscopy is a minimally invasive procedure for diagnosis and treatment of urinary tract pathology. Ergonomic and visualization challenges as well as radiation exposure are limitations to conventional ureteroscopy. Therefore, we have developed a robotic system to “power drive” a flexible ureteroscope with 3D tip tracking and pre-operative image overlay. The proposed system was evaluated using a kidney phantom registered to pre-operative MR images. Initial experiments show the potential of the device to provide additional assistance, precision, and guidance during urology procedures.

We consider the problem of finding automatically optimal positions of surgical lamps throughout the whole surgical procedure, where we assume that future lamps could be robotized. We propose a two-tiered optimization technique for the real-time autonomous positioning of those robotized surgical lamps. Typically, finding optimal positions for surgical lamps is a multi-dimensional problem with several, in part conflicting, objectives, such as optimal lighting conditions at every point in time while minimizing the movement of the lamps in order to avoid distractions of the surgeon. Consequently, we use multi-objective optimization (MOO) to find optimal positions in real-time during the entire surgery. Due to the conflicting objectives, there is usually not a single optimal solution for such kinds of problems, but a set of solutions that realizes a Pareto-front. When our algorithm selects a solution from this set it additionally has to consider the individual preferences of the surgeon. This is a highly non-trivial task because the relationship between the solution and the parameters is not obvious. We have developed a novel meta-optimization that considers exactly this challenge. It delivers an easy to understand set of presets for the parameters and allows a balance between the lamp movement and lamp obstruction. This metaoptimization can be pre-computed for different kinds of operations and it then used by our online optimization for the selection of the appropriate Pareto solution. Both optimization approaches use data obtained by a depth camera that captures the surgical site but also the environment around the operating table. We have evaluated our algorithms with data recorded during a real open abdominal surgery. It is available for use for scientific purposes. The results show that our meta-optimization produces viable parameter sets for different parts of an intervention even when trained on a small portion of it.

An intraluminal endoscopic approach is desirable for most colonoscopic procedures and is growing in favor for other surgeries as tools are enhanced. Flexible robotic manipulators could further enhance the dexterity and precision of commercial endoscopic systems. In this paper, we explore the capabilities of concentric tube robots to work as tool manipulators at the tip of a colonoscope to perform endoscopic submucousal dissection (ESD) and endoscopic full thickness resection (EFTR). We provide an overview of the kinematic modeling of these manipulators, a design of a prototype manipulator and the transmission actuation system. Our analysis examines the workspace and stiffness of these manipulators being controlled at the tip of a colonoscope. We compare the results to reported surgical requirements and propose solutions for enhancing their effectiveness including notching tubes with a larger Young’s Modulus. We also determine the resolution and accuracy of the actuation system.

PURPOSE: Patient-specific heart and valve models have shown promise as training and planning tools for heart surgery, but physically realistic valve models remain elusive. Available proprietary, simulation-focused heart valve models are generic adult mitral valves and do not allow for patient-specific modeling as may be needed for rare diseases such as congenitally abnormal valves. We propose creating silicone valve models from a 3D-printed plastic mold as a solution that can be adapted to any individual patient and heart valve at a fraction of the cost of direct 3D-printing using soft materials. METHODS: Leaflets of a pediatric mitral valve, a tricuspid valve in a patient with hypoplastic left heart syndrome, and a complete atrioventricular canal valve were segmented from ultrasound images. A custom software was developed to automatically generate molds for each valve based on the segmentation. These molds were 3D-printed and used to make silicone valve models. The models were designed with cylindrical rims of different sizes surrounding the leaflets, to show the outline of the valve and add rigidity. Pediatric cardiac surgeons practiced suturing on the models and evaluated them for use as surgical planning and training tools. RESULTS: Five out of six surgeons reported that the valve models would be very useful as training tools for cardiac surgery. In this first iteration of valve models, leaflets were felt to be unrealistically thick or stiff compared to real pediatric leaflets. A thin tube rim was preferred for valve flexibility. CONCLUSION: The valve models were well received and considered to be valuable and accessible tools for heart valve surgery training. Further improvements will be made based on surgeons’ feedback.

Significant mitral valve regurgitation affects over 2% of the population. Over the past few decades, mitral valve (MV) repair has become the preferred treatment option, producing better patient outcomes than MV replacement, but requiring more expertise. Recently, 3D printing has been used to assist surgeons in planning optimal treatments for complex surgery, thus increasing the experience of surgeons and the success of MV repairs. However, while commercially available 3D printers are capable of printing soft, tissue-like material, they cannot replicate the demanding combination of echogenicity, physical flexibility and strength of the mitral valve. In this work, we propose the use of trans-esophageal echocardiography (TEE) 3D image data and inexpensive 3D printing technology to create patient specific mitral valve models. Patient specific 3D TEE images were segmented and used to generate a profile of the mitral valve leaflets. This profile was 3D printed and integrated into a mold to generate a silicone valve model that was placed in a dynamic heart phantom. Our primary goal is to use silicone models to assess different repair options prior to surgery, in the hope of optimizing patient outcomes. As a corollary, a database of patient specific models can then be used as a trainer for new surgeons, using a beating heart simulator to assess success. The current work reports preliminary results, quantifying basic morphological properties. The models were assessed using 3D TEE images, as well as 2D and 3D Doppler images for comparison to the original patient TEE data.

Real-time three-dimensional transesophageal echocardiography (RT3D-TEE) is increasingly used during minimally invasive cardiac surgeries (MICS). In many cath labs, RT3D-TEE is already one of the requisite tools for image guidance during MICS. However, the visualization of the catheter is not always satisfactory making 3D- TEE challenging to use as the only modality for guidance. We propose a novel technique for better visualization of the catheter along with the cardiac anatomy using TEE alone - exploiting both beamforming and post processing methods. We extended our earlier method called Delay and Standard Deviation (DASD) beamforming to 3D in order to enhance specular reflections. The beam-formed image was further post-processed by the Frangi filter to segment the catheter. Multi-variate visualization techniques enabled us to render both the standard tissue and the DASD beam-formed image on a clinical ultrasound scanner simultaneously. A frame rate of 15 FPS was achieved.

Segmentation of right ventricle from cardiac MRI images can be used to build pre-operative anatomical heart models to precisely identify regions of interest during minimally invasive therapy. Furthermore, many functional parameters of right heart such as right ventricular volume, ejection fraction, myocardial mass and thickness can also be assessed from the segmented images. To obtain an accurate and computationally efficient segmentation of right ventricle from cardiac cine MRI, we propose a segmentation algorithm formulated as an energy minimization problem in a graph. Shape prior obtained by propagating label from an average atlas using affine registration is incorporated into the graph framework to overcome problems in ill-defined image regions. The optimal segmentation corresponding to the labeling with minimum energy configuration of the graph is obtained via graph-cuts and is iteratively refined to produce the final right ventricle blood pool segmentation. We quantitatively compare the segmentation results obtained from our algorithm to the provided gold-standard expert manual segmentation for 16 cine-MRI datasets available through the MICCAI 2012 Cardiac MR Right Ventricle Segmentation Challenge according to several similarity metrics, including Dice coefficient, Jaccard coefficient, Hausdorff distance, and Mean absolute distance error.

Minimally invasive cardiac procedures requiring a trans-septal puncture such as atrial ablation and MitraClip^® mitral valve repair are becoming increasingly common. These procedures are performed on the beating heart, and require clinicians to rely on image-guided techniques. For cases of complex or diseased anatomy, in which fluoroscopic and echocardiography images can be difficult to interpret, clinicians may benefit from patient-specific atrial models that can be used for training, surgical planning, and the validation of new devices and guidance techniques. Computed tomography (CT) images of a patient’s heart were segmented and used to generate geometric models to create a patient-specific atrial phantom. Using rapid prototyping, the geometric models were converted into physical representations and used to build a mold. The atria were then molded using tissue-mimicking materials and imaged using CT. The resulting images were segmented and used to generate a point cloud data set that could be registered to the original patient data. The absolute distance of the two point clouds was compared and evaluated to determine the model’s accuracy. The result when comparing the molded model point cloud to the original data set, resulted in a maximum Euclidean distance error of 4.5 mm, an average error of 0.5 mm and a standard deviation of 0.6 mm. Using our workflow for creating atrial models, potential complications, particularly for complex repairs, may be accounted for in pre-operative planning. The information gained by clinicians involved in planning and performing the procedure should lead to shorter procedural times and better outcomes for patients.

Permanent breast seed implantation (PBSI) is a single-visit technique for accelerated partial breast irradiation that uses a template and needles to implant seeds of Pd-103 under 2D ultrasound (US) guidance. The short treatment time is advantageous given the widely hypothesized link between treatment burden and mastectomy use. However, limitations of 2D US contribute to high operator dependence and seed placement error that we aim to address by developing a 3D US guidance system. A 3D US scanner for PBSI and a mechanism for template localization have been developed and validated. The 3D US system mechatronically moves and tracks a 2D US transducer over a 5 cm translation and 60° tilt, reconstructing the 2D images into a 3D volume as they are acquired. Additionally, a localizing arm, tracked via encoded joints and mounted to the scanner, determines template position by localizing divots on a modified needle template. Volume reconstruction was validated using linear measurements of a grid phantom and volumetric measurements of two surgical cavity phantoms. Localizing arm measurement accuracy was established using a testing jig with divots at known positions. Imaging volume was rigidly registered to scanner geometry using a string phantom mounted to a test jig. Lastly, volunteer scans were conducted to demonstrate clinical applicability. Median linear and average volumetric measurements were within ±1.4% of nominal and ±4.1% of water displacement measurements, respectively. Median measurement accuracy of the localizing arm was 0.475 mm. Imaging volume target registration error was 0.458 mm. Volunteer scans produced clinical quality images.

PURPOSE: Computer-navigated breast tumor excision using tracked ultrasound is a technique for performing lumpectomies during early-stage breast cancer. An interactive method is used to contour tumors intra-operatively for excision. We evaluated this method’s effectiveness in contouring the entire tumor with minimal inclusion of healthy tissue in the excision volume. Additionally, we investigated the possibility of adding a safety margin to the contoured volume to ensure that the entire tumor is contained by the contour. METHODS: We conducted a study in which 10 participants contoured 5 tumors each using the intra-operative breast tumor contouring system. We analyzed their interactions with the system and their opinions of the contouring workflow. RESULTS: We found that only 0.19% of the tumor volume was not contained by the contour on average. The addition of a 0.4 mm safety margin to the final tumor contour guaranteed that the entire tumor would be contained. We also found a correlation between the amount of time spent on contour verification and excess healthy tissue included in the contour. Users’ perceptions of how well they excluded excess healthy tissue strongly correlated with reality. CONCLUSIONS: This workflow ensures that only a small amount of tumor volume is not contained by the contour and allows the radiologist confidence that they have contained the entire tumor in their contour. With the addition of a safety margin to the resulting tumor contour, the tumor can be completely contained.

Recent studies have shown the value of Temporal Enhanced Ultrasound (TeUS) imaging for tissue characterization in transrectal ultrasound-guided prostate biopsies. Here, we present results of experiments designed to study the impact of temporal order of the data in TeUS signals. We assess the impact of variations in temporal order on the ability to automatically distinguish benign prostate-tissue from malignant tissue. We have previously used Hidden Markov Models (HMMs) to model TeUS data, as HMMs capture temporal order in time series. In the work presented here, we use HMMs to model malignant and benign tissues; the models are trained and tested on TeUS signals while introducing variation to their temporal order. We first model the signals in their original temporal order, followed by modeling the same signals under various time rearrangements. We compare the performance of these models for tissue characterization. Our results show that models trained over the original order-preserving signals perform statistically significantly better for distinguishing between malignant and benign tissues, than those trained on rearranged signals. The performance degrades as the amount of temporal-variation increases. Specifically, accuracy of tissue characterization decreases from 85% using models trained on original signals to 62% using models trained and tested on signals that are completely temporally-rearranged. These results indicate the importance of order in characterization of tissue malignancy from TeUS data.

Medical imaging from CT and MRI scans has become essential to clinicians for diagnosis, treatment planning and even prevention of a wide array of conditions. The presentation of image data volumes as 2D slice series provides some challenges with visualising internal structures. 3D reconstructions of organs and other tissue samples from data with low scan resolution leads to a ‘stepped’ appearance. This paper demonstrates how to improve 3D visualisation of features and automated preparation for 3D printing from such low resolution data, using novel techniques for morphing from one slice to the next. The boundary of the starting contour is grown until it matches the boundary of the ending contour by adapting a variant of the Fast Marching Method (FMM). Our spoke based approach generates scalar speed field for FMM by estimating distances to boundaries with line segments connecting the two boundaries. These can be regularly spaced radial spokes or spokes at radial extrema. We introduce clamped FMM by running the algorithm outwards from the smaller boundary and inwards from the larger boundary and combining the two runs to achieve FMM growth stability near the two region boundaries. Our method inserts a series of uniformly distributed intermediate contours between each pair of consecutive slices from the scan volume thus creating smoother feature boundaries. Whilst hard to quantify, our overall results give clinicians an evidently improved tangible and tactile representation of the tissues, that they can examine more easily and even handle.

During image-guided prostate biopsy, needles are targeted at suspicious tissues to obtain specimens that are later examined histologically for cancer. Patient motion causes inaccuracies when using MR-transrectal ultrasound (TRUS) image fusion approaches used to augment the conventional biopsy procedure. Motion compensation using a single, user initiated correction can be performed to temporarily compensate for prostate motion, but a real-time continuous registration offers an improvement to clinical workflow by reducing user interaction and procedure time. An automatic motion compensation method, approaching the frame rate of a TRUS-guided system, has been developed for use during fusion-based prostate biopsy to improve image guidance. 2D and 3D TRUS images of a prostate phantom were registered using an intensity based algorithm utilizing normalized cross-correlation and Powell’s method for optimization with user initiated and continuous registration techniques. The user initiated correction performed with observed computation times of 78 ± 35 ms, 74 ± 28 ms, and 113 ± 49 ms for in-plane, out-of-plane, and roll motions, respectively, corresponding to errors of 0.5 ± 0.5 mm, 1.5 ± 1.4 mm, and 1.5 ± 1.6°. The continuous correction performed significantly faster (p < 0.05) than the user initiated method, with observed computation times of 31 ± 4 ms, 32 ± 4 ms, and 31 ± 6 ms for in-plane, out-of-plane, and roll motions, respectively, corresponding to errors of 0.2 ± 0.2 mm, 0.6 ± 0.5 mm, and 0.8 ± 0.4°.

PURPOSE: Collision detection for external beam radiation therapy (RT) is important for eliminating the need for dryruns that aim to ensure patient safety. Commercial treatment planning systems (TPS) offer this feature but they are expensive and proprietary. Cobalt-60 RT machines are a viable solution to RT practice in low-budget scenarios. However, such clinics are hesitant to invest in these machines due to a lack of affordable treatment planning software. We propose the creation of an open-source room’s eye view visualization module with automated collision detection as part of the development of an open-source TPS. METHODS: An openly accessible linac 3D geometry model is sliced into the different components of the treatment machine. The model’s movements are based on the International Electrotechnical Commission standard. Automated collision detection is implemented between the treatment machine’s components. RESULTS: The room’s eye view module was built in C++ as part of SlicerRT, an RT research toolkit built on 3D Slicer. The module was tested using head and neck and prostate RT plans. These tests verified that the module accurately modeled the movements of the treatment machine and radiation beam. Automated collision detection was verified using tests where geometric parameters of the machine’s components were changed, demonstrating accurate collision detection. CONCLUSION: Room’s eye view visualization and automated collision detection are essential in a Cobalt-60 treatment planning system. Development of these features will advance the creation of an open-source TPS that will potentially help increase the feasibility of adopting Cobalt-60 RT.

Fluoroscopic image guidance for minimally invasive procedures in the thorax and abdomen suffers from respiratory and cardiac motion, which can cause severe subtraction artifacts and inaccurate image guidance. This work proposes novel techniques for respiratory motion tracking in native fluoroscopic images as well as a model based estimation of vessel deformation. This would allow compensation for respiratory motion during the procedure and therefore simplify the workflow for minimally invasive procedures such as liver embolization. The method first establishes dynamic motion models for both the contrast-enhanced vasculature and curvilinear background features based on a native (non-contrast) and a contrast-enhanced image sequence acquired prior to device manipulation, under free breathing conditions. The model of vascular motion is generated by applying the diffeomorphic demons algorithm to an automatic segmentation of the subtraction sequence. The model of curvilinear background features is based on feature tracking in the native sequence. The two models establish the relationship between the respiratory state, which is inferred from curvilinear background features, and the vascular morphology during that same respiratory state. During subsequent fluoroscopy, curvilinear feature detection is applied to determine the appropriate vessel mask to display. The result is a dynamic motioncompensated vessel mask superimposed on the fluoroscopic image. Quantitative evaluation of the proposed methods was performed using a digital 4D CT-phantom (XCAT), which provides realistic human anatomy including sophisticated respiratory and cardiac motion models. Four groups of datasets were generated, where different parameters (cycle length, maximum diaphragm motion and maximum chest expansion) were modified within each image sequence. Each group contains 4 datasets consisting of the initial native and contrast enhanced sequences as well as a sequence, where the respiratory motion is tracked. The respiratory motion tracking error was between 1.00 % and 1.09 %. The estimated dynamic vessel masks yielded a Sørensen-Dice coefficient between 0.94 and 0.96. Finally, the accuracy of the vessel contours was measured in terms of the 99th percentile of the error, which ranged between 0.64 and 0.96 mm. The presented results show that the approach is feasible for respiratory motion tracking and compensation and could therefore considerably improve the workflow of minimally invasive procedures in the thorax and abdomen

Intraoperative mobile C-arm fluoroscopy is widely used for interventional verification in trauma surgery, high flexibility combined with low cost being the main advantages of the method. However, the lack of global device-to- patient orientation is challenging, when comparing the acquired data to other intrapatient datasets. In upper ankle joint fracture reduction accompanied with an unstable syndesmosis, a comparison to the unfractured contralateral site is helpful for verification of the reduction result. To reduce dose and operation time, our approach aims at the comparison of single projections of the unfractured ankle with volumetric images of the reduced fracture. For precise assessment, a pre-alignment of both datasets is a crucial step. We propose a contour extraction pipeline to estimate the joint space location for a prealignment of fluoroscopic C-arm projections containing the upper ankle joint. A quadtree-based hierarchical variance comparison extracts potential feature points and a Hough transform is applied to identify bone shaft lines together with the tibiotalar joint space. By using this information we can define the coarse orientation of the projections independent from the ankle pose during acquisition in order to align those images to the volume of the fractured ankle. The proposed method was evaluated on thirteen cadaveric datasets consisting of 100 projections each with manually adjusted image planes by three trauma surgeons. The results show that the method can be used to detect the joint space orientation. The correlation between angle deviation and anatomical projection direction gives valuable input on the acquisition direction for future clinical experiments.

All medical image segmentation algorithms need to be validated and compared, and yet no evaluation framework is widely accepted within the imaging community. Collections of segmentation results often need to be compared and ranked by their effectiveness. Evaluation measures which are popular in the literature are based on region overlap or boundary distance. None of these are consistent in the way they rank segmentation results: they tend to be sensitive to one or another type of segmentation error (size, location, shape) but no single measure covers all error types. We introduce a new family of measures, with hybrid characteristics. These measures quantify similarity/difference of segmented regions by considering their overlap around the region boundaries. This family is more sensitive than other measures in the literature to combinations of segmentation error types. We compare measure performance on collections of segmentation results sourced from carefully compiled 2D synthetic data, and also on 3D medical image volumes. We show that our new measure: (1) penalises errors successfully, especially those around region boundaries; (2) gives a low similarity score when existing measures disagree, thus avoiding overly inflated scores; and (3) scores segmentation results over a wider range of values. We consider a representative measure from this family and the effect of its only free parameter on error sensitivity, typical value range, and running time.

Deep learning models have outperformed some of the previous state-of-the-art approaches in medical image analysis. Instead of using hand-engineered features, deep models attempt to automatically extract hierarchical representations at multiple levels of abstraction from the data. Therefore, deep models are usually considered to be more flexible and robust solutions for image analysis problems compared to conventional computer vision models. They have demonstrated significant improvements in computer-aided diagnosis and automatic medical image analysis applied to such tasks as image segmentation, classification and registration. However, deploying deep learning models often has a steep learning curve and requires detailed knowledge of various software packages. Thus, many deep models have not been integrated into the clinical research work ows causing a gap between the state-of-the-art machine learning in medical applications and evaluation in clinical research procedures. In this paper, we propose "DeepInfer" - an open-source toolkit for developing and deploying deep learning models within the 3D Slicer medical image analysis platform. Utilizing a repository of task-specific models, DeepInfer allows clinical researchers and biomedical engineers to deploy a trained model selected from the public registry, and apply it to new data without the need for software development or configuration. As two practical use cases, we demonstrate the application of DeepInfer in prostate segmentation for targeted MRI-guided biopsy and identification of the target plane in 3D ultrasound for spinal injections.