Compensation for intraoperative tissue motion in the registration of preoperative image volumes with the OR is important for improving the utility of image guidance in the neurosurgery setting. Model-based strategies for neuroimage compensation are appealing because they offer the prospect of retaining the high-resolution preoperative information without the expense and complexity associated with full volume intraoperative scanning. Further, they present opportunities to integrate incomplete or sparse, partial volume sampling of the surgical field as a guide for full volume estimation and subsequent compensation of the preoperative images. While potentially promising, there are a number of unresolved difficulties associated with deploying computational models for this purpose. For example, to date they have only been successful in representing the tissue motion that occurs during the earliest stages of neurosurgical intervention and have not addressed the later more complex events of tissue retraction and resection. IN this paper, we develop a mathematical framework for implementing retraction and resection within the context of finite element modeling of brain deformation using the equations of linear consolidation. Specifically, we discuss the critical boundary conditions applied at the new tissue surfaces created by these respective interventions and demonstrate the ability to model compound events where updated image volumes are generated in succession to represent the significant occurrences of tissue deformation which take place during the course of surgery. In this regard, we show image compensation for an actual OR case involving the implantation of a subdural electrode array for recording neural activity.

The spline-based Mathematical Cardiac Torso (MCAT) phantom is a realistic software simulation designed to simulate single photon emission computed tomographic (SPECT) data. It incorporates a heart model of known size and shape; thus, it is invaluable for measuring accuracy of acquisition, reconstruction, and post-processing routines. New functionality has been added by replacing the standard heart model with left ventricular (LV) epicaridal and endocardial surface points detected from actual patient SPECT perfusion studies. LV surfaces detected from standard post-processing quantitation programs are converted through interpolation in space and time into new B-spline models. Perfusion abnormalities are added to the model based on results of standard perfusion quantification. The new LV is translated and rotated to fit within existing atria and right ventricular models, which are scaled based on the size of the LV. Simulations were created for five different patients with myocardial infractions who had undergone SPECT perfusion imaging. Shape, size, and motion of the resulting activity map were compared visually to the original SPECT images. In all cases, size, shape and motion of simulated LVs matched well with the original images. Thus, realistic simulations with known physiologic and functional parameters can be created for evaluating efficacy of processing algorithms.

Physically based soft tissue modeling is a state of the art in computer assisted surgery (CAS). But even such a sophisticated approach has its limits. The biomechanic behavior of soft tissue is highly complex, so that simplified models have to be applied. Under assumption of small deformations, usually applied in soft tissue modeling, soft tissue can be approximately described as a linear elastic continuum. Since there exist efficient techniques for solving linear partial differential equations, the linear elastic model allows comparatively fast calculation of soft tissue deformation and consequently the prediction of a patient's postoperative appearance. However, for the calculation of large deformations, which are not unusual in craniofacial surgery, this approach can implicate substantial error depending on the intensity of the deformation. The monitoring of the linearization error could help to estimate the scope of validity of calculations upon user defined precision. In order to quantify this error one even do not need to know the correct solution, since the linear theory implies the appropriate instruments for error detection in itself.

A new surgical simulator is developed for spine needle biopsy, that provides realistic visual and force feedback to a trainee in the PC environment. This system is composed of four parts: a 3D human model, visual feedback tool, force feedback device, and an evaluation section. The 3D human model includes multi-slice XCT images, segmentation results, and force-feedback parameters. A block-based technique is adopted for efficient handling of large amounts of data and for easy control of rendering parameters such as opacity. For visual feedback, we implement a virtual CT console box and a 3D visualization tool providing MIP, MPR, summed voxel projection, and realistic 3D color volume rendering view. The visualization tool is for interactive 3D path planning. A haptic device is used to provide force feedback to the biopsy needle during simulation. The interactive force is generated in a voxel-based manner. After each simulation, the evaluation section provides a performance analysis to the trainee. We implemented the system by attaching a 3DOF PHANToM^TM device to a PC with 600MHz Pentium III Dual CPUs and 512Mbyte RAM.

3D Trans-Urethral Ultrasound (TUUS) imaging is a new imaging technique for the diagnosis and treatment of prostate disease. Our current research focuses on the potential of TUUS in therapy guidance during tansperineal interstitial permanent prostate brachytherapy (TIPPB). TUUS may complement of potentially replace x-ray fluoroscopy and TRUS in providing data for determining the prostate boundary and radiation source locations. Prostate boundary detection and source localization using TUUS were tested on an ultrasound- equivalent prostate phantom and ina patient during TIPPB. Data collection was conducted with a 10 French, 10 MHz ultrasound catheter controlled by an Acuson Sequoia^TM workstation. 2D and 3D TUUS scans were acquired after radioactive seeds were placed in the phantom and in the patient. Data was reconstructed, processed, and analyzed using Analyze software. Segmentation of the prostate boundary was performed semi-automatically, and seed segmentation was performed manually. Image artifacts in TUUS data resulted in incorrect reconstruction of the seeds. Intelligent processing of the seed data improved reconstruction. Comparison to the CT data suggests that TUUS dat provides: 1) greater spatial resolution, 2) greater temporal resolution and 3) better contrast for soft tissue differentiation. The reconstructed source sizes and locations were measured and found accurate. Placement of the TUUS catheter into the urethra provides excellent 2D sections which can be used to acquire volumetric data for 3D analysis of the prostate and radioactive sources. Preliminary results suggest that TUUS will be useful for guidance of seed placement, post-implant seed localization, and intra-operative dosimetry.

We are investigating methods to register live-time interventional magnetic resonance imaging (iMRI) slice images with a previously obtained, high resolution MRI image volume. The immediate application is for iMRI-guided treatments of prostate cancer. We created and evaluated a slice-to-volume mutual information registration algorithm for MR images with special features to improve robustness. Features included a multi-resolution approach and automatic restarting to avoid local minima. We acquired 3D volume images from a 1.5 T MRI system and simulated iMRI images. To assess the quality of registration, we calculated 3D displacement on a voxel-by-voxel basis over a volume of interest between slice-to-volume registration and volume-to- volume registrations that were previously shown to be quite accurate. More than 500 registration experiments were performed on MR images of volunteers. The slice-to-volume registration algorithm was very robust for transverse images covering the prostate. A 100% success rate was achieved with an acceptance criterion of <1.0 mm displacement error over the prostate. Our automatic slice-to-volume mutual information registration algorithm is robust and probably sufficiently accurate to aid in the application of iMRI- guided thermal ablation of prostate cancer.

Radiation therapy of prostate cancer requires a high geometric accuracy of dose delivery. Unfortunately, this accuracy can be hardly obtained because of imperfect patients repositioning throughout the preparation and execution of the treatment. In order to correct for such errors we developed methods for online registration of portal images (PI) and digitally reconstructed radiographs (DRR). To improve the preconditions for registration of the Pis and DRRs image preprocessing, such as contrast-limited adaptive histogram equalization (CLAHE) of the Pis, noise and edge filtering of the Pis and the DRRs, has been performed. Then, similarity measures as mean square error, cross correlation and mutual information were investigated. A fully automatic bony-based registration method has been developed. As a reference method a semi-automatic stereometric fiducial-based registration has also been developed. This method uses gold seeds implanted into the prostate as internal fiducial markers for assessment of the similarity measures. Small lead spheres as well were adhered as external fiducial markers on the Alderson phantom. This method was also used for the verification of the automatic bony-based registration. The registration of repositioning was then performed fully automatically in less than 30 s with the Alderson phantom and with the patient data. At the present stage, there remains a registration error of less than 2 mm.

A prototype high definition multi-leaf collimator system (HDI) has been developed and installed on the linear accelerator for use in conformal radiotherapy. The HDI technique utilizes the dynamic shift of the 3D-target volume to feather the multi-leaf collimator defined field edges. During each feathering, the leaf positions are adjusted according to the updated target image projected into the MLC plane. The purpose of this study is to demonstrate that this device can improve spatial resolution of a conformal radiation therapy treatment. The results of this study indicate that the HDI technique can be a useful tool for treating small, or highly irregular shaped targets, and for sparing adjacent critical structures for certain cases.

The purpose of the work was to develop image functions for volumetric segmentation, feature extraction, and enhanced 3D visualization of local regions using CT datasets of human lungs. The system is aimed to assist the radiologist in the analysis of lung nodules. Volumetric datasets consisting of 30-50 thoracic helical low-dose CT slices were used in the study. The 3D topological characteristics of local structures including bronchi, blood vessels, and nodules were computed and evaluated. When a location of a region of interest is identified, the computer would automatically compute size, surface of the area, and normalized shape index of the suspected lesion. The developed system can also allow the user to perform interactive operation for evaluation of lung regions and structures through a user- friendly interface. These functions provide the user with a powerful tool to observe and investigate clinically interesting regions through unconventional radiographic viewings and analyses. The developed functions can also be used to view and analyze patient's lung abnormalities in surgical planning applications. Additionally, we see the possibility of using the system as a teaching tool for correlating anatomy of lungs.

Purpose: Phantom trials using the PET data for localization of hot spots have demonstrated positional accuracies in the millimeter range. We wanted to perform biopsy based on information from both anatomic and functional imaging modalities, however we had a communication challenge. Despite the digital nature of DSM stereotactic X-ray mammography devices, and the large number of such devices in Radiology Departments (approximately 1600 in the US alone), we are not aware of any methods of connecting stereo units to other computers in the Radiology department. Methods: We implemented a local network between an external IBM PC (running Linux) and the Lorad Stereotactic Digital Spot Mammography PC (running DOS). The application used IP protocol on the parallel port, and could be run in the background on the LORAD PC without disrupting important clinical activities such as image acquisition or archiving. With this software application, users of the external PC could pull x-ray images on demand form the Load DSM computer. Results: X-ray images took about a minute to ship to the external PC for analysis or forwarding to other computers on the University's network. Using image fusion techniques we were able to designate locations of functional imaging features as the additional targets on the anatomic x-rays. These pseudo-features could then potentially be used to guide biopsy using the stereotactic gun stage on the Lorad camera. New Work to be Presented: A method of transferring and processing stereotactic x-ray mammography images to a functional PET workstation for implementing image-guided biopsy.

A statistical classification algorithm, for MRI segmentation, based on the k Nearest Neighbor rule (kNN) has been implemented with Message Passing Interface (MPI) by partitioning the dataset into similar sized subvolumes and delivering each part to one processor inside a cluster. We have tested the algorithm in two different CPU architectures (SPARC and Intel) and four different configurations including a Beowulf cluster, two Sun clusters and a symmetric multiprocessor. The experiments provide a good speedup in all the cases and show a very good performance/price ratio in the PC-Linux cluster. We present results using a three channel, high resolution original dataset in times less than two minutes in the best cases and we use the segmented maps to make clinically relevant 3D visualizations in interactive times.

Jaw motion has been used to diagnose jaw pain patients, and we have developed a 3D computer animation technique to study jaw motion. A customized dental clutch was worn during motion, and its consistent and rigid placement was a concern. The experimental protocol involved mandibular movements (vertical opening) and MR imaging. The clutch contained three motion markers used to collect kinematic data and four MR markers used as fiducial markers in the MR images. Fiducial marker misplacement was mimicked by analytically perturbing the position of the MR markers +/- 2, +/- 4, and +/- 6 degrees in the three anatomical planes. The percent difference between the original and perturbed MR marker position was calculated for kinematic parameters. The maximum difference across all perturbations for axial rotation, coronal rotation, sagittal rotation, axial translation, coronal translation, and sagittal translation were 176.85%, 191.84%, 0.64%, 9.76%, 80.75%, and 8.30%, respectively, for perturbing all MR markers, and 86.47%, 93.44%, 0.23%, 7.08%, 42.64%, and 13.64%, respectively, for perturbing one MR marker. The parameters representing movement in the sagittal plane, the dominant plane in vertical opening, were determined to be reasonably robust, while secondary movements in the axial and coronal planes were not considered robust.

A single camera optical tracking system has been implemented to track the freehand motion of an object in 3D. The system consists of a single standard NTSC video camera and a frame grabber that is utilized to digitize the image of a target plate with four non-collinear fiducial marks. The tracking algorithm is based on texture mapping to determine the rotation and translation of the plate (and hence object) from the images of four fiducial marks. Utilizing a SGI VW320 workstation (with integrated frame grabber) and a CCD video camera, tracking accuracy of better than 0.1 mm for object velocity of 4.0 cm/s was achievable at full video frame rate for all directions.

A robust near real-time MRI based surgical guidance and navigation scheme has been developed, validated and used. The key concept of the method is to use intra-operative MRI to facilitate the trajectory alignment process of a biopsy needle in neurobiopsy. Since the trajectory corresponding to the biopsy needle pivoted at an entry point on patient skull has two degrees of freedom, the orientation of the needle can be tracked using a 2D imaging plane placed perpendicular to the desired trajectory. Using a near real- time visual feedback during the adjustment of an alignment guidance device, the required trajectory alignment was translated into a simple in-plane targeting task on computer monitor. The orientation adjustment was achieved remotely via a set of MR-compatible strings, which were connected to a joystick. The concept of MR-guided targeting was successfully validated on a phantom set-up. This MR based guidance technique has practically allowed neurosurgeons to accomplish the required needle alignment to an arbitrary trajectory remotely in a straight forward procedure on any conventional MR scanner. Before needle insertion, the trajectory can be validated. Two successful biopsy cases using the new methodology and device have shown that the remotely operated device under MR-guidance is both effective and accurate for neurosurgery.

During a typical image-guided neurosurgery procedure, the surgeon used anatomical information from tomographic image sets to help guide the surgery. These images provide high- level details of the patient's anatomy. The images do not, however, provide the surgeon with information regarding brain function. The identification of cortical function in addition to the display of tomographic images during surgery would allow the surgeon to visualize critical areas of the anatomy. This would be beneficial during surgical planning and procedures by identifying eloquent cortical regions (such as speech, sensory, and motor areas) that should be avoided. We have designed and implemented a system for recording and displaying cortical brain function during image-guided surgery. Brain function is determined using an optically tracked cortical stimulator. The image-space location of each stimulation event is recorded, and the user has the ability to label this location according to function type. Functional data can be displayed on both tomographic and rendered images. Tracking accuracy of the cortical stimulator has been determined by comparing its position to that of a tracked surgical probe with known localizing accuracy.

We introduce a mechanically constrained, 3D ultrasound- guided core-needle breast biopsy device. With modest breast compression, 3D ultrasound scans localize suspicious masses. A biopsy needle is mechanically guided into position for firing into the sampling region. Th needle is parallel to the transducer, allowing real-time guidance during needle insertion. Lesion sampling is verified by another ultrasound image after firing. Two procedures quantified targeting accuracy of this apparatus. First, we biopsied eleven breast phantoms containing 123 embedded, cylindrical lesions constructed from PVA-C (poly(vinyl alcohol) cryogel) with diameters ranging from 1.6 to 15.9mm. Identification of the colored lesion in the biopsy sample and analysis of the post-biopsy US images provided a model for the success rates. Using this, we predict that our apparatus will require six passes to biopsy a 3.0 mm lesion with 99% confidence. For the second experiment, agar phantoms were embedded with four rows of 0.8mm stainless steel beads. A 14-gauge needle was inserted to each bead position seen in a 3D ultrasound scan and the tip position was compared to the pre-insertion bead position. The inter-observer standard errors of measurement were less than 0.15 and 0.28mm for the bead and needle tip positions, respectively. The off-axis 3D 95% confidence intervals were determined to have widths between 0.43 and 1.71mm, depending on direction and bead position.

The purpose of the study was to evaluate a computer aided diagnosis (CAD) workstation with automatic detection of pulmonary nodules at low-dose spiral CT in a clinical setting for early detection of lung cancer. Two radiologists in consensus reported 88 consecutive spiral CT examinations. All examinations were reviewed using a UNIX-based CAD workstation with a self-developed algorithm for automatic detection of pulmonary nodules. The algorithm was designed to detect nodules with at least 5 mm diameter. The results of automatic nodule detection were compared to the consensus reporting of two radiologists as gold standard. Additional CAD findings were regarded as nodules initially missed by the radiologists or as false positive results. A total of 153 nodules were detected with all modalities (diameter: 85 nodules <5mm, 63 nodules 5-9 mm, 5 nodules >= 10 mm). Reasons for failure of automatic nodule detection were assessed. Sensitivity of radiologists for nodules >=5 mm was 85%, sensitivity of CAD was 38%. For nodules >=5 mm without pleural contact sensitivity was 84% for radiologists at 45% for CAD. CAD detected 15 (10%) nodules not mentioned in the radiologist's report but representing real nodules, among them 10 (15%) nodules with a diameter $GREW5 mm. Reasons for nodules missed by CAD include: exclusion because of morphological features during region analysis (33%), nodule density below the detection threshold (26%), pleural contact (33%), segmentation errors (5%) and other reasons (2%). CAD improves detection of pulmonary nodules at spiral CT significantly and is a valuable second opinion in a clinical setting for lung cancer screening. Optimization of region analysis and an appropriate density threshold have a potential for further improvement of automatic nodule detection.

Our goal was to perform a pre-clinical test of the performance of a new pre-commercial system for detection of primary early-stage lung cancer on chest radiographs developed by Deus Technologies, LLC. The RapidScreen^TM RS 2000 System integrates state of the art technical development in this field.

Magnetic Resonance (MR) tagging techniques have recently received much attention for the generation of suitable data sets for cardiac motion analysis. Several different techniques have been developed for the analysis of 3D cardiac wall motion from such data sets. Unfortunately, due to the lack of gold standard test data sets, most of the techniques are not fully evaluated. We have developed a virtual MR tagging system for evaluation of such 3D motion estimation techniques. It generates virtual tagging images, in particular SPAMM images, of the deformable model that undergoes a predetermined motion sequence in a virtual environment. The input to the system is a set of vertices that characterizes the shape of the volumetric model at each time phase. After the user interactively specifies the imaging and tagging parameters, the system automatically marks the intersections between the image and tagging planes within the model tissue to track its motion, computing the deformed tagging lines over the time sequence. The output is the set of tomographic images of the virtual object with SPAMM grid patterns reflecting the known 3D formation field, and it can be used to evaluate an arbitrary cardiac motion estimation technique.

Patient-to-image misalignment becomes exacerbated by common surgical events such as brain sag, drug interactions, retraction, and resection. One strategy to remedy this mis- registration is to employ computational models in conjunction with low-cost intraoperatively-acquired data (e.g. surface tracking, and co-registered ultrasound) to deform preoperative imaging data to account for OR actions. In this paper, we present preliminary data from a cortical surface scanning system and study the impact of surface- based information on model-updates. Preliminary data is presented using a 3D laser scanning technology in conjunction with an iterative closest point (ICP) algorithm to register and track phantom and ex vivo data. Simulations are presented to analyze the direct use of displacement data versus modeling the underlying physical load in a clinical example of gravity-induced deformation. Results demonstrate dramatic differences in subsurface deformation fields highlighting that the nature of the surgical load (i.e. surface or body force) must be thoughtfully discriminated to accurately update images. Furthermore, the results suggest that the application of surface displacements to update image volumes must be consistent with the physical origin of deformation rather than applied in a direct interpolative sense.

Since the spine is one of the most complex joint structures in the human body, its surgical treatment requires careful planning and high degree of precision to avoid any unwanted neurological compromises. In addition, comprehensive biomechanical analysis can be very helpful because the spine is subject to a variety of load. In case for the osteoporotic spine in which the structural integrity has been compromised, it brings out the double challenges for a surgeon both clinically and biomechanically. Thus, we have been developing an integrated medical image system that is capable of doing the both. This system is called orthopedic surgical analyzer and it combines the clinical results from image-guided examination and the biomechanical data from finite element analysis. In order to demonstrate its feasibility, this system was applied to percutaneous vertebroplasty. Percutaneous vertebroplasty is a surgical procedure that has been recently introduced for the treatment of compression fracture of the osteoporotic vertebrae. It involves puncturing vertebrae and filling with polymethylmethacrylate (PMMA). Recent studies have shown that the procedure could provide structural reinforcement for the osteoporotic vertebrae while being minimally invasive and safe with immediate pain relief. However, treatment failures due to excessive PMMA volume injection have been reported as one of complications. It is believed that control of PMMA volume is one of the most critical factors that can reduce the incidence of complications. Since the degree of the osteoporosis can influence the porosity of the cancellous bone in the vertebral body, the injection volume can be different from patient to patient. In this study, the optimal volume of PMMA injection for vertebroplasty was predicted based on the image analysis of a given patient. In addition, biomechanical effects due to the changes in PMMA volume and bone mineral density (BMD) level were investigated by constructing clinically relevant finite element models. In conclusion, we were able to demonstrate the feasibility of our orthopedic surgical analyzer in a case for percutaneous vertebroplasty.

Spinal surgery is often necessary to ease back pain symptoms. Neuronavigation (NN) allows the surgeon to localize the position of his instruments in 3D using pre- operative CT scans registered to intra-operative marker positions in cranial surgeries. However, this tool is unavailable in spinal surgeries for a variety of reasons. For example, because of the spine's many degrees of freedom and flexibility, the geometric relationship of the skin to the internal spinal anatomy is not fixed. Guided by the currently available imperfect 2D images, it is difficult for the surgeon to correct a patient's spinal anomaly; thus surgical relief of back pain is often only temporary. The Image Guidance Laborator's (IGL) goal is to combine the direct optical control of traditional endoscopy with the 3D orientation of NN. This powerful tool requires registration of the patient's anatomy to the surgical navigation system using internal landmarks rather than skin markers. Pre- operative CT scans matched with intraoperative fluoroscopic images can overcome the problem of spinal movement in NN registration. The combination of endoscopy with fluoroscopic registration of vertebral bodies in a NN system provides a 3D intra-operative navigational system for spinal neurosurgery to visualize the internal surgical environment from any orientation in real time. The accuracy of this system integration is being evaluated by assessing the success of nucleotomies and marker implantations guided by NN-registered endoscopy.

A new method for stenosis quantification from 3D X-ray angiography images has been evaluated on both phantom and clinical data. On phantoms, for the parts larger or equal to 3 mm, the standard deviation of the measurement error has always found to be less or equal to 0.4 mm, and the maximum measurement error less than 0.17 mm. No clear relationship has been observed between the performances of the quantification method and the acquisition FoV. On clinical data, the 3D quantification method proved to be more robust to vessel bifurcations than its 3D equivalent. On a total of 15 clinical cases, the differences between 2D and 3D quantification were always less than 0.7 mm. The conclusion is that stenosis quantification from 3D X-4ay angiography images is an attractive alternative to quantification from 2D X-ray images.

Our goal is to improve the visualization of the intracranial vasculature during interventional procedures where radiographically dense objects would normally hinder clinical assessment. We describe our technique of Volumetric Subtraction Angiography (VSA), which removes bone, metal objects, and associated artifacts from the 3D contrast-enhanced image of the patient's vasculature. This work utilizes a prototype Computed Rotational Angiography (CRA) system that uses a C-arm mounted x-ray image intensifier to acquire 2D projections of the vasculature. A modified cone-beam computed tomography (CT) algorithm is then used to reconstruct a 3D image with isotropic voxels. Two volumes of data are acquired, and the anatomical mask is volumetrically subtracted (voxel-by-voxel) from the intra- arterial, contrast-enhanced image, producing a 3Dangiogram.

We are treating tumors using radiofrequency (RF) ablation under interventional MRI (iMRI) guidance. We investigated the ability of MR to monitor the treated region by comparing MR thermal lesion images to cellular damage as seen histologically. Our new methodology allows 3D registration that should enable more accurate correlation than previous 2D methods. Using a low-field (0.2T) open magnet iMRI system for probe guidance, we applied RF ablation to the thigh muscle of four New Zealand White rabbits. To relate in vivo MR and histology images, we obtained intermediate ex vivo MR images and pictures of thick tissue slices obtained using a specially designed apparatus. Registration was done with a computer algorithm that matches tracks of needle fiducials placed near the tissue of interest. After registration, we determined the region inside the circular, hyperintense rim in MR closely corresponds to the region of necrosis as determined by histology on animals sacrificed 30 minutes after ablation. This is good evidence that iMRI images can be used for real-time feedback during thermal RF ablation treatments.

Integration of tomographic angiograms into neurosurgical navigation should decrease the probability of vascular injury and allow localization of vascular lesions. Information from angiograms is often presented using maximum intensity projections (MIPs), which provide a more intuitive presentation of 3D vascular structures. Conventional MIPs involve the whole image volume during ray casting. Our goal was to construct surgically appropriate MIPs that excluded information contralateral to the operation site and to quantify the accuracy of vessel depiction using this new method. For each angiogram slice, the center of mass (COM) was calculated. Together, the COM coordinates formed a boundary plane that clipped the contralateral information from ray casting. A separate depth buffer was created to preserve 3D information. MIPs were examined quantitatively using a mathematical model of the head containing vascular structures of known diameter. The vessel widths of the resulting MIPs were then measured and compared. To examine the spatial accuracy of MIP images, a vascular phantom was created, which had rigid vessels of known diameter and extrinsic fiducial markers to perform a physical to image space registration. Studies with the mathematical model showed that the vessels appeared smaller in MIPs than their actual diameters. This decrease is attributed to the statistical properties of the ray casting process that are affected by the pathlength. Studies with the vascular phantom show correct localization of the probe in tomographic and projective image space. From these studies, we concluded that additional methods for providing information concerning vessel proximity during surgical guidance should be investigated. Surgically appropriate MIPs provide comparable images to conventional MIPs; however, they allow more focus on the vascular structures in proximity to the target site.

Image-guided surgery and therapy is assuming an increasingly important role, particularly considering the current emphasis on minimally-invasive surgical procedures. Volumetric CT and MR images have been used now for some time in conjunction with stereotactic frames, to guide many neurosurgical procedures. With the development of systems that permit surgical instruments to be tracked in space, image-guided surgery now includes the use of frame-less procedures, and the application of the technology has spread beyond neurosurgery to include orthopedic applications and therapy of various soft-tissue organs such as the breast, prostate and heart. Since tracking systems allow image- guided surgery to be undertaken without frames, a great deal of effort has been spent on image-to-image and image-to- patient registration techniques, and upon the means of combining real-time intra-operative images with images acquired pre-operatively. As image-guided surgery systems have become increasingly sophisticated, the greatest challenges to their successful adoption in the operating room of the future relate to the interface between the user and the system. To date, little effort has been expended to ensure that the human factors issues relating to the use of such equipment in the operating room have been adequately addressed. Such systems will only be employed routinely in the OR when they are designed to be intuitive, unobtrusive, and provide simple access to the source of the images.

As a demonstration platform, we are developing a robotic biopsy testbed incorporating a mobile CT scanner, a small needle driver robot, and an optical localizer. This testbed will be used to compare robotically assisted biopsy to the current manual technique, and allow us to investigate software architectures for integrating multiple medical devices. This is a collaboration between engineers and physicians from three universities and a commercial vendor. In this paper we describe the CT-directed biopsy technique, review some other biopsy systems including passive and semi- autonomous devices, describe our testbed components, and present our software architecture. This testbed is a first step in developing the image-guided, robotically assisted, physician directed, biopsy systems of the future.

Methods of visualizing the inner colonic wall by using CT images has actively been pursued in recent years in an attempt to eventually replace conventional colonoscopic examination. In spite of impressive progress in this direction, there are still several problems, which need satisfactory solutions. Among these, we address three problems in this paper: segmentation, coverage, and speed of rendering. Instead of thresholding, we utilize the fuzzy connectedness framework to segment the colonic wall. Instead of the endoscopic viewing mode and various mapping techniques, we utilize the central line through the colon to generate automatically viewing directions that are enface with respect to the colon wall, thereby avoiding blind spots in viewing. We utilize some modifications of the ultra fast shell rendering framework to ensure fast rendering speed. The combined effect of these developments is that a colon study requires an initial 5 minutes of operator time plus an additional 5 minutes of computational time and subsequently enface renditions are created in real time (15 frames/sec) on a 1 GHz Pentium PC under the Linux operating system.

Autologous grafts serve as the standard grafting material in the treatment of maxillofacial bone tumors, traumatic defects or congenital malformations. The pre-selection of a donor site depends primarily on the morphological fit of the available bone mass and the shape of the part that has to be transplanted. To achieve sufficient incorporation of the autograft into the host bone, precise planning and simulation of the surgical intervention based on 3D CT studies is required. This paper presents a method to identify an optimal donor site by performing an optimization of appropriate similarity measures between donor region and a given transplant. At the initial stage the surgeon has to delineate the osteotomy border lines in the template CT data set and to define a set of constraints for the optimization of appropriate similarity measures between donor region and a given transplant. At the initial stage the surgeon has to delineate the osteotomy border lines in the template CT data set and to define a set of constraints for the optimization task in the donor site CT data set. The following fully automatic optimization stage delivers a set of sub-optimal and optimal donor sites for a given template. All generated solutions can be explored interactively on the computer display using an efficient graphical interface. Reconstructive operations supported by our system were performed on 28 patients. We found that the operation time can be considerably shortened by this approach.

We present an aid for facial deformities treatment. We designed a system for surgical planning and prediction of human facial aspect after maxillo-facial surgery. We study the 3D reconstruction process of the tissues involved in the simulation, starting from CT acquisitions. 3D iso-surfaces meshes of soft tissues and bone structures are built. A sparse set of still photographs is used to reconstruct a 360 degree(s) texture of the facial surface and increase its visual realism. Reconstructed objects are inserted into an object-oriented, portable and scriptable visualization software allowing the practitioner to manipulate and visualize them interactively. Several LODs (Level-Of- Details) techniques are used to ensure usability. Bone structures are separated and moved by means of cut planes matching orthognatic surgery procedures. We simulate soft tissue deformations by creating a physically-based springs model between both tissues. The new static state of the facial model is computed by minimizing the energy of the springs system to achieve equilibrium. This process is optimized by transferring informations like participation hints at vertex-level between a warped generic model and the facial mesh.

B-Mode ultrasound is often used during neurosurgery to provide intra-operative images of the brain though a craniotomy, but the use of 3D ultrasound during surgery is still in its infancy. We have developed a system that provides real-time freehand 3D ultrasound reconstruction at a reduced resolution. The reconstruction proceeds incrementally and the 3D image is overlayed, via a computer, on a pre-operative 3D MRI scan. This provides the operator with the necessary feedback to maintain a constant freehand sweep-rate, and also ensures that the sweep covers the desired anatomical volume. All of the ultrasound video frames are buffered, and a full-resolution, compounded reconstruction proceeds once the manual sweep is complete. We have also developed tools for manual tagging of homologous landmarks in the 3D MRI and 3D ultrasound volumes that use a piecewise cubic approximation of thin-plate spline interpolation to achieve interactive nonlinear registration and warping of the MRI volume to the ultrasound volume: Each time a homologous point-pair is identified by the use, the image of the warped MRI is updated on the computer screen after less than 0.5 s.

Deep brain stimulator (DBS) implantation is a promising treatment alternative for suppressing the motor tremor symptoms in Parkinson disease (PD) patient. The main objective is to develop a minimally invasive approach using high spatial resolution and soft-tissue contrast MR imaging techniques to guide the surgical placement of DBS. In the MR-guided procedure, the high spatial resolution MR images were obtained intra-operatively and used to target stereotactically a specific deep brain location. The neurosurgery for craniotomy was performed in the front of the magnet outside of the 10 Gauss line. Aided with positional registration assembly for the stereotactic head frame, the target location (VIM or GPi or STN) in deep brain areas was identified and measured from the MR images in reference to the markers in the calibration assembly of the head frame before the burrhole prep. In 20 patients, MR- guided DBS implantations have been performed according to the new methodology. MR-guided DBS implantation at high magnetic field strength has been shown to be feasible and desirable. In addition to the improved outcome, this offers a new surgical approach in which intra-operative visualization is possible during intervention, and any complications such as bleeding can be assessed in situ immediately prior to dural closure.

This paper presents a procedure for combining MR anatomical information and a stereotactic reference obtained from a CT study with a Leksell frame attached to the patient's head, in order to guide neurosurgery of functional disorders. MRI acquisition can be performed well before the surgery, without the stereotactic frame. The day of the intervention, after attaching the Leksell frame to the patient, a CT image (1.5 mm slices) is acquired. This study will provide the stereotactic reference and includes on ly the part of the brain where the disorder is located. Before surgery, physicians register the MRI with the CT using a procedure based on an automated algorithm (Mutual Information) and visually check the result. MRI is used to locate the target in the brain, while the frame visible in the CT allows to calculate the stereotactic 3D coordinates. Frame references are located on the MRI image allowing to calculate Leksell coordinates of any given point. When the exact position of relevant structures has been recorded, the physicians proceed with the surgery. The protocol has been tested in ten patients, showing a positive surgical outcome with a significant decrease of the functional disorders. The method has proved to be accurate enough, avoiding the use of stereotactical frames during the MR acquisition and making the clinical procedure simpler and faster.

A visual estimation technique has been developed too quickly, yet quantitatively, determine the sharpness quality of CRT displays. While high-resolution camera measurement equipment accurately characterizes display sharpness, the equipment cost is high and the measurements are time consuming to perform. Previously reported visual sharpness assessment techniques are either qualitative or the quantitative measures do not possess adequate sensitivity. The rating scheme investigated in this study provides a practical solution for tracking monitor sharpness in a clinical environment. The target consists of high frequency, high contrast pattern with an embedded, magnified fiducial marker set based upon a Gaussian model for the CRT spot. The magnification of the marker set allows the reference to remain nearly invariant to the actual sharpness of the display. In this study, three commercially available diagnostic displays were evaluated, each at two luminance levels and seven static focus settings. High-resolution CCD camera measurements were acquired for each display and setting combination. The visual sharpness estimate target was then displayed and scored by observers. High correlation was found between the visual ratings and the photometric measurements. More importantly, the sensitivity of the target produced observer ratings, which distinguish between the measured CRT spot sizes at different focus levels.

Display processing is used to transform digital radiography raw data in log-signal units to display values for presentation using a workstation or film printer. Radiographic appearance with respect to subject latitude and detail contrast varies significantly depending on the signal equalization and grayscale rendition used for processing. A human observer study was conducted to define the latitude and detail contrast that is judged optimal for a broad spectrum of chest radiographs. Raw data for 12 chest radiographs acquired with storage phosphor digital radiography systems were transformed using 52 different combinations of latitude and detail contrast. For specific latitude values, contrast was adjusted by varying the equalization gain. Three radiologists at three different medical centers evaluated the images. Each image was compared to a reference image using a calibrated display on a computer workstation. For PA views, processing that produced a detail contrast of 3.14 ((Delta) D/(Delta) logE) and latitude of 1.47 ((Delta) logE for (Delta) D = 1.75) was determined to be best for all cases and was achieved with an equalization gain of 2.64. For lateral views, a detail contrast of 3.42 and latitude of 1.17 was best for all cases (gain = 2.29). For individual cases, the preferred processing varied from the global average primarily with respect to latitude.

Constant-current, active-matrix organic light-emitting displays (AM-OLEDs) with the advanced hydrogenated amorphous silicon thin film transistor (a-Si:H TFT) pixel electrode circuits have been designed in our laboratory for medical applications. An extensive pixel electrode circuit simulation and analysis indicate that a continuous pixel electrode excitation can be achieved with these circuits, and a pixel electrode driving output current level up to 1.4 (mu) A can be reached with an a-Si:H TFT technology. Small feed-through voltage (few tenth of mV) that can be achieved with this circuit will enhance the display gray level controllability needed for medical imaging. Each pixel electrode has a threshold voltage compensation circuit to adjust the pixel electrode driving current level for threshold voltage shifts of both the organic light-emitting diodes (OLEDs) and the current driving a-Si:H TFT. For a 16-inch VGA full-color AM-OLED with a pixel electrode size of ~60x115 micrometers ², the output current level is equivalent to a pixel current density of 20 mA/cm². Assuming the OLEDs with an external quantum efficiency of 1%, the AM-OLED brightness of ~88, ~960, and ~160 cd/m² for red (650 nm), green (540 nm), and blue (480 nm) light emission, respectively, can be achieved with this type of pixel electrode circuits.

Diagnostic workstation hanging protocols describe how to lay out radiographic images according to predefined user preferences. In our prior work, we developed the concept of Structured Display Protocols (SDP) that structure the presentation of data according to the diagnostic task. In this work an object-oriented analysis procedure was used to define the workflow (the process model) for a radiology activity as well as the data objects viewed or manipulated in that activity (the data model). Within the workflow there exist specific phases of activity (modes) designed to satisfy specific sub-tasks in the diagnostic process. Some tasks may, or may not, be performed, depending upon the case. These tasks are mapped to mode-specific tools. A critical feature of a SDP is that each viewing mode presents only the data (and tools) relevant to a specific task. The results of each step were represented using specific UML diagrams. From the UML diagrams, structured display protocols were implemented on a publicly available workstation and were then used in routine clinical practice by radiologists and film librarians.

The wide dynamic range present in digitized mammographic data, partially resulting from the non-uniform thickness of tissue during breast compression, makes it difficult to find window and level values that are appropriate to display the entire image. Further, this factor combined with the non- linearity of the relationship between density and log exposure, confound attempts to automatically derive tissue composition information directly from uncorrected data. This project attempts to address these issues by making appropriate local image corrections based on the characteristic curves of film and digitizer, as well as on the variations in tissue thickness during breast compression. Subjective comparisons of the display techniques developed in this project, to mammography displays based on local histogram equalization methods to reduce image dynamic range, clearly demonstrate superior performance of the methods presented in this paper. In addition to this subjective observation about image display, we also investigated the possibility of using corrected data to improve the performance of tissue composition measurements. A neural network classifier was developed to use features derived from the volume-corrected histogram of the corrected mammographic data to estimate tissue composition. Results indicate that tissue composition measurements are more highly correlated to radiologists' estimates, when they are derived from corrected images.

In this paper, we demonstrate a technique for modeling liver motion during the respiratory cycle using intensity-based free-form deformation registration of gated MR images. We acquired 3D MR image sets (multislice 2D) of the abdomen of four volunteers at end-inhalation, end-exhalation, and eight time points in between using respiratory gating. We computed the deformation field between the images using intensity-based rigid and non-rigid registration algorithms. The non-rigid transformation is a free-form deformation with B-spline interpolation between uniformly-spaced control points. The transformations between inhalation and exhalation were visually inspected. Much of the liver motion is cranial-caudal translation, and thus the rigid transformation captures much of the motion. However, there is still substantial residual deformation of up to 2 cm. The free-form deformation produces a motion field that appears on visual inspection to be accurate. This is true for the liver surface, internal liver structures such as the vascular tree, and the external skin surface. We conclude that abdominal organ motion due to respiration can be satisfactorily modeled using an intensity-based non-rigid 4D image registration approach. This allows for an easier and potentially more accurate and patient-specific deformation field computation than physics-based models using assumed tissue properties and acting forces.

This paper introduces an extendable cross-platform software framework Julius for medical visualization and surgical planning, consisting of two conceptual layers: the Julius Software Development Kit (JSDK) and the Julius Graphical User Interface (JGUI). The JSDK can be used stand-alone to speed up development of research tools. While the JGUI acts like a front end for the JSDK and offers easy handling combined with time-saving functionality to increase performance and productivity. Julius features a modular, cross-platform design and comes with a full set of components, like semi-automatic segmentation, registration, visualization and navigation.

Distortion between the operating field and preoperative images increases as image-guided surgery progresses. Retraction is a typical early-stage event that causes significant tissue deformation, which can be modeled as an intraoperative compensation strategy. This study compares the predictive power of incremental versus single-step retraction models in the porcine brain. In vivo porcine experiments were conducted that involved implanting markers in the brain whose trajectories were tracked in CT scans following known incremental deformations induced by a retractor blade placed interhemispherically. Studies were performed using a 3D consolidation model of brain deformation to investigate the relative predictive benefits of incremental versus single-step retraction simulations. The results show that both models capture greater than 75% of tissue loading due to retraction. We have found that the incremental approach outperforms the single-step method with an average improvement of 1.5%-3%. More importantly it also preferentially recovers the directionality of movement, providing better correspondence to intraoperative surgical events. A new incremental approach to tissue retraction has been developed and shown to improve data-model match in retraction experiments in the porcine brain. Incremental retraction modeling is an improvement over previous single- step models, which does not incur additional computational to overhead. Results in the porcine brain show that even when the overall displacement magnitudes between the two models are similar, directional trends of the displacement field are often significantly improved with the incremental method.

During neurosurgery, the challenge for the neurosurgeon is to remove as much as possible of a tumor without destroying healthy tissue. This can be difficult because healthy and diseased tissue can have the same visual appearance. To this aim, and because the surgeon cannot see underneath the brain surface, image-guided neurosurgery systems are being increasingly used. However, during surgery, deformation of the brain occurs (due to brain shift and tumor resection), therefore causing errors in the surgical planning with respect to preoperative imaging. In our previous work, we developed software for capturing the deformation of the brain during neurosurgery. The software also allows preoperative data to be updated according to the intraoperative imaging so as to reflect the shape changes of the brain during surgery. Our goal in this paper was to rapidly visualize and characterize this deformation over the course of surgery with appropriate tools. Therefore, we developed tools allowing the doctor to visualize (in 2D and 3D) deformations, as well as the stress tensors characterizing the deformation along with the updated preoperative and intraoperative imaging during the course of surgery. Such tools significantly add to the value of intraoperative imaging and hence could improve surgical outcomes.

To improve depth perception a projection scheme was developed, which incorporates visual perspective transformation in the MR image reconstruction, in which an observation point (OP) was selected in space relative to the image volume. The image formation process was modeled as a camera, whose lens center (LC) coincide with the OP, and a picture plane (PP) was placed at the focal plane. The finite sized PP corresponded to what the observer saw. Each pixel in the PP defined a ray projection trajectory (PT) from LC. The intensity values for a set of discrete points along the PT were interpolated from the image volume. From these projection profiles, various projection images can be derived depending how the intensity profile is manipulated. To demonstrate the effectiveness of the method, we obtained three dimensional image data sets, which included anatomical (TR/TE/flip=13/6/9) and angiographical data sets (TR/TE/Flip=30/5.4/15) in transverse orientation from a patient head at 1.5T. Both quasi-surface rendering and MaxIP algorithms were used along the viewing trace of perspective projection than rather the parallel projection. For multiple Ops with different distances from the 3D image volume, consecutive views were obtained at a constant interval azimuthally. When displayed in cine-mode, the MaxIP images appeared realistic with improved depth perception, different portions of the image moved naturally according to the expectation of human visual system.

We present a system involving a computer-instrumented fluoroscope for the purpose of 3D navigation and guidance using pre-operative diagnostic scans as a reference. The goal of the project is to devise a computer-assisted tool that will improve the accuracy, reduce risk, minimize the invasiveness, and shorten the time it takes to perform a variety of neurosurgical and orthopedic procedures of the spine. For this purpose we propose an apparatus that will track surgical tools and localize them with respect to the patient's 3D anatomy and pre-operative 3D diagnostic scans using intraoperative fluoroscopy for in situ registration and localization of embedded fiducials. Preliminary studies have found a fiducial registration error (FRE) of 1.41 mm and a Target Localization Error (TLE) of 0.48 mm. The resulting system leverages equipment already commonly available in the operating room (OR), providing an important new functionality that is free of many current limitations, such as the inadequacy of skin fiducials for spinal neuronavigation, while keeping costs contained.

In computerized image and graphic applications, texture mapping is one of the most commonly used methods to improve the realism or to enhance the visual effect of object rendering without too much increase in computational complexity. In conventional texture mapping, a 3D object has to be transferred to the polygonal structure, and then mapped with a texture or photographic image. However, it is usually computationally expensive to transfer the original data in voxels to polygons, and is even more complex to map the 2D texture image onto the 3D polygonal structure. Even though the computation is huge, the polygonal structure is not efficient in preserving the internal information of volume data. Because most volume data acquired by medical imaging devices or 3D scanners are in voxel format, it is more appropriate to handle these data directly in voxel format. In this paper, we propose a new texture mapping method based on chain-coding flattening to handle the voxel- based data directly. Therefore, the computation is reduced significantly and the internal information can be utilized and preserved thoroughly. The method flattens a 3D object surface onto a 2D plane and then uses 2D warping technique to generate the correspondences between the object surface and the texture image. Therefore, polygonal transformation required in the conventional approach is no longer necessary and texture mapping is handled with inexpensive 2D computation. Experimental results have shown the effectiveness and efficiency of the proposed algorithm.

A system has been developed to produce live 3D volume renderings from an MR scanner. Whereas real-time 2D MR imaging has been demonstrated by several groups, 3D volumes are currently rendered off-line to gain greater understanding of anatomical structures. For example, surgical planning is sometimes performed by viewing 2D images or 3D renderings from previously acquired image data. A disadvantage of this approach is misregistration which could occur if the anatomy changes due to normal muscle contractions or surgical manipulation. The ability to produce volume renderings in real-time and present them in the magnet room could eliminate this problem, and enable or benefit other types of interventional procedures. The system uses the data stream generated by a fast 2D multi- slice pulse sequence to update a volume rendering immediately after a new slice is available. We demonstrate some basic types of user interaction with the rendering during imaging at a rate of up to 20 frames per second.

Diffusion weighted magnetic resonance imaging (DW MRI) is a technique that measures the diffusion properties of water molecules to produce a tensor-valued volume dataset. Because water molecules can diffuse more easily along fiber tracts, for example in the brain, rather than across them, diffusion is anisotropic and can be used for segmentation. Segmentation requires the identification of regions with different diffusion properties. In this paper we propose a new set of rotationally invariant diffusion measures which may be used to map the tensor data into a scalar representation. Our invariants may be rapidly computed because they do not require the calculation of eigenvalues. We use these invariants to analyze a 3D DW MRI scan of a human head and build geometric models corresponding to isotropic and anisotropic regions. We then utilize the models to perform quantitative analysis of these regions, for example calculating their surface area and volume.

Previously shell rendering was shown to be an ultra fast method of rendering digital surfaces represented as sets of voxels. The purpose of this work is to describe a new extension to the shell rendering method that creates isosurfaces called t-shells using triangulated shell elements. The great speed of the shell rendering technique is made available through the use of data structures that describe the shell and through algorithms that traverse this information to produce the 2D projection of the 3D data. In traditional shell rendering, each shell element is a triple comprised of the offset from the start of the row, the neighborhood code, and the surface normal. We modify this data structure by replacing the neighborhood code with a code that indicates the configuration of triangles within that area. The t-shell algorithm modifies the original shell rendering algorithm to project 1 of the 256 possible triangulated configurations (rather than the rasterization of a single, uniform shell element). We present the general t-shell algorithm as well as the results of two preliminary implementations as applied to input data which consist of different object from various parts of the body and various modalities with a variety of surface sizes and shapes. We present the results of some initial timing experiments as well as some preliminary, sample renditions.

3D ultrasound (US) provides physicians with a better understanding of human anatomy. By manipulating the 3D US data set, physicians can observe the anatomy in 3D from a number of different view directions and obtain 2D US images that would not be possible to directly acquire with the US probe. In order for 3D US to be in widespread clinical use, creation and manipulation of the 3D US data should be done at interactive times. This is a challenging task due to the large amount of data to be processed. Our group previously reported interactive 3D US imaging using a programmable mediaprocessor, Texas Instruments TMS320C80, which has been in clinical use. In this work, we present the algorithms we have developed for real-time 3D US using a newer and more powerful mediaprocessor, called MAP-CA. MAP-CA is a very long instruction word (VLIW) processor developed for multimedia applications. It has multiple execution units, a 32-kbyte data cache and a programmable DMA controller called the data streamer (DS). A forward mapping 6 DOF (for a freehand 3D US system based on magnetic position sensor for tracking the US probe) reconstruction algorithm with zero- order interpolation is achieved in 11.8 msec (84.7 frame/sec) per 512x512 8-bit US image. For 3D visualization of the reconstructed 3D US data sets, we used volume rendering and in particular the shear-warp factorization with the maximum intensity projection (MIP) rendering. 3D visualization is achieved in 53.6 msec (18.6 frames/sec) for a 128x128x128 8-bit volume and in 410.3 msec (2.4 frames/sec) for a 256x256x256 8-bit volume.

The use of flat-panel imagers for cone-beam CT signals the emergence of an attractive technology for volumetric imaging. Recent investigations demonstrate volume images with high spatial resolution and soft-tissue visibility and point to a number of logistical characteristics (e.g., open geometry, volume acquisition in a single rotation about the patient, and separation of the imaging and patient support structures) that are attractive to a broad spectrum of applications. Considering application to image-guided (IG) procedures - specifically IG therapies - this paper examines the performance of flat-panel cone-beam CT in relation to numerous constraints and requirements, including time (i.e., speed of image acquisition), dose, and field-of-view. The imaging and guidance performance of a prototype flat panel cone-beam CT system is investigated through the construction of procedure-specific tasks that test the influence of image artifacts (e.g., x-ray scatter and beam-hardening) and volumetric imaging performance (e.g., 3D spatial resolution, noise, and contrast) - taking two specific examples in IG brachytherapy and IG vertebroplasty. For IG brachytherapy, a procedure-specific task is constructed which tests the performance of flat-panel cone-beam CT in measuring the volumetric distribution of Pd-103 permanent implant seeds in relation to neighboring bone and soft-tissue structures in a pelvis phantom. For IG interventional procedures, a procedure-specific task is constructed in the context of vertebroplasty performed on a cadaverized ovine spine, demonstrating the volumetric image quality in pre-, intra-, and post-therapeutic images of the region of interest and testing the performance of the system in measuring the volumetric distribution of bone cement (PMMA) relative to surrounding spinal anatomy. Each of these tasks highlights numerous promising and challenging aspects of flat-panel cone-beam CT applied to IG procedures.

We are developing a video see-through head-mounted display (HMD) augmented reality (AR) system for image-guided neurosurgical planning and navigation. The surgeon wears a HMD that presents him with the augmented stereo view. The HMD is custom fitted with two miniature color video cameras that capture a stereo view of the real-world scene. We are concentrating specifically at this point on cranial neurosurgery, so the images will be of the patient's head. A third video camera, operating in the near infrared, is also attached to the HMD and is used for head tracking. The pose (i.e., position and orientation) of the HMD is used to determine where to overlay anatomic structures segmented from preoperative tomographic images (e.g., CT, MR) on the intraoperative video images. Two SGI 540 Visual Workstation computers process the three video streams and render the augmented stereo views for display on the HMD. The AR system operates in real time at 30 frames/sec with a temporal latency of about three frames (100 ms) and zero relative lag between the virtual objects and the real-world scene. For an initial evaluation of the system, we created AR images using a head phantom with actual internal anatomic structures (segmented from CT and MR scans of a patient) realistically positioned inside the phantom. When using shaded renderings, many users had difficulty appreciating overlaid brain structures as being inside the head. When using wire frames, and texture-mapped dot patterns, most users correctly visualized brain anatomy as being internal and could generally appreciate spatial relationships among various objects. The 3D perception of these structures is based on both stereoscopic depth cues and kinetic depth cues, with the user looking at the head phantom from varying positions. The perception of the augmented visualization is natural and convincing. The brain structures appear rigidly anchored in the head, manifesting little or no apparent swimming or jitter. The initial evaluation of the system is encouraging, and we believe that AR visualization might become an important tool for image-guided neurosurgical planning and navigation.

In this paper we present an extendable framework fo interactive biomechanical simulation and surgical planning. Three dimensional reconstructions of patient specific data are visualized and the elastic properties of individual anatomical structures are modeled. To avoid interpenetration of the virtual objects fast collision detection has been implemented. The kinematics are controlled by a special haptic interface which provides force feedback to the user. The ongoing work will lead to an entire system for physical-based biomechanical simulation and pre-operative visualization of surgical outcomes.