Background: Current display technologies can be suboptimal, even inadequate, when conveying spatially-complex healthcare concepts to patients and providers. This can lead to difficulties in sharing medical information, with potentially deleterious effects on patient outcomes, research, and trainee education. Methods: Investigators used off-the-shelf augmented reality (AR) technologies to facilitate visual communication for healthcare. Using untethered headset devices (Microsoft HoloLens), proof-of-concept work was completed for two use-cases: 1.) multi-user shared AR visualizations and 2.) AR-guided invasive procedural performance. The research team collaborated to create: 1.) a shared AR environment that enabled multiple users to independently visualize and manipulate 3D patient anatomic models with position, rotation, and scale synchronized across users; and 2.) a hybrid [AR-physical] covered box configuration containing CT-scanned targets and custom trajectory guidance system for simulated needle aspiration. As a pilot study exploring technical feasibility and experimental viability of the selected approach, measurements of 1.) size, displacement, angular rotation, and 2.) expert aspiration success were used as primary metrics. Results: The mean difference between AR models and physical objects was 2.0±0.4% and 1.7±0.4% of all dimensions on two HoloLens devices. One shared model configuration exhibited deviations of 7.8±2.8mm in location and 0.5±0.9° in angular orientation, and another showed differences of 6.5±2.1mm and 0.1±0.7° . For AR-guided procedure simulations, two expert surgeons required 3 attempts in 10 minutes and 1 attempt in 3 minutes to successfully aspirate the hybrid targets. Conclusion: AR technologies were used to enable core elements of an interactive shared medical visualization environment and a guided procedure simulation.

© (2019) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE). Downloading of the abstract is permitted for personal use only.

Lawrence Huang, Scott Collins, Leo Kobayashi, Derek Merck, and Thomas Sgouros, "Shared visualizations and guided procedure simulation in augmented reality with Microsoft HoloLens," Proc. SPIE 10951, Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, 1095112 (Presented at SPIE Medical Imaging: February 18, 2019; Published: 8 March 2019); https://doi.org/10.1117/12.2511321.