Augmented Reality for Neurosurgery
In neurovascular surgery, and in particular surgery for arteriovenous malformations (AVMs), the surgeon must map pre-operative images of the patient to the patient on operating room (OR) table in order to understand the topology and locations of vessels below the visible surface. This type of spatial mapping is not trivial, is time consuming, and may be prone to error. Using augmented reality (AR) we can register the microscope/camera image to pre-operative patient data in order to aid the surgeon in understanding the topology, the location and type of vessel lying below the surface of the patient. This may reduce surgical time and increasing surgical precision. In this project as well as studying a mixed reality environment for neuromuscular surgery, we will examine and evaluate which visualization techniques provide the best spatial and depth understanding of the vessels beyond the visible surface.
Augmented Reality in the Operating Room for Neurovascular Surgery
A: Colour coding of a vascular DS-CTA volume based on blood flow. B: Vessels overlaid on the patient skin prior to draping (left). The AR view is used at this step to help tailor the extent of the craniotomy. On the right we see vessels overlaid on the cortex prior to resection, here the AR view is used to determine the optimal resection corridor. The blue arrows point to the pink markers that indicate the location of deep feeding arteries. The orange arrow indicates the major arterialized vein, shown as red and not blue. C: Different visualization techniques for combining the live camera image (prior to resection) with the virtual vessels (green, red, blue) are shown.The use of simple alpha-blending between the real and virtual worlds does not provide spatial information (top). More sophisticated techniques such as modulating transparency in the area of interest and using edges (from the virtual vessels and/or camera image) and using fog are applied. D: Based on the virtual information the surgeon placed a micropad on the brain surface above a virtual marker representing a deep feeding artery to help with the resection approach and vessel localization.
Publications
M. Kersten-Oertel, M., Gerard, I., Drouin, S., Mok, K., Sirhan, D., Sinclair, D. S. and Collins, D. L. Augmented reality in neurovascular surgery: feasibility and first uses in the operating room. IJCARS (2015): 1-14.
Kersten-Oertel, M., Gerard, I. J., Drouin, S., Mok, K., Sirhan, D., Sinclair, D. S., & Collins, D. L. (2015). Augmented Reality for Specific Neurovascular Surgical Tasks. In Augmented Environments for Computer-Assisted Interventions (pp. 92-103). Springer International Publishing.
M. Kersten-Oertel, I. Gerard, S. Drouin, K. Mok, D. Sirhan, D. Sinclair, D. L. Collins. “Augmented Reality in Neurovascular Surgery: First Experiences." Augmented Environments for Computer-Assisted Interventions. Lecture Notes in Computer Science Volume 8678, 2014, pp 80-89, 2014.
Augmented Reality (using a phantom)
Enhancing visualization of AR views
Publications
M. Kersten-Oertel, S. Drouin, S. J. S. Chen, D. L. Collins. '"Volume Visualization for Neurovascular Augmented Reality Surgery". Augmented Reality Environments for Medical Imaging and Computer-Assisted Interventions. Lecture Notes in Computer Science Volume 8090, pp 211-220, 2013.
M. Kersten-Oertel, S.J.S Chen, S. Drouin, D. Sinclair, D. L. Collins. “Augmented Reality Visualization for Guidance in Neurovascular Surgery.” Stud Health Technol Inform 173:225–9. Proceedings of Medicine Meets Virtual Reality (MMVR), New Port, CA, Feb 9-11, 2012.
S. Drouin, M. Kersten-Oertel, S. J. S. Chen, D. L. Collins. "A Realistic Test and Development Environmentfor Mixed Reality in Neurosurgery". Augmented Environments for Computer Assisted Interventions' (Proceedings of MICCAI AE-CAI Workshop 2011) Lecture Notes in Computer Science, Volume 7264:13-23, 2012.
Visualizing Vascular Volumes
Cerebral vascular images obtained through angiography are used by neurosurgeons for diagnosis, surgical planning and intra-operative guidance. The intricate branching of the vessels and furcations, however, make the task of understanding the spatial three-dimensional layout of these mages challenging. In this paper, we present empirical studies on the effect of different perceptual cues (fog, pseudo-chromadepth, kinetic depth, and depicting edges) both individually and in combination on the depth perception of cerebral vascular volumes and compare these to the cue of stereopsis. Two experiments with novices and one experiment with experts were performed. The results with novices showed that the pseudo-chromadepth and fog cues were stronger cues than that of stereopsis. Furthermore, the addition of the stereopsis cue to the other cues did not improve relative depth perception in cerebral vascular volumes. In contrast to novices, the experts also performed well with the edge cue. In terms of both novice and expert subjects, pseudo-chromadepth and fog allow for the best relative depth perception, although experts unlike novices also performed well with the edge cue. By using such cues to improve depth perception of cerebral vasculature we may improve diagnosis, surgical planning, and intra-operative guidance.
Take the Augmented Reality Visualization Questionnaire here
Publications
Marta Kersten-Oertel, S. J. S. Chen, D. Louis Collins. An Evaluation of Depth Enhancing Perceptual Cues for Vascular Volume Visualization in Neurosurgery. IEEE Trans Vis Comput Graph. March, 2014.
M. Kersten-Oertel, S. J. S. Chen, D. L. Collins. "A Comparison of Depth Enhancing Perceptual Cues for Vessel Visualization in Neurosurgery." CARS, June 27-30, 2012.
Visualizing Blood Flow
Cerebral arteriovenous malformations (AVMs) are a type of vascular anomaly consisting of large intertwined vascular growth (the nidus) that are prone to serious hemorrhaging and can result in patient death if left untreated. Intervention through surgical clipping of feeding and draining vessels to the nidus is a common treatment. However, identification of which vessels to clip is challenging even to experienced surgeons aided by conventional image guidance systems. In this work, we describe our methods for processing static preoperative angiographic images in order to effectively visualize the feeding and draining vessels of an AVM nidus. Maps from level-set front propagation processing of the vessel images are used to label the vessels by colour. Furthermore, images are decluttered using the topological distances between vessels. In order to aid the surgeon in the vessel clipping decision-making process during surgery, the results are displayed to the surgeon using augmented virtuality.
Decluttering and visualizing blood flow
Publications
S. J. S. Chen, M. Kersten-Oertel, S. Drouin, and D. L. Collins. "Visualizing the path of blood flow for image guided surgery of cerebral arteriovenous malformations". SPIE Medical Imaging, San Diego, CA, Feb 4-9, 2012.
M. Kersten-Oertel, S. J. S. Chen, D. L. Collins. "Enhancing depth perception of volume-rendered angiography data". VIS 2011 Poster Session, Providence, RI, Oct. 23-38, 2011.
Related Publications
Marta Kersten-Oertel, Pierre Jannin, D. Louis Collins, "The state of the art of visualization in mixed reality image guided surgery". Computerized Medical Imaging and Graphics, 37(2):98-112, March 2013.
Marta Kersten-Oertel, Pierre Jannin, D. Louis Collins, "DVV: A Taxonomy for Mixed Reality Visualization in Image Guided Surgery". IEEE Transactions on Visualization and Computer Graphics, 18(2):332-352, Feb. 2012.
M. Kersten-Oertel, P. Jannin, D. L. Collins, "DVV: Towards a Taxonomy for Mixed Reality Visualization in Image Guided Surgery". Medical Imaging and Augmented Reality (Proceedings of MIAR 2010), Lecture Notes in Computer Science, 6326:334-343, 2010.