Bridging the Gap

As a Radiologist I was trained to build complex 3-dimensional models in my mind as I read a stack of 2-dimensional CT or MRI images. Currently, however, the way I must ultimately convey this information is through a written radiology report or an in-person review. As many radiologists will tell you, we have seen how this gap between our 2D world and the real 3D world has caused confusion and potential for complications in surgery. I have seen colleagues literally drawing a picture of the anatomy on a piece of scratch paper for a surgeon who was planning an operation.

I felt there must be a better way and looked to how I could better display information in 3D. After testing a number of methods, it was when I found Augmented Reality (A.R.) that I saw “light bulbs” start to truly go off, in my surgical colleague’s minds. I believe A.R. provides the level of experiential interactivity that surgeons require and that fully expresses the mental images I have created during my interpretation. I began quickly creating CT and MRI based A.R. models for every complex surgical case I saw.


But Not So Fast…

The challenge, I noted was accessibility. While the cost of the head mounted devices (HMDs) was relatively low compared to other medical equipment it was not readily available and a user learning curve was definitely present. My colleagues and I certainly prefer the experience provided by these HMDs, though I wished to share these cases with a broader audience.

I then learned of Google ARCore, Apple’s ARKit, and the MergeVR’s Merge Cube. In these platforms, I could quickly add a model that could be used for review, but also projected onto a conference monitor and displayed to an audience of my colleagues. The ability to quickly iterate, share, and easily manipulate these 3D objects was remarkable.

A Broader Reach

What I have seen in my experience with mobile platforms is that these tools offer a way to provide Augmented Reality experiences for a broader audience. The ability to pull out your phone and quickly share a 3D Augmented Reality model of a heart defect or a skull deformity with a colleague is extremely satisfying. Pairing this experience with an interactive device such as the Merge Cube also removes a significant portion of the user’s learning curve.

I see these tools as having tremendous potential in the fields of education (models like these at my fingertips would have been liquid gold when I was a medical student!), training, and most critically in pre-surgical planning. Having the ability to, for instance send a complex CT-based model to your surgical team the day before to understand the 3-dimensional complexities, or to send an unusual case to a colleague on the other side of the world would allow for better communication and understanding before entering the operating room. The fact that these can be viewed anywhere you can bring a mobile device (from the office, to the O.R., to your kitchen table) I believe opens a tremendous amount of potential for improved and more efficient communication of medical imaging information. These technologies hold the potential to truly begin bridging the 2D-3D gap in medical imaging, and I am very excited to be a part of innovation in this area!


View the original post here: The Potential Of Mobile Augmented Reality