Experimental AR projection system lets surgeons see ‘through’ a patient’s skin

Imagine if surgeons or other clinicians were able to more easily see their patients’ internal anatomy without having to cut into them. That’s the basis for an intriguing project, developed by researchers at the University of Alberta. Called ProjectDR, it’s an augmented reality system that allows medical images such as CT scans, ultrasound, X-rays, and MRI data to be displayed on an individual so that it moves along with them.

“The ability to project medical images directly onto a patient’s body helps those viewing these images to recover patient-specific anatomical information in context for more accurate interpretation and action,” Pierre Boulanger, professor in the Department of Computing Science, told Digital Trends. “Using this system, patients can be digitized before the procedure, and these images can then be used as guides to perform surgeries — including the real-time location of the tools used by the surgeon relative to the projected information. The real-time aspect of the system allows this alignment to adapt to patient motion and to display video images like the one used in endoscopic surgeries.”

University of Alberta

As Boulanger points out, the required computer hardware for the ProjectDR system is actually relatively simple. It includes a 3D-optical tracking system for global positioning, a range sensor — like a Microsoft Kinect — to digitize the curved surface of the patient’s skin, and a special laser scanning projector to display the images without losing focus.

The computer tracks, processes, and renders the augmented images using specialized real-time algorithms developed by the researchers. Clinicians in the room don’t require any individual hardware, like a smartphone or tablet, to view the images. It’s also compatible with the strict aseptic requirements of an operating room since it involves nothing more nefarious than projecting laser light onto a patient’s body.

“We are now in the process of doing pilot testing with surgeons — first into simulated operating conditions and then in real-world operating room conditions for simpler procedures,” he said. “In many ways, this technology is disruptive and requires rethinking how surgical procedures are performed. We are also testing the system for teaching anatomy and surgical procedures. As for commercialization, a few companies are interested in the concept, and licensing negotiations are currently on the way.”