Gregorio Marañón Hospital creates a surgical solution that combines augmented reality with 3D printing

The solution combines Hololens glasses with personalized 3D printing so that the surgeon has a holographic view of the patient's radiological tests and 3D reconstructions of their pathology.

Surgeons and engineers of the Hospital Gregorio Marañón They have developed a system for the operating room, that combines the use of augmented reality glasses and custom 3D printing, to project holograms of their radiological tests onto the patient who is going to be operated on, 3D reconstructions of your pathology and surgical planning.

This tool is supported by a very precise navigation system that allows the patient's posture on the surgical table to be identified and the virtual images to be displayed with a submillimeter degree of accuracy. It is as if the surgeon will be provided with X-rays while operating.

Allows you to improve results, Minimise risks and shorten operation times. The Challenge of Positioning Systems, Surgical Navigation and Augmented Reality in the ERA Operating Room, Until now, To be able to identify the exact position of the patient in order to automatically and with sufficient accuracy project the virtual information previously processed on the computer.

The Gregorio Marañón Hospital has achieved a development that allows the projection of the radiological studies of the patient (TAC, RM, PET, and so on) directly on the patient himself with a submillimeter margin of error, and this has been achieved thanks to the combination of augmented reality technology with Microsoft Hololens custom 3D printing glasses.

Thanks to the hospital's expertise in 3D printing (technology implemented at the center since the year 2015 and applied as a communication tool, Simulation and therapeutic planning for more than 18 Medical-Surgical Specialties of the Hospital) A solution was identified that has made it possible to synchronize the virtual 3D images of the patient with the real patient himself.

This leap was achieved with the 3D printing of a custom template that includes an optical marker that tells the mixed reality glasses where to project the previously generated 3D holograms.

The template is designed in 3D in the hospital itself based on the reconstruction of the patient's radiological study (TAC, RM, PET), so it adapts in a certain position and only in a specific patient, being manufactured with the exact anatomical shape of the patient at the selected point.

This process is carried out in the hospital's Printing Laboratory where an optical marker is integrated as a graphic pattern, which is recognized by the glasses and which tells the system where to project the desired 3D reconstructions.

This whole system is as if the surgeon were given X-rays during surgery, since radiological studies can be projected onto the body to build new layers of advanced visual information on the patient and allows the patient to select the radiological study or 3D object of the patient as a hologram.

During the operation, surgeons can select, using voice commands or gestures, the radiological and 3D models that they want to be projected onto the patient.

It is an aid system that increases the anatomical and 3D information of the patient during the intervention itself, without interfering with the normal procedure. The operation is performed according to the usual surgical protocol, but with an advanced visualization tool that helps shorten surgical times, Minimise risks and improve outcomes.

Case Study

The Gregorio Marañón Hospital has already performed an operation with this technology to remove a tumor in a patient's leg, becoming a pioneer when it comes to creating this combination of technologies and taking it to a real surgery, working with that information in the operating room itself.

On the other hand, are working so that any smartphone with augmented reality support can identify the marker embedded in the 3D printed template and, About her, project the 3D reconstruction on the mobile phone screen. This is very useful in order to universalize the application and use it in simulation or medical training environments at low cost.

The development has been carried out through the collaborative work of the Gregorio Marañón Health Research Institute, through an interdisciplinary team of researchers led by Dr. Rubén Pérez Mañanes and Dr. José Antonio Calvo (Orthopedic Oncology Surgeons) and Javier Pascau (Director of the Bachelor's Degree in Biomedical Engineering at the University of UC3M), together with the Madrid company 6DLab.

Other articles on 6Dlab, Hospital Gregorio Marañón, Carlos III University, Universidad Carlos III de Madrid