Heal: Emerging technology making the world healthier

A New Way to Approach and Experience Medical Care

What could Star Trek teach us about health care and surgery?

Science fiction writers love playing with virtual worlds that harness theoretical physics for real-world applications and make it possible for characters to go beyond the reality we experience here on Earth. Star Trek’s holodeck, dreamed up by series creator Gene Roddenberry after he met with a holography researcher in the 1970s, is a prime example.

In the Star Trek holodeck, holographic images are projected to create simulations of locations, objects, and people that appear to be real and touchable. But when the holodeck’s program ends, the simulations fade and Captain Picard is standing in a gridded, enclosed room.

Researchers at New York University (NYU), including computer science professor Ken Perlin, are trying to make the holodeck a reality. “It’s been inspired by that sense of freedom—basically the promise of the idea that we don’t need to be bound by the rules of the particular physical reality we happen to be inhabiting,” Perlin said.

Exploring what happens when possibility becomes reality.

The goal of the NYU Holodeck is to create an immersive world that spans the virtual and the physical, allowing people to collaborate even if they’re thousands of miles from each other. “We’re going to say, ‘Well, I want to collaborate with you, and we’re going to be exploring a molecule together, or flying around in the universe, or looking at a cell or walking around a structural model of the economy, or whatever problem it is that we need to solve,” Perlin said.

The NYU Holodeck was announced in September 2016, and one of the team’s current goals is to find the least intrusive way possible for people to inhabit the same physical space in virtual reality. That involves developing precision object-tracking technology, as well as working out conceptual kinks, such as the best way to hand a physical object to a person in a virtual environment.

This type of technology could have a profound effect on medicine. Perlin cited examples of a neurosurgeon using the NYU Holodeck to do preoperative planning, or medical students watching a diagram, floating in the air, that animates a procedure. Doctors in Provo, Utah, are already working with the HoloLens for similar purposes, converting 2-D medical images into 3-D projected holograms that can be used both to train medical students and to clarify procedures for patients.

But Perlin also knows there are unconceived possibilities for the Holodeck. “There will be things that we will be able to do, for example, in the realm of the teacher–student relationship that we’re not even thinking of yet,” he said.

For example, EchoPixel has introduced interactive 3-D holograms to select hospitals across the country, allowing doctors to turn data from 2-D CT and MRI scans into 3-D interactive images that can be manipulated as if they were actual body parts. This technology has already proven effective, with doctors detecting more congenital heart defects in 40 percent less time. There is also optimism among doctors that this noninvasive examination technique could encourage patients to come in for checkups more frequently.

Not only could virtual and holographic advances make medicine more efficient, but they could also help doctors make their patients more comfortable and more involved—two adjectives rarely associated with medicine. The incorporation of virtual, manipulated, and interactive simulations of the body and its treatment could help doctors see more and see better. It could also augment transparency between them and their patients.

The Possibility Report is an ongoing series about how technology is changing our understanding of the world around us. This article is part of HEAL, our discussion about the ways technology can be used to heal human bodies, animal populations, and the entire planet.