'Mixed Reality' Technology Brings Mars to Earth

Estimated reading time: 2 minutes

What might it look like if you were walking around on Mars? A group of researchers at NASA's Jet Propulsion Laboratory, Pasadena, California, has been working on methods to take this question from the realm of imagination to the mind-bending domain of mixed reality.

As a result, NASA and Microsoft have teamed up to offer the public a guided tour of an area of Mars with astronaut Buzz Aldrin this summer in "Destination: Mars," an interactive exhibit using the Microsoft HoloLens mixed reality headset. "Mixed reality" means that virtual elements are merged with the user's actual environment, creating a world in which real and virtual objects can interact.

The "Destination: Mars" exhibit will open at NASA’s Kennedy Space Center Visitor Complex in Florida this summer.  Guests will "visit" several sites on Mars, reconstructed using real imagery from NASA's Curiosity Mars Rover, which has been exploring the Red Planet since August 2012. Buzz Aldrin, an Apollo 11 astronaut who walked on the moon in 1969, will serve as “holographic tour guide” on the journey. Curiosity Mars rover driver Erisa Hines of JPL will also appear holographically, leading participants to places on Mars where scientists have made exciting discoveries and explaining what we have learned about the planet.

“This experience lets the public explore Mars in an entirely new way.  To walk through the exact landscape that Curiosity is roving across puts its achievements and discoveries into beautiful context,” said Doug Ellison, visualization producer at JPL.

"Destination: Mars" is an adaptation of OnSight, a Mars rover mission operations tool co-developed by Microsoft and JPL. A pilot group of scientists uses OnSight in their work supporting the Curiosity Mars rover's operations.

"We're excited to give the public a chance to see Mars using cutting-edge technologies that help scientists plan Curiosity's activities on Mars today," said Jeff Norris, project manager for OnSight and "Destination: Mars" at JPL. "While freely exploring the terrain, participants learn about processes that have shaped this alien world."

Abigail Fraeman, a Curiosity science team member at JPL, uses OnSight to make recommendations about where the rover should drive and which features to study in more detail. Recently OnSight helped her and a colleague identify the transition point between two Martian rock formations, which they would like to study in further detail.

"OnSight makes the whole process of analyzing the data feel a lot more natural to me," Fraeman said. "It really gives me the sense that I'm in the field when I put it on. Thinking about Martian geology is a lot more intuitive when I can stand in the scene and walk around the way I would if I were in the field."

By utilizing the same technologies and datasets as OnSight, "Destination: Mars" offers participants a glimpse of Mars as seen by mission scientists.

JPL is also developing mixed reality applications in support of astronauts on the International Space Station and engineers responsible for the design and assembly of spacecraft. NASA astronaut Scott Kelly, who recently returned from his historic "Year in Space" activities, used one of these applications to make the first Skype call from space to mission control in February 2016.

"By connecting astronauts to experts on the ground, mixed reality could be transformational for scientific and engineering efforts in space," Norris said.

“As we prepare to send humans to Mars in the 2030s, the public will now be able to preview the experience the astronauts will have as they walk and study the Martian surface,” said Dave Lavery, program executive for Solar System Exploration at NASA Headquarters in Washington.