In Steven Spielberg’s 2018 adaptation of Ernest Cline’s 2011 novel Ready Player One, on the Planet Doom in the network of simulated places known as The OASIS there’s a climactic battle for control over that fictional multiverse.
In the film, set decades in the future, fans of The OASIS run down physical streets decked out in VR gear while their systems seamlessly map virtual worlds onto their physical surroundings — thus maintaining the illusion indefinitely. Sure it looks pretty socially awkward and raises uncomfortable questions about the value people place on their physical environment, but it still represents a remarkable technical achievement if it could actually be done:
Even technical viewers, though, steeped in VR technology might look at such depictions and wonder if this sort of thing would ever be feasible. It isn’t readily apparent, for instance, how a virtual world could compensate and change to allow for navigation while strolling through public spaces outdoors.
Turns out Microsoft researchers are already figuring out those answers. The publication of a new paper presented as part of the ACM Symposium on User Interface Software and Technology (UIST) titled “DreamWalker” comes from Stanford University PhD student Jackie Yang, who was a Microsoft Research intern during the work, and Microsoft researchers Eyal Ofek, Andy Wilson, and Christian Holz. The paper reveals in detail how they built a city-scale redirected walking system for VR on top of a Samsung Odyssey headset and additional sensing hardware.
DreamWalker
The paper’s full title is“DreamWalker: Substituting Real-World Walking Experiences with a Virtual Reality” and you can read the entire 14-page PDF yourself.
In summary, though, “we explore a future in which people spend considerably more time in virtual reality, even during moments when they walk between locations in the real world.” DreamWalker builds on earlier research while working “in unseen large-scale, uncontrolled and public areas on contiguous paths in the real-world that are void of moving vehicles.”
According to the paper:
“The Windows Mixed Reality system provides inside-out tracking on a Samsung Odyssey VR headset, updating sensed 6D locations at 90 Hz. Empirically, we measured 1 m of drift over a course of just 30 m through the inside-out tracking alone. Two Intel RealSense 425 cameras provide RGB depth images, slightly angled and rotated 90 degrees to achieve a large field of view (86◦ × 98◦). We built a custom adapter for the backpack computer that converts Thunderbolt 3 to four USB 3 ports and thus supports the bandwidth required to stream both RGB depth cameras at a resolution of 640×480 (depth) and 640×480 (RGB) at 30 Hz. Finally, GPS data comes from the sensor inside a Xiaomi Mi 8 phone…”
The paper describes how measurements from all these devices is combined and analyzed to keep the person in VR on a safe and comfortable path through the real world without ever taking the headset off such that “8 participants walked across campus along a 15-minute route, experiencing a virtual Manhattan that was full of animated cars, people, and other objects.”
In the video below you can see different ways the software encourages people to stay on the right path, including using “humanoid” animated virtual characters which “move into the location of detected obstacles and guide the user towards the destination.”
The conclusion for the paper states “each participant confidently walked for 15 minutes in DreamWalker, which showed the potential of our system to make repetitive real-world walking tasks more entertaining.”
