If you were to stand near the person in the video above you would just see a man punching the air in an empty green room. Dylan Fitterer is in VR though and he’s basically dancing.
He’s bouncing back and forth to the beat of the music he hears — a song called Vortal Combat from Valve’s Half-Life series. He wields a pair of colorful shields and is deflecting sounds with each carefully timed punch. The sounds are arching toward him through space like flaming blue and red fireballs and his shields — his “audioshields” — are his only defense.
This gap between what a person experiences while in VR and what a person sees from the outside is a very big problem. It’s why I think so many people found themselves at least a little bit alarmed by this image of Facebook CEO Mark Zuckerberg strolling by unsuspecting journalists immersed in Gear VR.
The problem is bigger than simply photos that bring to mind George Orwell’s 1984. It’s the simple fact that, to the unaided eye, it’s impossible for others to see what a person in VR sees. This inability to communicate virtual experiences opens up VR early adopters to ridicule, as is evidenced by some of the negative reactions to the Zuckerberg photo.
More importantly, this inability to communicate VR experiences is a barrier to adoption. People all over the world interested in this technology cannot fully comprehend what they will be buying. That is, until the developers of Fantastic Contraption started working on their mixed reality streaming with the HTC Vive. A visual effects expert is taking the technology to the next level, using it to pioneer a way to make game trailers that truly show what a VR game feels like to a person playing it. Essentially, they use a camera and a green screen to sandwich a virtual foreground and background around a person’s real-world movements.
Now Fitterer, the developer of Audioshield, becomes the second HTC Vive developer to use this approach to present his game. It’s only going to continue from here as more developers figure out how to do this and, for the first time, people curious about VR will finally be able to see the effect a person’s actions in the real world have on their virtual environment.
This mixed reality capture holds enormous potential to spread understanding about VR and, with it, make it more likely people will make the leap and invest in hardware and software to have these experiences in their own homes. The more I’ve seen of this technology the more I suspect the way people use sites like YouTube and Twitch is poised for a complete transformation. These videos are only the beginning. In a couple months enthusiasts and developers will be flooding the Internet with mixed reality videos.
Lastly, as these videos start to proliferate, I wonder if they will aid adoption of the HTC Vive over the next six months more than the Oculus Rift. Oculus is shipping Rift with only an Xbox One controller for input and its Oculus Touch controllers aren’t expected to arrive until late this year. For its initial launch, the Rift is largely a seated or standing experience and it’s unclear if these videos will be as effective at showing the power of VR when you can’t reach out and interact with it.
