It’s the NBA Finals and in the final ten seconds. You are sitting on the edge of your seat court side, brought to you on your couch thanks to virtual reality. Your team just called a time out to advance the ball. You turn to the bench eagerly to watch the coach talk to the players… or well you would have if the refs hadn’t just gathered by the scorers table in front of the camera. If only you could just lean around…
Well now you will be able to. NextVR has announced today that they have figured out a way to add full positional tracking to live 360-degree video using light field technology. This means that, for example, when a ref decides to hop in front of the camera that is set up court side, you will have the ability to look around him and see the action that is happening in front of him, just as if you were actually there.
This represents a huge leap forward for the technology. Adding full positional tracking suddenly the scene becomes far more lifelike. Fans of VR who have followed its development, may remember experiencing this jump when the Oculus Rift went from DK1 to DK2. The addition of positional tracking fundamentally changed the experience for the better and VR just does not feel complete with out it. Even the best 360 video content out there right now, 3D and all, feels a bit flat, there is a degree of dimensionality that is missing that messes with your brain a bit. It’s a bit like the uncanny valley, but for video. By adding positional tracking you are able to have a presence in the scene, which can be very powerful and really increases the level of immersion.
Immersive animated cinema has been gaining some traction with Oculus Story Studio, and fine animated content like VR Butts. The medium had a big advantage over its live action cousin due to positional tracking (and to the fact that for now, animation tends to look better in VR), but that gap is rapidly shrinking with this news.
NextVR accomplished this feat by utilizing light field technology. Light field technology works by capturing the direction light hits a camera as well as the intensity and color of that light. Then, using a proprietary technique, NextVR is able to take that information and build on-the-fly incredibly accurate geometric models of the entire scene. In a prior post about NextVR, David Cole – Co-Founder and CTO at NextVR – explained NextVR’s camera “senses the geometry of the world as a wireframe,” which allows it to then map the video directly to that frame creating a near-seamless 180/360 image.
With this announcement the purpose of that technology is far more clear. By creating a wireframe model to lay the video on, it creates a new way of displaying video; it’s 3D, but not in the way you’ve become accustomed to. “This innovation is a required next step toward creating the ‘holodeck’,” says Cole. “By incorporating dynamically-generated 3D geometry with ultra-high resolution stereoscopic video, we’ve created most the vivid and life-like VR experience currently possible.”
NextVR’s announcement didn’t stop at changing the face of 360 video, the company also announced that they will be supporting the family of volumetric display AR devices (meaning that NextVR’s content will work on the Microsoft HoloLens and the rumored light field display on the Magic Leap). This is a big first as well, as it means that NextVR is currently the only company that offers full support for “the full gamut of VR and AR displays coming to market,” including the newly announced HTC/Valve collaboration the Vive. The future of live streaming in VR just got a whole lot brighter and more immersive.
