Today, the majority of people try VR for the first time on a Samsung Gear or Google Cardboard and, more than likely, they watch a 360-degree video. That’s a shame really, because most 360 video experiences are terrible, especially when streamed over the Internet. I’m not referring to the content, per se (although there is A LOT of bad 360 video content), but the viewing experience itself. 360 videos are blurry, the files take forever to download and they lack motion parallax – which might cause motion sickness if you move too much.
The motion parallax issue won’t be solved until light field technology improves significantly (hurry up, Lytro). But resolution and streaming enhancements are definitely something we can tackle right away. To fully realize the resolution of today’s HMDs, a VR video needs 8K of total resolution, which results in 100% coverage of the ~1200×1200 ocular resolution per eye and yields a pleasant experience. The standard 360 video today is often delivered as only 4K resolution video, of which only 20% or so is seen at any given time within the Field-of-View of the HMD. If we tried to simply stream at 8K resolution, we would run into bandwidth issues and most HMDs would choke or blow-up (eg: they can’t play a 8K video). Also, a 3-minute 360 video at 8K resolution can be a whopping 1gb+ in size.
So, how do you stream high resolution content (8K) with low bandwidth thresholds? The answer is Field of View Adaptive Streaming (FOVAS): stream just what the user is looking at in high-res and the rest of the scene in low-res, saving bandwidth while maintaining optimal user experience. Both Facebook and a startup called Pixvana are using the approach to deliver VR videos that look great.
FOVAS is a core component of the new Steam 360 Video Player, debuting in early 2017. A partnership between Valve, Pixvana and Akamai, Steam 360 Video will allow VR filmmakers to publish their content to the Steam store. Utilizing Pixvana’s FOVAS player technology, along with Akamai’s CDN capabilities, Steam 360 Video will offer high-quality VR streaming to global audiences.
It looks like we will have to wait until early 2017 to see the full Steam 360 Video Player. However, I did have the chance to check out Pixvana’s FOVAS technology in-action on their new SPIN Player demo, for HTC Vive and Oculus Rift. (If you have a Vive or Rift you can try it yourself, here.)
The SPIN Player demo consists of several 360 videos. Before each 360 video starts, the type of camera and resolution of the video appear on the screen. The videos vary in resolution from 8K-10K and were shot with a variety of camera systems. As you turn your head in the environment, you can see the FOVAS algorithms at work. Wherever you look, the image gets sharper almost immediately (typically within 100-400ms). If you moved your head slowly, you probably wouldn’t even notice that the image is updating in real time.
The monoscopic videos I saw in the Pixvana SPIN Player were crisper than anything I’ve seen for streaming 360 video. The bandwidth savings were equally impressive – “The SPIN Player can stream 8K and 10K video at just 3 mb/s”, says Sarah Stumbo, a VR Software Engineer at Pixvana.
8K+ resolution does not solve all the issues with 360 video though. If the subject of the shot was more than eight feet away from the camera, their effective angular resolution can make things soft and seem out of focus. 360 video has a long way to go before we are walking around inside of movies and reliving experiences in a hyper-realistic way – but this is a much needed step in the right direction.
