Known worldwide for its contributions to the MPEG format — the compression technology used in MP3 audio files and MP4 videos — Germany’s Fraunhofer has recently turned its attention to the next frontier in media: virtual reality. After unveiling affordable VR headset microdisplay hardware last year, the company is now showing off next-generation video compression software using the new MPEG-OMAF standard, the first VR specification enabling 360-degree videos to stream over 5G networks.
Based in “significant” part upon Fraunhofer video compression technologies, MPEG-OMAF breaks wraparound videos into grids of tiles encoded at multiple resolutions. The explanatory image above uses red tiles to indicate areas that are being streamed at low resolution, versus normally colored tiles that are being streamed at high resolution.
Unlike traditional videos, which stream from servers at one user-selected resolution, these VR videos dynamically use high-resolution tiles where the viewer is currently looking, and low-resolution tiles for parts that are out of sight. As the user’s head position changes, the headset or display device requests a different mix of streamed tiles optimized for the user’s current focus area.
This trick enables the entire 360-degree video to continue streaming while devoting maximum detail to whatever the user is viewing. It parallels the recent use of foveated rendering to maximize real-time 3D graphics for VR users, guaranteeing that head-moving viewers will always be able to see something through their peripheral vision, even if it’s lower in fidelity.
International cellular standards organization 3GPP has adopted the MPEG-OMAF standard for 5G VR streaming, so it will likely underpin plenty of 360-degree virtual reality video streams — just like MP3 and MP4 defined prior generations of digital audio and video. Current 360-degree VR videos streamed over 4G and even Wi-Fi networks tend to suffer from low overall resolutions, across-the-board compression artifacts, and high latency, all of which the new standard and higher bandwidth networks could eliminate.
This article by Jeremy Horowitz originally appeared in VentureBeat.