Head of VR/AR Design at Facebook, Jon Lax, stated on Twitter that Oculus Quest is “the end of our first chapter of VR”, and that “what’s next is where things really get interesting”.
I appreciate @fredwilson's measured take on building technology over the long haul. Quest is the end of our first chapter of VR. What's next is where things really get interesting. https://t.co/xa3gdcQo0S
— Jon Lax (@jlax) July 2, 2019
Lax’s Tweet was liked by two other high level Facebook execs, Andrew Bosworth (VP AR/VR) and Maria Fernandez Guajardo (Head of Enterprise, AR/VR).
Facebook considers the Quest, Go, Rift, and Rift S to all be first generation VR headsets. The recently released Quest is not considered to be a new generation, rather it simply brings the first generation experience to a standalone form factor.
The company has regularly showed off glimpses of its research towards next generation VR. Many of its job listings reference working on “our next-generation consumer product platforms that will provide breakthrough simulated reality user experiences“.
Here’s a quick rundown on what the company’s research and patents might be telling us about what Lax means by “what’s next”:
Varifocal
All VR headsets on the market today are fixed focus. Each eye is given a separate image, but the image is focused at a fixed distance from the lenses. This means that your eyes point (verge) towards the virtual object you’re looking at, but focus (accommodate) to the fixed focal length of the display. This is called the vergence-accommodation conflict. It makes VR feel less real, and causes eye strain and headaches. It can also make near objects look blurry.
At Facebook’s annual F8 conference in May 2018, the company showed off a prototype headset called Half Dome. Half Dome is a varifocal headset with eye tracking, which mechanically moves the display panels to change the focal distance to be the same as the distance to the virtual object you’re looking at. This solves the vergence-accommodation conflict.
At Oculus Connect 5 in September 2018, Chief Scientist Michael Abrash showed a software level improvement to Half Dome called DeepFocus. DeepFocus is a deep neural network which renders realistic focus blur for VR. So just like in real life when you look at an object close up, your view around it will be blurry.
Why it matters: varifocal headsets with realistic blur could allow for extremely long VR sessions without eye strain or visual discomfort, and make VR feel more real.
Eye Tracking (And The Things It Enables)
Almost all consumer headsets today have a field of view of roughly 100 degrees horizontal, with a resolution of roughly 1500×1500 per eye. Widening the field of view reduces the effective angular resolution, because spreading the pixels across a wider area means there are less in a given part of the area.
For the kinds of resolution needed to support a wide field of view, foveated rendering will be needed.
The human eye is only high resolution in the very center, as you can notice by looking around your room. VR headsets can take advantage of this by only rendering where you’re directly looking in high resolution. Everything else can be rendered at a significantly lower resolution. This is called foveated rendering.
Eye tracking also allows for the lens distortion to be dynamically corrected based on the exact position of the user’s eye relative to the lens. This could allow for optical engineers to design lenses that are prone to varying (but predictable) distortion throughout the eyebox, which could enable wider field of view lenses.
Why it matters: Eye tracking will allow for headsets with much higher resolutions and a wider more immersive field of view.
Finger Tracking
Today’s VR uses controllers for all tasks. These devices resemble a game console controller split in half, but with a grip trigger and more precise haptics.
Facebook and other major companies in the VR space are researching the ability to use your hands freely in space.
While finger tracking won’t replace hand controllers for gaming, but social VR and passive experiences may be a wider use-case for non-gamers.
Why it matters: You won’t need to put on controllers for a purely social or passive VR experience.
Face & Body Tracking
With today’s consumer social VR, users have primitive cartoonish representations of themselves or non-human creatures.
But what if you could have a photo-realistic representation of your friends in VR? That’s what Facebook is working on, using IR cameras in the nose gap of the VR headset and machine learning.
The company is now even talking full body tracking using a single external sensor, using a muscular simulation model.
There are existing ways to do body tracking in VR, but are either low quality or involve strapping hundreds of dollars of “pucks” to yourself, which each have to recharge.
Why it matters: You’ll be able to play VR with your friends as photorealistic avatars.
Room Sensing & Meshing
Today’s VR headsets allow you to manually define a playspace free of obstacles. This means you’ll typically use the largest open space in your home.
But often that still feels limiting, and VR users are still wary of bumping into their surroundings.
Future VR headsets could be able to bring the geometry of room into VR in a more natural way, blurring the lines between real and virtual. Your desk could be represented as a sandbag in VR, or your lampside as a large plant.
Why it matters: You’ll be able to use more of your physical space and be situationally aware in VR.
PC Wireless
The Oculus Quest is already wireless, but its PC VR counterpart Rift S uses a cable. Facebook is known to be researching wireless technologies. In 2016 at Oculus Connect 2, Chief Scientist Michael Abrash predicted that by 2021 PC VR would be wireless.
Other companies like HTC have already shipped wireless PC VR adapters, but at a price of $300. Facebook’s research seems to be aimed at lowering the cost.
In October 2018 the company filed for a patent describing a system where positional tracking is used to send the wireless data in a narrow beam, rather than a unidirectional antenna. This could significantly reduce the energy requirements, and cost, of wireless VR. Last month, Facebook was granted the patent.
Why it matters: you’ll be able to rotate and move freely with the graphics of PC.
What do you think is coming from the future of VR headsets? Let us know down in the comments below!
