Oculus Chief Scientist VR Industry Predictions: From 2014 to Now

by Jamie Feltham • October 18th, 2016

It may only be two years old, but Oculus’ Connect developer conference has already established some cherished traditions. Many look forward to John Carmack’s annual assault on developer’s brains with an information overload, but Chief Scientist Michael Abrash’s own talk, which usually follows the primary keynote, is also not to be missed.

This year, Abrash made some bold predictions about what VR headsets will look like half a decade from now.

abrash

Abrash’s 2021 predictions for VR, presented at Oculus Connect 3 in 2016.

At the start of his talk, however, he also mentioned that he’d made some predictions back at the first Oculus Connect in 2014, which had largely come true.

So what exactly did Abrash predict? What was he right about? What’s still to come? We stepped back in time to take a look and found four statements the head of Oculus Research had made that are still an important part of the conversation today.

Prediction: VR will spur a massive acceleration in graphics

screenshot-2016-10-18-at-4-49-53-pm

What he said: “The reason that graphics is going to have to up its game in a hurry is that because VR engages so much more of the perceptual system than a monitor does, it’s held to a much higher standard. Parallax, accurate positioning, wide field of view and stereovision together provide vastly more information to the perceptual system, which responds very strongly to VR done right and complains just as strongly about VR that isn’t quite right.”

What he means: With VR, you can’t accept any substitutions. That’s why we need graphics that can cope with the demands of imitating real life, be it seeing more of the world than you’re used to with a traditional first-person shooter, or running top-tier games with the stresses of stereoscopic display placed upon them. VR is a demanding process and it’s on GPU manufacturers to provide hardware for consumers that provide believable experiences not just in visual fidelity but also framerate and performance.

Was he right?: Absolutely. In the months following the release of both the Oculus Rift and HTC Vive both Nvidia and AMD have upped their game with more powerful GPUs, like the GeForce 1080 and more affordable offerings like the Radeon RX 480. There’s still a long way to go, but we’ve already made great progress.

Prediction: Eye-tracking will play a big part in the future of VR

screenshot-2016-10-18-at-5-55-10-pm

What he said: “Eye tracking will almost certainly be a part of the future of VR. We wouldn’t actually draw anything close to 32K by 24K pixels [on a display], we’d take advantage of the steep drop in resolution away from the fovea, the tiny high-resolution area of the retina, to render only as much resolution as the eye can detect. But, of course, we’d need to know where the fovea was pointed every frame, which means we’ll need to integrate eye-tracking with new display, optics, GPU, and rendering technology in order to make a truly great VR visual experience possible.”

What he means: Foveated rendering. Abrash himself went much more in-depth on the idea at this year’s event — but it’s interesting to see just how early on he had conceptualized it. Essentially it refers to using eye-tracking to pinpoint where a user is looking on a display and then only fully rendering the area the retina is focusing on. Less detail is rendered where you won’t notice it, freeing up resources for other tasks.

Was he right?: Yes, but we’re not there yet. Companies are starting to dabble with foveated rendering but we only have eye-tracking in one VR headset, FOVE. Now Abrash is predicting we’ll need a solution far more advanced than what’s currently available to achieve big gains in VR quality.

Prediction: Social VR will be a core type of experience

screenshot-2016-10-18-at-5-22-24-pm

What he said: “Once we have virtual places to be in, we’ll want to see ourselves and others in that space. Sensors that can capture the inter-personal cues humans key off of: head-pose, eye-movement, facial expression, hand gestures, body posture and movement, and then map them onto avatars in real time will enable social interaction in these virtual spaces.”

What he means: Experiences in which you can meet up with other players and believe they’re standing with you in VR environments is going to be hugely important to the future of the technology. We’ll want to share our amazing life-like moments with others in real time, just as we would when we meet up in real life.

Was he right?: Abrash’s vision came to life at OC3 this year when Facebook demonstrated its own social VR experience that used Rift and Touch to deliver the expressions he mentioned two years ago, at least in a cartoon-ish fashion. We’re far off from doing this photo-realistically, but what we have now is still pretty convincing.

Prediction: VR will grow to include all senses

screenshot-2016-10-18-at-4-19-08-pm

What he said: “VR is about driving the perceptual system, the more of it the better. So, eventually, it will involve all the senses: audio, tactile, balance, kinesthetic, maybe even smell and taste. The entire body will become the sensor, not just the eyes, and the display will consist of not just pixels, but of everything that drives perception.”

What he means: Headsets are just the start. VR has a long list of senses to conquer before we can truly consider it the “alternate reality” that Abrash predicted it would become as he wrapped up. We want to walk into rooms and feel the surfaces of desks and sofas, smell the sea air and sand in our toes as we walk along the beach and, in time, maybe even taste pizzas we grab infinite slices of as we sit down to watch a movie with friends halfway across the world.

Was he right?: He probably will be, one day. Work in haptics and audio also has some way to go to effectively replace our natural equivalents, and the kinds of technologies that would replace taste and smell may not even exist yet. As they’re solved in the decades to come, however, we’ll have VR that is indistinguishable from real life.

Tagged with: , , ,

What's your reaction?
Like
Wow
0%
LOL
0%
Dislike
0%
  • AcroYogi

    > in the decades to come, however,
    > we’ll have VR that is indistinguishable from real life

    might want to amend that. in the *centuries* to come…

    acceptable as a substitute for real life? perhaps. some people that choose to live in VR 24/7, including sleep? perhaps. *indistinguishable* from real life? No chance in the next 50 years.

    Put another way, if there is a time, say around 2112, when VR becomes truly indistinguishable from RL, who’s to say that that didn’t already happen a few millennia ago, and we’re all just locked in? Sword Art Online? 😉

    • Midhav

      If we have 32K screens and the realistic graphics of that timeframe we’ll have worlds that are visually indistinguishable. If we have the technology to adequately render smell and sound, it’ll do the same for other senses. If we have omnidirectional treadmills and other apparatus, it’d be locomotively indistinguishable. Only the headset would add some sense of non-immersiveness.

      All of these are achievable within the next 20 years. The next 50 years? Advances in nanotechnology could permit virtual realities to be as real as our own. If AI were to reach superhuman general intelligence (as predicted to occur over the next 50), growth in such a field as NT is plausible, and would be immeasurably world-shaking.

      I’m looking forward to that and hoping that I’m absolutely right.

    • 50 years is a short eternity if we manage to keep riding the exponential train.

  • jimrp

    This is not a prediction. This has been known if VR ever got a foot hold in the door (correctly) all of these things would happen. Its an 80’s kinda thing and im an 80’s kinda guy. 🖒. Its up to the individual for responsible use. Not a community or government to hold your hand. Though yes on the government side they will try to use this and not for the good.

  • Gerald Terveen

    okay … this has a promising headline and yet is one of the most “pointing out the obvious and he nailed it” articles I have seen.

    PS: maybe next time pick some prediction that are actually challenged 😉