Google: Wireless Positional Tracking “Solved”, But Heat Still A Problem For VR

by Joe Durbin • November 1st, 2016

Last week, at a Google press event UploadVR had the chance to speak with Google Tango’s director of engineering, Johnny Lee. Tango is a software/hardware solution that allows smartphones and other mobile devices to understand their location in 3D space and translate that information into powerful applications and programs. According to Lee, this technology is also the key to solving one of the biggest problems in the burgeoning virtual reality industry: inside-out positional tracking for wireless VR headsets.

“We’ve solved it here,” Lee said, gesturing at a Tango-powered, Lenovo Phab 2 Pro smartphone. “As you can see [inside-out positional tracking] clearly works on this phone…We’ve even had people strap a tablet sized device with Tango built in into a custom VR headset and the positional tracking worked just as well as it does here…The only thing really holding us back right now are the thermals. Right now, phones just get way too hot if we ask them to run positional tracking and split-screen stereoscopic image at 90 fps…As Daydream matures and Tango continues to improve, the sophistication around tracking will also evolve over the next 2-3 years.

Lee declined to comment when asked when exactly people can expect to see a commercially released VR headset with this kind of capability.

Tango’s director of product, Nikhil Chandhok, concurred with Lee’s assertions that, with Tango, inside-out positional tracking for VR is possible, but there are still hurdles to cross: 

“We have it working. We can do inside-out tracking with six degrees of movement and we can do it today. There are just some concerns we need to address first. For example: how do you keep people safe when they are walking around freely with a headset on?…The VR use cases are just much more demanding than what we’re doing right now, however, there will be more interesting use cases in the future.”

Today’s high-end VR systems — the Oculus Rift, Playstation VR, and HTC Vive — all use what’s called “outside-in” positional tracking. This means that they all require some sort of camera, laser or sensor that live outside of the headset itself in order to understand their positions in space. Without these outside components, one could not lean into objects in an experience, and would be limited to the left, right, up, down, tracking currently seen in mobile VR headsets such as Google Cardboard or the Samsung Gear VR.

Inside-out positional tracking would allow virtual reality to become truly wireless and enable a whole host of incredibly powerful use cases. It is the latest “holy grail” for the VR scene, with most of the major companies scrambling to be first to market with the world’s first wireless, position-tracked HMD. At this year’s Oculus Connect conference, the Facebook-owned company showed off a prototype code-named “Santa Cruz” that utilized a form of inside-out tracking separate from Tango, and Microsoft is using its HoloLens tracking to bring the tech to wired VR headsets. Qualcomm and Intel are just a few of the companies trying to solve this problem as well.

The race is on for positional tracking supremacy.

Tagged with: , , , , , , , ,

  • Wollan

    Time-of-flight IR (i.e Project Tango and Kinect) has usually had a latency challenges. Would like to hear more details here. And in this video you can see the virtual city shift a bit across the table.
    If both are amongst the “concerns they need to address” then I believe the remaining 10% of the project might turn into 90%.

  • Pistol Pete

    Man VR mobile tech is moving fast! Great things coming in the near future.

  • Sean Lumly

    Based on the video, using the word “solved” seems generous. While the table surface is tracked, it is far from stable, as there is a tremendous amount of “wobble” in the composited 3D image relative to its backdrop — wobbling that would be unbearable (to me and, I’m sure many) in VR.

  • Joe Banes

    Didn’t Oculus already “solve” this with Santa Cruz. Not only is it wireless and has apparently great inside out tracking but it is wireless and from what we know right now self contained. Though admittedly it might have been connected wirelessly to a computer. Also consider project Alloy and Qualcomms reference model. This hardly seems like something Google solved. Or am I missing something key here?

    • Google started years ago. Tango was a concept everyone was wondering where it went to. When I saw Alloy, my first idea was “why Google Tango has not made this before?”

      • lovethetech

        Tango was developed and fine tuned using the Alloy’s parent , the Realsense from Intel.

    • Doctor Bambi

      You have a point there, but Santa Cruz is really a different beast, a headset strictly for VR/MR experiences. Project Tango first and foremost, is computer vision for your phone. The fact that it could also serve as positional tracking in VR is more of a bonus feature. It’s just nice to hear confirmation from the devs that this will in fact work for such a use case.

    • DougP

      Re: “Didn’t Oculus already “solve” this with Santa Cruz”
      It seems that HTC & TPCast are the ones who’ve *already* solved this, as in a product that’s available & not just demo/prototype kit.

  • Buddydudeguy

    Still just a cell phone strapped to your face with very low end mobile hardware. Let me know when this tech is in the context of a PC HMD.

  • wheeler

    I’m skeptical of all of these companies claiming to have “solved” inside-out tracking. E.g. If it’s PSVR quality, the problem is definitely not solved. Have they used the systems with experiences that require high precision? For example, a game where you have to look through a scope? Because even the sub-mm accuracy of the lighthouse tracking is just barely good enough for a game like Onward, and FPSs will be what all of the kiddies want to play.

    • lovethetech

      Hololens solved and is the “Holy Grail” of spatial tracking and spatial mapping. Check it out if u live in big cities. Thanks to every buyer of Kinect.

  • c.j.d.s.

    This doesn’t make any sense. Inside out tracking is solved with Microsoft Hololens and on Structure IO. SLAM is by no means a “solved” problem anyway. I’m pretty sure if Google can do SLAM in a meaningful way it would be much bigger than this. And current tracking technologies have already been on portable devices for years

  • If their inside-out tracking was good enough to minimise the sim sickness you get when positional tracking is absent then it would be a fantastic feature .
    They shouldn’t wait until they’ve solved the hardest possible scenarios

  • Nicholas

    The Vive uses inside-out tracking (albeit with laser markers). Wish these articles were more accurate.

    • lovethetech

      When the authors are fans of “A” platform, that’s all we get. They do not care about end customers experience.

    • Serpher

      The trick is to do the same without laser markers.

      • Nicholas

        It’s not really a trick, just a lot of video processing. Not surprised they’re running into heat issues. I still think the lighthouse system is elegant and efficient by comparison without the need for raw video feeds. They should really be focusing their efforts into removing the tether instead (self-contained phone experiences don’t count… ).

    • what are you talking about?! The Vive is useless without the base stations. A true inside-out headset will not require any.

      • Nicholas

        “Inside-out” refers to the sensors behind placed on the headset/controllers, which is exactly what the Vive does. What you’re thinking of is markerless inside-out, which does not rely on positional markers in the environment. I’m not sure why they think this is a problem that urgently needs solving, compared to the HMD tether which is a far bigger annoyance.

        • DougP

          Re: “problem that urgently needs solving, compared to the HMD tether which is a far bigger annoyance”
          According to HTC, the HMD tether problem has been solved.

  • silvaring

    This makes me think that the whole idea of Qualcomm making a standalone headset with positional tracking is due to the thermal issues from phone based solutions. 2 years away though? I don’t see why it would be so long if Qualcomm releases its reference spec this year as it has originally planned.

    • lovethetech

      When the authors are fans of “ONE” platform, that’s all we get. They do not care about end customers experience.

  • Mobile VR will continue to be slightly gimmicky for the next 3-4 years. The newer GPUs for gaming rigs are barely getting to a point that support 90+fps at decent resolution. Knowing the product lifecycle of mobile phones and chipsets, I would be shocked to see anything in mobile approaching a GTX970 or 1070 for a few years, especially when having to compute positional tracking. Daydream is a step in the right direction but Qualcomm’s product roadmap is probably a bigger indication of market maturity.

  • Serpher

    Tracking without external cameras is cool, but the real problem lies in transmitting 90fps 4K content over wireless network with very low latency. Mobile solutions are not up to par with PC power.

    • Adrian Meredith

      foveated rendering, that way it only needs to be 4k over a small area

      • Serpher

        You’re right. FOVE is an essential part of optimizing video stream.