Microsoft May Leapfrog Facebook and Google in VR With HoloLens Tracking

by Ian Hamilton • October 26th, 2016

Microsoft’s play for command of next generation personal computing can be seen today with the announcement of a series of Windows-powered VR headsets starting at $300.

The headsets planned from HP, Dell, Lenovo, Asus, and Acer will use what’s known as “inside-out” position tracking. The system is already used on the $3,000 HoloLens, an advanced augmented reality system, but with it employed on a low-cost VR headset Microsoft may have leapfrogged efforts by both Facebook and Google, powered by Windows.


“Most consumers will experience Windows Holographic for the first time through VR, not through HoloLens,” wrote Raven Zachary, founding partner of mixed reality startup Object Theory, in an email. “This means shifting from being a HoloLens-focused mixed reality company to a Windows Holographic focused mixed reality and virtual reality company. HoloLens will continue to be at the core of what we do, but Windows Holographic is the larger and more diverse ecosystem that HoloLens exists within.”

Head tracking is an essential part of all mixed reality technology, both VR and AR, and the so-called inside-out variety in a VR headset makes set up dramatically easier. A computer needs to know your head’s location at all times with an enormous degree of accuracy to make you believe objects are floating in the room, or to make you think you are somewhere else entirely. Microsoft is essentially promising headsets powered by Windows that will always know where they are, but starting at much lower prices than existing headsets while also being much easier to set up.

“As we have built up expertise and credentials in Windows Holographic, we likely will not be working with other platforms in the foreseeable future,” Zachary wrote.


An example of the Windows Holographic user interface.

A Rift from Facebook-owned Oculus or a Vive from HTC can track your head throughout a room, but these “outside-in” systems require careful placement, or mounting, of sensors around the perimeter of the room for a full experience. If you want to take VR into another room, for example, you have to find places for the sensors again. This is not the case with the headsets on the way from Microsoft’s partners.

“It’s reasonable to assume Microsoft has licensed the ASIC [Application-specific integrated circuit] they developed for HoloLens to their partners to make inside-out VR cheap and plentiful,” Mark Pesce, a 3D Web and VR pioneer, told Upload VR. “The strategy has become clear. Low end VR on Xbox and these new headsets, all the way through high end mixed reality on HoloLens. Microsoft has positioned itself across the entire space, and will drive mixed reality into the enterprise in 2017.”

John Carmack, the chief technology officer at Oculus, told me at the company’s second developer conference in September last year that inside-out tracking in a mobile headset “is a problem that I want solved right now. I wish somebody had spent all of this last year on it.” Microsoft, it turns out, was only six months away from shipping such a solution in HoloLens.


The Santa Cruz inside-out tracking prototype from Oculus.

In stark contrast to the comments made by Carmack only a year before, at this year’s developer conference Facebook made clear its future for the Oculus platform is a standalone headset with this same kind of inside-out tracking. Microsoft, however, may be one step ahead.

Last month it was revealed Microsoft hired USC’s pioneering mixed reality expert Mark Bolas. In that article I noted:

Microsoft already demonstrated a solid inside-out tracking system with its $3,000 HoloLens, but getting that kind of robust tracking system to work with other kinds of headsets, including wide field of view VR, is the place Bolas might be able to offer assistance.

With today’s announcement, we have clear confirmation this is Microsoft’s direction.

Going forward, the same manufacturers developing these headsets in partnership with Microsoft might also create smaller screen-less PCs, like backpacks or pocket computers, that make Windows-powered VR completely portable.

I wouldn’t count out Facebook or HTC’s partner Valve just yet, however, because Microsoft revealed nothing around input with its announcement today — leaving wide open the question of how experiences inside these headsets will be controlled. Rather, Microsoft seems to be describing these headsets as “accessories” for Windows rather than independent platforms. Voice input is something Microsoft can do, but hand-tracking is a separate and enormously difficult problem despite companies like Leap Motion working hard at it. In fact, outside-in tracking systems capable of seeing more advanced controllers, like the upcoming Oculus Touch and those we saw shown from Valve recently, might still be a route toward VR becoming its own independent platform.


Very early prototype controllers from Valve that simulated gripping and dropping objects.

Then again, Microsoft might still have more tricks up its sleeve. In the meantime, for Windows-focused developer Object Theory, a home-grown solution to input might be a possibility.

“We’ve experimented with input devices communicating over Wi-Fi to the HoloLens and sending real-time X,Y,Z coordinates in Unity,” Zachary wrote. “If we had a client that needed precision hand controllers, we’re pretty sure we could build a solution for them either through our own custom code running on smartphones or utilizing third party hardware. That said, I’d like to see Microsoft provide a solution here. They showed a 6DOF controller in their Intel Developer Forum concept video a month or two ago for the Windows Holographic Shell, and I will be curious to see if they productize this or just publish the specs for the third party hardware partners to release their own.”

Tagged with: ,

What's your reaction?
  • Rook

    nice writeup… hunger for more info

  • mellott124

    Thought Bolas went to work for Microsoft, not Oculus. Typo?

    • Ian Hamilton

      yes, typo. My apologies. I just updated it.

  • Joe

    I think you meant that Microsoft hired Bolas, not Oculus.

    • Ian Hamilton

      Sorry about that typo, it is fixed.

  • Alen Ladavac

    A correction – Vive doesn’t use outside-in. The Lighthouse devices are positional beacons, not sensors (hence the name). The headset and controllers are doing inside out tracking, using the infrared laser markers provided by the Lighthouses.

    • Matthew Ong

      True, but it’s a much much easier kind of inside out tracking and not really what people mean when they use that term typically.

      • Nicholas

        The Vive is what is what I’d call inside-out tracking. It’s just not markerless inside-out tracking, which is what they should be calling it.

        Bundling it in with the Rift’s outdated tracking and calling it outside-in is just plain wrong.

      • Alen Ladavac

        What people mean when they say inside-out, is that the HMD tracks itself, instead of an external camera tracking the HMD. That’s the only important difference. The article says that this is good because this can move from room to room, while outside-in tracking cannot. Well exactly – that’s why Lighthouse tracking could move between rooms (as long as there is always at least one Lighthouse in the view). The only reason why the current systems cannot move between Lighthouses is that the software is not written to support it.
        Imagine Lighthouse as a kind of a GPS system. The difference is important for privacy and moveability. With Lighthouses, only your device knows it’s position – no one on the outside is tracking you (privacy) and you can have a backpack and move around without a physical (or even wireless) connection to the Lighthouses (moveability).
        Lighthouses are just advanced visual markers which make inside-out tracking much cheaper and much less resource-intensive.

  • Leonardo Phillips

    I see a problem with this technology: the resolution of Structured Light tracking systems depends of the distance from the emitter and the camera. In an HMD this distance may be several centimeters but in a motion controller this may be difficult.

    • lovethetech

      Microsoft has 30+ million Kinect with spatial tracking usage data.
      Hololens is the holy grail of spatial tracking so far in the consumer and as well as in lab products in this price range.

      I think they know better than arm chair experts like us.

  • Ian Goode

    Will Microsoft work with openVR or will they have their own proprietary APIs and platform? I can’t see any mention of content partners or any kind of software side to this bar Windows Paint.