CES is a strange conference. Across endless aisle after endless aisle a mixture of companies looking for suppliers, distributors, press or investment present wares in hope of making the year ahead one of major success for their business. The conference always presents a roadmap for the future of technology and it has been that way since the ’60s. Seeing the actual pathway ahead, though, is difficult to find through the manufactured hype and “best of” awards.
So I understand the comments we see from folks who want to know what was “good” at the conference who might be confused by vastly differing reports and reactions. Is the reporter able to describe what actually happened in their demo, or is their vocabulary lacking in describing if something went wrong? Do they gloss over issues? Are they sold the promise of the technology if bugs are worked out in a couple more years?
With this in mind, and the joy I know is already delivered so rapidly by Valve Index at home and Oculus Quest in a carrying case in my hotel room, I found myself largely unimpressed by almost all the AR on display at CES and much of the VR too.
“There is no doubt Oculus Quest has shown what the right mainstream consumer device is and while there was nothing immediately that will give the Quest a run for it’s money, I did try the new Qualcomm reference design and some other tech that suggests next year there should be many more all-in-ones that should get consumers and enterprise excited for the near future of VR,” AR and VR investor Tipatat Chennavasin explained in a direct message.
There were still hints of the future at CES 2020, but I need to address the difficulty and cost involved in polishing these technologies to the point of actual usefulness for businesses or mass appeal to consumers. That’s what this post is about.
So read on to understand what’s truly going on in VR and AR.
VR: Advances In Field Of View, Controller Tracking, And Headset Size
VR headsets at CES 2020 demonstrated advances on several fronts. None of these were entirely new, or entirely perfect, but all point to possibilities for the future.
Smaller Headset Designs
Several companies at CES demonstrated pancake lenses which dramatically reduce the physical size of VR headsets. These also generally featured a reduced field of view compared with current designs.
The one I found most interesting was Panasonic’s “VR Eyeglasses” which combined an incredibly light design with an innovative lens separation adjustment. Read my story about the Panasonic glasses.
While Pico showed a similar conceptual design alongside their Neo 2 and Neo 2 Eye I didn’t see any physical adjustment for lens separation on their glasses design. So the Pico glasses design made me extremely uncomfortable in a matter of seconds. The distance between my pupils is wider than many and I usually need that adjustment (which is found on both Index and Quest) to have a comfortable experience in a VR headset.
Huawei’s slim VR Glass design was also demonstrated by Nolo combined with their front-facing 6DoF tracking system as an add-on. BoxVR worked reasonably well in this configuration, but the $500 VR Glass plus Nolo kit only tracked my hands while facing forward. Under no circumstances would I be interested in a VR headset in 2020 or beyond that doesn’t let me turn around.
Combining pancake lenses with microdisplays is nothing new of course — eMagin showed this off in 2015 before even the release of the Oculus Rift and HTC Vive. Since then, microdisplay suppliers have claimed that headsets using this kind of display system are just around the corner. But now we’re finally starting to see major consumer electronics companies explore this idea, which may mean it is now closer than ever to being in an actual product.
Expanded Field of View
Some VR headsets at CES 2020, like the nearly $8000 Xtal and all the Pimax systems, featured an expanded field of view compared with headsets like Oculus Quest and Valve Index.
Xtal demonstrated an impressive $50,000 motion simulator with their headset integrated into a flight helmet with its wide field of view and 4K per eye resolution offering incredible sights flying in an F-18 over Dubai. While enjoyable, I noted some distortions at the edges of the lenses.
The Pimax Demo
Pimax frequently makes promises then delays delivering on them again and again. We’ve also heard numerous reports of problems with their hardware ranging from minor to major.
While my colleagues found Pimax demos at previous events, my first Pimax Demo was at CES 2020 and the first time I put a Pimax 8KX on my head I tried Boneworks.
The first thing I noted is that content popped in at the edges of the display a second or two after the rest of the scene. When I mentioned it, the demo (said to be running from an RTX 2080 TI) was switched to “parallel projections” and that solved the problem, but within about five minutes the right eye started blinking to black every few seconds.
I asked for another 8KX to try with Boneworks and this one didn’t have the problems I saw before. Its wide field of view invited me to look around a little more and the high resolution of the panels were certainly nice, but I found myself also constantly distracted by the significant distortions and bending of the scene at the edges of the lenses — more significant than the distortions I saw with Xtal. My right hand also lost tracking with 8KX, but that’s not entirely unusual for SteamVR Tracking in convention settings.
We’ve heard some people with Pimax headsets artificially reduce the field of view of the system through software adjustment to avoid these distortions while still enjoying a relatively expanded field of view. My colleague Tatjana returned to the Pimax booth later in the CES week to try both the 8KX and Artisan. Aristan is Pimax’ newest headset which ends up priced comparable to Valve Index when you include controllers and base stations. She played Boneworks and Fruit Ninja and found herself sick to her stomach afterward. I, too, left the Pimax booth a bit woozy.
There are Pimax defenders out there and those who will find the expanded field of view a good fit for their strong stomachs and top of the line graphics cards. Indeed, I got sick repeatedly over my 12-hour play through of the Boneworks campaign with Valve Index, but didn’t get uncomfortable as quickly as I did with that Pimax demo at CES.
Surely a wider field of view is something we want in future VR headsets but I remain unconvinced we’ll see it at an affordable price other than from a major platform like Facebook, Valve, Sony, Google, or Apple with large teams developing both optics and eye tracking technologies which could make these devices more comfortable.
Overall, the repeated delays for Pimax products and less than stellar impressions at CES 2020 means that we will keep using words like “claim” to describe plans for products pitched by this company.
Alternative Tracking Systems With Pico
The Pico Neo 2 and Neo 2 Eye, priced at $700 and $900 respectively, featured occlusion-resistant electromagnetic tracking. In the Neo 2, for example, I played Angry Birds and was able to pull back the slingshot while keeping my head turned away from the controllers. This is something I can’t do in an Oculus Quest. The Neo 2 also balances weight to the back better than Quest’s front-heavy design.
The Neo 2 Eye demonstrated Tobii’s eye tracking for the first time I’ve seen in a standalone design. Also for the first time I’ve seen, the headset tracked my eye movements without calibration. Typically, Tobii asks each wearer to point their eyes at a few dots at different corners of the scene to calibrate, and I was told that option is still available. The eye tracking without calibration seemed to work alright — I was able to target enemies in a scene by just pointing my eyes at them — but when I pointed my eyes downward it seemed to lose tracking.
I’d need more time with the $700 Neo 2 to say more, but I found it to be the only Pico device I’ve ever tried I might actually be interested in using. However, Neo 2 is only sold to businesses.
AR: Tracking Lacking, Power Problems And Severe Field Of View Limits For Years To Come
During the course of CES my colleague David Heaney described simply something that’s hard to grasp about the market for AR head-worn gadgets: Recognizing occlusion in your environment is to AR what 6DoF tracking is to VR.
This is to say that if an AR headset with see-through optics inserting digital content into your real-world environment cannot reliably understand when a person or object blocks simulated content from your view, the illusion meant to be provided by the hardware is broken. The same is true of VR headsets that do not track the position of your head. For example, if you wear an Oculus Go VR headset and decide to lean forward or stand up, the illusion of VR that you were enjoying is instantly broken.
Discomfort or confusion often follows in either case of severely limited VR or AR, and this represents a death sentence for the hardware. Failing to provide 6DoF tracking, or recognize occlusion, is so uncomfortable, I believe some headset wearers are discouraged from wearing the hardware ever again. Of course, some can learn to work around this limit.
There are fans of Oculus Go who understand its limitation and use the device as a low-cost personal media viewer. Nreal AR glasses, for example, provide a relatively large field of view for a similar genre of content. Still, without a complete and constantly improving understanding of the environment around the wearer, AR glasses like the $500 Nreal (and there were a lot of copycats at CES) are likely to consistently fail this test. Without passing this test walking out in the real world, the appeal of these kinds of AR hardware platforms is minimal to developers, to businesses, and to consumers.
“I would agree that for AR glasses to be truly useful and game changing, they would need not only solid 6DOF tracking (which none of the slim AR glasses I tried had), object occlusion, and object permanence but also really intuitive and reliable 6DOF input which was not also shown,” Chennavasin wrote. “I appreciate the effort being done by all the companies but it still looks like consumer AR is still years away.”
This roadblock for AR relates to two others — power consumption and field of view. Most AR glasses fail to deliver anything close to the amount of digital content to your eyes seen through most VR headset designs. This means that even in HoloLens 2 and Magic Leap 1 — two dedicated AR headsets — you need to move far away from the digital content to get a full view of it and truly enjoy the sense of immersion it brings. Again, some can get used to this limitation but the bulky size and high price of these AR systems ($2,300 for ML1 and $3,500 for HoloLens 2) also prevents them from appealing to consumers.
Lastly, building the most detailed map of your environment typically means on-board cameras need to keep scanning the room to keep the map updated. Using those cameras drains power. One of the key reasons the first generation of phone-powered VR headsets were retired so quickly is because those headsets drained power from a device you needed to last the day for other tasks. So do phone-powered AR systems like Nreal have a shot if they drain phone battery power just like the already-defunct Gear VR, and are used for the same types of content as the $200 Oculus Go starting at more then double the price?
“I agree that power is a (possibly the) key constraint for HMDs. It’s one reason why active depth cameras are not the best solution for occlusion, and we are seeing 6D’s approach of using low power RGB sensors, combined with efficient Neural Net co-processors providing solutions to occlusion,” explained Matt Miesnieks, the CEO of spatial recognition company 6D.ai, in a direct message. “The first low cost AR headsets do need refinement, but we have some visibility into roadmaps, and as that refinement is primarily driven by software, we will see improvements very rapidly.”
With Nreal in particular I saw software that needed significant refinement, both in tracking and stability, to become more usable.
Short Term Potential Advances In AR And VR
CES 2020 confirmed to me what we’ve already seen elsewhere.
Tilt Five’s forthcoming consumer AR system uses a novel retro reflective method delivered as a game board that tunes for both a relatively wide field of view and the specific case of tabletop games. These aren’t glasses you’ll take out into the real world and since you are localized to the game board, occlusion isn’t as much of a concern. Tilt Five cost $879 during its recent Kickstarter for a three-pack of glasses and there are extensions to the board to expand it vertically to provide more height to the augmentation effect. Altogether, Tilt Five represents one of the most promising short-term AR projects we’ve seen. I contacted Tilt Five CEO Jeri Ellsworth and she suggested there may even be a way to charge companion phones, with a a pass through hub, while also powering the glasses.
“We have a huge advantage by containing our system.,” Ellsworth wrote in a message. “We also limit the compute on the device…by doing re-projection and in headset tracking which saves power.”
Shipping a hardware crowdfunding product isn’t easy, so there’s a lot to prove for Tilt Five, but after CES 2020 I’m still left thinking Ellsworth’s company remains the most interesting AR project approaching the consumer market in the near-term.
When it comes to VR, we know the future of the medium features wireless, higher field of view, more tracking of body features in a wider range of conditions and smaller, more well-balanced headsets. But taking all the best pieces of CES 2020 VR systems — Pico’s better balance, Xtal’s wide field of view, Panasonic’s slim design — and then polishing these features, adding on more and supplying them with content is an entirely different level of challenge than preparing an interesting demo.
I’m talking about spending billions of dollars and there are only a few companies positioned to make that sort of investment in the next couple years.
Correction: An earlier version of this article incorrectly listed the software adjustment which fixed the first problem noted in the Pimax demo. It is now correct.
