Nreal Light AR glasses will get controller-free hand tracking in a near-future software update, thanks to a partnership with Qualcomm-backed Clay AIR.
Nreal Light
Nreal is a China-based company founded in 2017 with the goal of delivering lightweight consumer AR glasses before the major tech companies. Their first product is called Nreal Light. Instead of having on-board processing, the Nreal Light glasses are tethered to either a high end recent Android phone or an Nreal compute pack.
Specifically, the company claims the glasses will work with any Android phone which uses the Snapdragon 855 processor. That should include the Samsung Galaxy S10, Google Pixel 4, OnePlus 7, Galaxy Note 10, and more.
Nreal opened preorders back in November for a “developer kit” which includes the glasses and compute pack, for around $1200. The glasses alone are expected to be priced around $500 to consumers.
Nreal planned to ship the Nreal Light in 2019, but delayed until spring 2020. However, production was recently halted due to the novel coronavirus.
Controller-Free Hand Tracking
Nreal’s controller-free hand tracking is powered by California-based optical hand tracking company Clay AIR, which is backed by Qualcomm.
Developers can use the SDK to display the user’s hands as full hands, separate fingers, a bounding box, cursor or customized skins. The SDK will report when the user makes gestures such as pinch, point, grab, swipe and zoom.
This should make for a significantly improved user experience over Nreal’s previous default input method: your smartphone used as a 3DoF rotational laser pointer (similar to an Oculus Go controller). You can still opt for 6DoF controllers via Finch if you want, for an extra $200.
Nreal is not the first AR headset to get hand tracking. It’s a feature present in both Microsoft’s HoloLens 2 and the Magic Leap One. But it is the first mobile headset to do so without a depth sensor, just like Oculus Quest was the first mobile VR headsetto do this. Real time hand tracking without hardware-level depth information is significantly more difficult to do, requiring state of the art machine learning algorithms.
AR like Nreal is trying to build is extremely hard to do well. Outside-the-home AR faces a larger set of problems to overcome as compared with inside-the-house VR. A range of lighting conditions and an ever-changing world present significant hurdles for both AR displays and tracking, and our last few demos of Nreal were not without issue.
The fact that this hand tracking is provided by a third party (Clay AIR) also raises the possibility that other manufacturers, including potential future Oculus Quest competitors, could integrate this same technology.
We’ve seen the interesting ways developers have been experimenting with controller-free hand tracking in the VR space on Quest. If Nreal is able to sort out the production problems and gets this hardware out to a wider audience, we’ll keep a close eye on what possibilities it opens for developers in AR.
