A friend of mine always used to say ‘if your keyboard only worked 60% of the time, you would throw it against the wall.’ We expect our technology to work every time, and when it fails we get infinitely frustrated. That is where Leap Motion has been with VR for the last couple years, a mess of tracking errors and Hanson hands that frustrated both users and developers with a tantalizing promise not quite realized.
But those frustrations haven’t stopped the team at Leap Motion from relentlessly pursuing their goal of bringing hands into VR and today their latest breakthrough is being released into the wild, and boy is it a massive improvement.
https://www.youtube.com/watch?v=7HnfG0a6Gfg
Dubbed Orion, and available today, the third version of Leap Motion’s software stack comes with a number of major improvements to the tracking from lower latency, to faster more reliable tracking, to a lack of Hanson hands. The improvements stem from a goal set in late 2014 when Leap Motion decided to pivot their development from interaction with flat screens to interaction with VR.
“Orion software represents a paradigm shift in how we do hand tracking,” says David Holtz, Leap Motion’s co-founder and CTO. “It’s built from the ground up specifically for VR. It’s radically smoother, faster, more reliable, and far more capable than even the best of what’s existed before.”
In my hands on with the new software I found that many of the situations where the previous Leap Motion software failed, Orion succeeded. For example, even in situations where I bent my fingers out of sight of the camera the system was usually able to compensate and predict where my fingers were based on the orientation of the rest of my hand. It wasn’t always perfect, but I no longer wanted to toss the device against the wall.
Rather, I found myself entranced by the “Blocks” demo. Simple yet effective, the demo has you create and play with a set of blocks demonstrating both natural interactivity and gesture based interaction, as well as a hand tracked user interface. During the limited time I had with it I found myself lost in the simple pleasure of picking up, knocking down and creating objects in virtual space, while basking in the glory of having real, working hands in VR.
But my play wasn’t without purpose. I was able to test things like complex and quick gestures like using my thumb and index finger to quickly flick a box off the top of a stack, or smack it from hand to hand. The latter was slightly more difficult because it highlighted the lack of haptics, something that is a problem for this type of interaction.
There is an inherent disconnect when you try to pick something up naturally and you don’t feel it there that is very similar to the Uncanny Valley in robotics. You know you are getting there but you are just far enough away from it that you notice. Audio helps with this however as Leap Motion attempted to incorporate what they are calling “audio haptics,” a synesthesia effect that comes with incorporating a visceral and well timed sound effect with the visuals on screen – for example, a satisfying “thud” when the cube makes contact with your hand.
These effects helped add to the immersion, but the real star was the fact that my hands no longer went all cattywampus when I made a fast or difficult gesture. Even when Leap fails (which it still will if you try) it doesn’t fail in the same catastrophic way as it used to, they have smoothed out the edge cases. For example, overlapping your hands (something the software still isn’t able to resolve) leads to the bottom hand being hidden, rather than going crazy. This actually helps increase the level of immersion because it is far less noticeable than a hand dancing off into the corner. The system also handles other difficult situations far better, such as tracking a hand against another surface, like a pair of pants or a desk something that the previous edition of the stack struggled with.
Finger Tracking is coming as an embedded feature in some headsets this year
But Orion isn’t completely about the software, just about 99%. Leap Motion has new hardware coming down the pipeline alongside Orion but it won’t be shipping as a standalone dev kit. Instead Leap Motion is currently working with multiple OEM manufacturers to directly integrate their hardware and software into future headsets. According to Michael Buckwald, Leap’s co-founder and CEO, “99% of the changes [with Orion] are in the software stack” and there will only be minor differences (such as an increased IPD) with the new hardware.
With whom Orion will be integrated with still remains the million dollar question however, as the ‘major’ headset designs have been all but finalized heading into the consumer release. That said, Buckwald seemed to hint that what we have seen may only be the tip of the iceberg.
“I do think that VR is like an iceberg in that there are three companies that people tend to know about that are building headsets… but beneath the surface almost every major technology company has a VR program and many of them are focusing on mobile, self-contained headsets because they realize that to get to a level of ‘mainstreamness’ that is suitable for them it is hard to get there with setups that require gaming computers and trackers and such.”
Buckwald says that these types of headsets are one of the main areas they are going to target because they need an input that doesn’t require the user to carry an extra peripheral around with them. He did mention that “there are tethered devices we are working with, in some cases first generation in others second” so the mobile space isn’t the only one where we will be seeing official Leap Motion integration in the near future.
Hand and finger based interaction is going to be a crucial piece of the puzzle going forward for VR and AR as there is not more natural input than the two hands you use on an everyday basis. In order to achieve the maximum amount of presence in VR possible, it is hugely important to bring your hands into the experience. It is something that Leap Motion recognized over a year and a half ago, and today they have brought us multiple steps closer to making that dream a full on reality.
So if you have a Leap Motion at home collecting dust, bust it out again and give it an update – you may be pleasantly surprised.
