Softkinetic, founded in 2007, have been in the gesture recognition business for eight years and in that time they have established themselves as a leader in that field. The company develops solutions on three fronts from hardware, to middleware, to user experiences. If you haven’t heard of them before, don’t worry, they work more behind the scenes with their technology HTLib (full body tracking software) is available for developers for use with the Playstation Move camera as well as having developed the technology behind the first generation of Intel’s RealSense technology. But now the company has shifted its focus into the VR and AR realm and they have some big plans in the future, including providing what they believe will be the solution for positional tracking in mobile VR.
In an interview conducted last week at GDC, Eric Krzselo – Softkinetic’s Co-Founder, told me that he was “100 percent,” confident that the company would have a solution for positional tracking in mobile VR. Krzeslo further added that, “by the end of this year, we’ll have something that we’ll be able to present to John Carmack,” who had just reiterated earlier that day during his keynote he had yet to see any working inside-out (where the tracking is done from inside the HMD/Phone as opposed to with a sensor outside the device like with the DK2 or Kinect) positional tracking, but believed it was possible. Noting the 80-20 rule, Krzselo said the company had most of the work done but the last bit was going to be the hardest. While Krzselo recognizes they have some ways to go still, he is extremely confident they will get to “the right level of quality,” by the end of the year. He was also careful to note that sometimes “you never know when you do R&D,” but followed it up by reiterating his confidence, “we think we are going to get there.”
A lack of positional tracking – or the ability to look through a space, not just look around it, is one of the major problems that is preventing mobile VR from taking off like a rocket. For those who have tried the Oculus Rift DK2 (or the Vive, Crescent Bay, or Morpheus) and any form of mobile VR (Gear VR, Cardboard, etc.) the difference in experience is massive.
This is why figuring out a solution for mobile VR positional tracking is so important. There have been a number of solutions tried, many of them making use of what is called Simultaneous Localization and Mapping, or SLAM. This method uses technology borrowed from robotics to construct and update a map of an environment while simultaneously tracking where the user is within that map. One of the problems that many solutions have run into thus far is they produce an unacceptable amount of jitter – which you can see in this video from Univrses (a solution that uses what is called Monocular SLAM, using one lens).
Softkinetic’s solution, while the details are still internal, uses 3D sensors as opposed to a 2D camera, and according to Krzselo, uses them “in a different way than to use the traditional way people are using 2D sensors, other than SLAM.” This solution, says Krzselo, has eliminated the jitter problem. Using a 3D camera provides a set of challenges, especially when it comes to getting precise detail, but it is a problem that Softkinetic as been working on, especially through the 3D printer scanning that it has been doing with Intel’s RealSense system. Through that work, “[they] have discovered some interesting avenues to very, very accurately, in a very, very stable way get the position of the camera.”
The new prototype cameras that Softkinetic is working on (one of which is above) are vastly smaller than the previous editions, and come with an extremely large horizontal field of view, which is vital for things like hand tracking in VR, which Softkinetic is also looking to bring. According to Krzselo the company has internal prototypes with “over 200-degrees horizontal FOV.” To compare, the Microsoft Kinect 2 has a horizontal FOV of 58-degrees, and the Leap Motion (currently the leading hand tracking solution for VR) has a horizontal FOV of 150-degrees. Additionally, the new camera will use an incredibly small amount of power, “0 watts practically,” meaning that it won’t tax mobile batteries, leaving more juice for games and experiences. While these are far from the only specs that will need to be measured, it is clear that this upcoming camera will be a marked improvement over what is currently on the market.
Softkinetic is looking to make a strong push into the VR world this year, as Krzselo believes that it truly is the future, “virtual reality is incredibly exciting for the gaming community, as it provides a truly magical engagement with the world around us.” That initiative begins with a partnership with Razer’s OSVR program, which aims to make VR hardware and software Open Source in an effort to make it seemless to integrate various different peripherals into a game or system (think of it as the Android of VR). Developers can begin playing with the software integration using the company’s current generation DepthSense camera the Creative Senz3D (which Krzeslo notes is two years old) which can be integrated using the ReachVR toolkit available on www.reachvr.com. Additionally Softkinetic has adapted its Close Interation Library (CILib) for use with VR and AR environments, which allows for robust tracking solutions that go beyond typical skeletal tracking.
Hand and positional tracking are two of the biggest problems in VR because they are two of the things most needed to create a fully immersive experience. We saw a glimpse of the future with the Vive and the advanced lighthouse tracking system, but even that limits your tracking to inside a box. If Softkinetic can make true on its promise to deliver something that Carmack would be proud of, then VR is going to get a lot more expansive.
