Facebook is in the final stages of acquiring a New York based startup called CTRL Labs which was developing an wristband which tracks the user’s fingers by reading electrical signals inside their arm.
It works by detecting electrical signals passing through the user’s wrist to the fingers. Based on how the signal changes passing through the tendons and muscles of the arm their position can be determined. Machine learning is used to convert these position changes into finger poses.
https://www.youtube.com/watch?v=4NMhx5WlpEs
The technology is very similar to what’s described in a patent application filed by Facebook back in February. It’s possible that the CTRL Labs team were able to solve problems that Facebook’s team wasn’t, or that Facebook wanted to combine their efforts. It’s also possible that the startup held intellectual property that Facebook would need to commercialize this technology.
The ability to directly use each of your fingers adds an entirely new level of interactivity to VR. However, that ability is absent from almost all consumer VR headsets today. Leap Motion shipped a finger tracking kit for the Oculus DK2 all the way back in 2014, but the tracking quality left a lot to be desired.
Facebook tells us the deal hasn’t yet closed but that they are working with regulators for approval and hope the deal can close quickly. Upon that happening, CTRL Labs team would be integrated into Facebook Reality Labs, the largest known VR research division in the world.
“They will be joining our Facebook Reality Labs team where we hope to build this kind of technology, at scale, and get it into consumer products faster,” wrote Andrew Bosworth, Facebook’s VP of AR/VR in a post to Facebook.
If this technology truly works, it could bring finger tracking to VR and/or AR without having to do power hungry processing on multiple cameras pointed at your fingers. Furthermore, since it doesn’t rely on cameras the tracking would work at all angles regardless of the headset’s orientation. We’re excited to see what finger tracking solutions Facebook delivers in the coming years.
