At Day 2 of F8 2019 today Facebook showed off a demo of body tracking with no markers or worn trackers. The tracking is able to track down to the muscle activation level, not just skeletal, giving a more detailed output:
This is just a research project, not a product, but the results look impressive. It’s described as being done with a single external sensor, not an array.
Body tracking is currently available for the HTC Vive by using Vive Trackers, but as each tracker is $99 and you need many for a high quality result, this isn’t affordable. Additionally, each tracker has to be kept charged and then worn whenever users want to use body tracking in VR.
A markerless approach from an affordable sensor could bring body tracking to mainstream VR one day. This would enable entirely new types of gameplay and greatly enhance social VR.
Facebook intends for this technology to be used alongside the photorealistic avatars it showed off in March. An avatar that looks like you and moves like you would feel, to a friend in VR with you, like you.
The company also showed off a larger scale body tracking demo where two players interacted with a virtual ball on a virtual pitch. This demo was key because it showed the latency was low enough for this kind of sport. They didn’t however say what amount of hardware was being used to achieve this. OptiTrack markers can be seen attached to the headset, so this was likely a multi-camera setup:
Facebook is the company behind the Oculus brand of VR headsets, so these technologies will likely eventually end up in Oculus products. The company did stress however that a consumer launch of this technology was “years away”.
You can watch the full presentation here:
