Earlier this year, Facebook showed off its research towards photorealistic avatars and high quality VR face tracking to drive them.
This week, Facebook showed off improved facial tracking for these avatars, more accurately capturing the user’s facial expression.
The researchers use two headsets to achieve this tracking quality. The regular headset features three cameras facing the user. The training headset has these three cameras and six extra cameras which gives a better view of the user’s facial expressions.
The Facebook researchers train a neural network withh the training headset such that the model can be used to attain high quality results with the regular headset. The extra cameras on the training headset provide ground truth. Facebook claims that with a trained model there is “almost no obvious quality drop” from 9 to 3 cameras.
Facebook isn’t the only VR company working on face tracking of course. At GDC in March, HTC announced a lip tracking addon for the Vive Pro Eye. However, HTC stated that it currently has no plans for a consumer version.
Still Years Away
When Facebook showed off codec avatars in March, the company was clear to control expectations, stating that the technology was still “years away” from reaching consumer headsets. There is no indication that this advancement will speed up this timeline. Last month, the company’s Oculus division launched two new VR headsets (Oculus Quest and Rift S) but neither headset has cameras pointing towards the user’s face.
One of Facebook’s primary reasons for acquiring Oculus in 2014 was a belief in the power of social VR. It seems likely that the company’s next generation of headsets will include the hardware for face tracking- even if the shippable software isn’t at this quality at launch.
