A US patent granted to Sony reveals research on VR face tracking, a vital technology for the future of multiplayer and social VR.
Titled Integration of tracked facial features for VR users in virtual reality environments, the patent describes the use of one or more cameras on the underside or nosegap of the headset to capture the user’s mouth and facial expression in real time.
This tracked data would be represented on a virtual avatar, shown to other users online in multiplayer games. Imagine seeing your friend gasp, smile, or laugh inside a shared virtual world- that future is on the horizon. As with previous Sony patents, the headset also contains eye tracking so that your gaze is represented.
The patent describes a few interesting techniques to deal with the occlusion issues- the cameras not being able to see the whole face. A flex sensor pressing against the user’s face is proposed, as well as a way to sense the exact position of the nose flap so that it isn’t detected as part of the user’s face.
Facebook, the company behind the Oculus VR brand, has shown off similar technology over the past few years. Videos released by Facebook show just how much detail can be captured with just cameras on a headset:
There’s no indication that this technology will ship in PlayStation VR 2, Sony’s highly anticipated upcoming headset for PlayStation 5. It could be research for headsets even further in the future. Oftentimes patented ideas by major companies never come to fruition at all. If the headset does contain these features though, it will enable developers to take multiplayer VR to the next level.
