The controller tracking of the Oculus Quest and Rift S needed to be patched to work properly near Christmas trees and other holiday lights.
Oculus Touch controllers are built with a constellation of infrared LEDs under the plastic of the tracking ring. These lights are tracked by the cameras on the headset in order to determine the position of the controller.
Holiday lights like those on Christmas trees can look a lot like these LEDs to the cameras. This means the algorithm has more sources of light in each frame to analyze, and sometimes it can’t tell the difference between the controller LEDs and the irrelevant LEDs at all. This could make the controller tracking work poorly, showing the wrong position for the controller.
The solution works because the headset tracking algorithm already remembers static landmarks seen by the cameras in the room — that’s how it works without external sensors. By keeping a track of these landmarks the system can reject blobs of light which stay in the same position and don’t move.
This process on its own, however, is not enough to eliminate all the issues. So Facebook also trained a neural network to detect and filter out blobs of lights that are too small or too large to be a controller LED given its last position.
You can read a full technical explanation of the solution in Facebook’s blog post.
This isn’t the first time Facebook improved the controller tracking on Quest and Rift S. Both headsets use the Oculus Insight tracking system and launched on the same day. At launch, the controllers wouldn’t track when brought too close to the headset and tracking could break when one was placed in front of the other. This made games like shooters difficult to play until a patch was released one month after launch which fixed these issues.
