Facebook says its Oculus Quest and Rift S headsets store room mapping data locally and do not send that data to their servers.
The explanation comes after attempts to clarify how the system functions for consumers and how it was developed and tested internally with employees. Facebook’s two key VR products of 2019 — Oculus Quest and Rift S — both rely on the camera-based Oculus Insight system to function.
According to Facebook, Insight is a “visual-inertial SLAM” system which uses “image data from cameras in the headset to generate a 3D map of the world pinpointing landmarks” combined with inertial measurement units in both controllers and the headset. This is done by the Oculus Quest’s Snapdragon processor, or by your computer on Rift S. The camera imagery is not stored, but a 3D cloud of points identifying the placement of unique static features of the environment is saved locally on your device for any room where you have a Guardian boundary set up. This allows the system to recognize what room you’re in and load the Guardian boundary you previously created. Facebook tells us that this data doesn’t leave the local device.
We first reported in October last year that a white light on top of the Oculus Quest is “hard-wired to the sensor power rail.” That means whenever the camera sensors are supplied with power the light should turn on. In late August this year, Facebook explained in a fair amount of detail what happens when that light turns on. In particular, they explained how measurements from the cameras fuse in real-time with inertial measurement units in both the controllers and headset.
“This information is only stored locally,” a Facebook representative wrote in an email. “Not sent back to us.”
Quest owners could test some of this themselves by using the headset with the Wi-Fi connection turned off — either on the headset itself or at your local networking box. You won’t be able to download games or play online but the system will still let you move around in VR just fine.
Here’s Facebook’s video explaining the Oculus Insight system:
Facebook employs the terminology “3D map of your environment” to describe this collection of measurements “which helps locate your headset and controllers in a known space so Quest/Rift S can work and keep you safe.”
“We don’t collect and store images or 3D maps of your environment on our servers today — images are not stored anywhere, and 3D maps are stored locally on the headset [for Quest] and on your local PC, where you have access to delete it [for Rift S],” a Facebook representative wrote in an email. “That said, we’ll notify consumers if this information is required for VR experiences we provide on Quest/Rift S in the future.”
“The only information we keep on our servers consists of performance metrics that don’t contain any recognizable detail about your environment,” an email states. “These metrics help us improve Oculus Insight.”
Facebook’s Employee Testing
An alarming post to Reddit sparked our most recent series of questions to Facebook.
The now-deleted post was allegedly from someone who worked at Facebook and said they debugged the Insight Tracking system during its employee-based internal testing phase. The post claimed that during the employee-driven process of testing the Guardian passthrough system there was a way to submit bug reports. These reports allowed workers back at Facebook headquarters to see screenshots, some of which included views into the bedrooms of the company’s executives. A Facebook representative wrote in an email that the “report is not a true representation of our internal testing practices.”
“For our internal hardware testing, employees volunteer to take home and test products and agree to share their experiences to help improve the product for consumers,” the representative wrote. “Employees are given the option to include a screenshot when they report bugs (to assist with debugging) and they can choose if they’d like to submit it for review.”
The representative added that employees were “made aware/understand that if they submit a bug in that passthrough mode, that an image of their surroundings will be included with the report. As we noted previously, this is for internal testing purposes only.”
If you try to take a screenshot at home with a consumer version of the Quest while in Guardian view — like when setting up a new play space — the cameras will be shown as black in the resulting screenshot.
So to reiterate, according to Facebook, there was a bug reporting system used only with employees which came with disclosures to let workers know that when they submit a report from the headset while in view of the surrounding environment, like during Guardian setup, the view of their physical rooms may be sent to Facebook’s servers.
“If an employee is in passthrough mode – i.e. Guardian Setup – for testing purposes, then yes, the visuals of what might be in view of the headsets (their desk, etc.) would be included in the screenshot that they can choose to include if they were to submit a bug report,” the representative wrote.
Critical, then, to Facebook’s position regarding the implications of that original post:
“People are made aware/understand that if they submit a bug in that passthrough mode, that an image of their surroundings will be included with the report,” Facebook explained. “As we noted previously, this is for internal testing purposes only. We strive to provide the same transparency to our employees that we provide to our consumers who use our headsets.”
I asked Facebook to send me a screenshot of the disclosure employees saw as I think there’s public interest in seeing it. Facebook said they can’t share that externally.
