Oliver Kreylos is a researcher at UC Davis focusing primarily on virtual and augmented realities. He is known as something of a sleuth in the industry and was one of the first people to determine the actual field of view of the HoloLens before Microsoft released the official specs. More recently, Kreylos has been digging into the Oculus Rift Sensor and its ability to function as a basic webcam.
The right side of the above image was pulled by Kreylos from one of his Oculus Sensors. These “Sensors” are essentially cameras used to track the position of the Rift and Touch controllers in three dimensional space by picking up the imperceptible glow of infrared lights all over the headset and controllers. What Kreylos demonstrated, however, is that with a bit of informed tinkering, one can actually pull a recognizable image from the Oculus Sensor as if it was a low resolution black and white camera.
How is This Possible?
According to Kreylos, the combination of some basic Linux know-how and a general understanding of the way a computer works is all it takes to pull an image from his Oculus Sensor.
In an email written to UploadVR, he explains::
Oculus decided to hide the fact that the Rift ‘sensor’ is a bog-standard webcam. Normally, when you plug a webcam into a USB port, the camera announces itself as a device in the USB video class (uvc), and the host PC then loads the standard uvc camera driver, and the camera works plug&play.
The Rift camera is still a uvc camera, but in the USB descriptor that’s sent to the host when it’s plugged in, the byte that should say ‘video class’ says ‘vendor-specific class.’ That’s why it doesn’t show up as a camera when you plug it into a PC without Oculus’ driver software installed.
To make it work, I had to patch the Linux kernel’s uvc camera driver. When the driver sees the USB vendor and product ID that match the Rift camera, it ignores the class type that’s in the USB descriptor, and starts treating it as a uvc camera. And because it actually is one, from that point on it works.”
Essentially what this means is that Oculus told the Sensor not to announce itself to the Windows operating system as a camera. If it did, Windows might automatically pull up its standard webcam drivers, which is not what Oculus wants. Instead, there’s a custom driver they want Windows to use to interface with the Sensor so that it functions only in tandem with the Rift. Kreylos describes this as a layer of “obfuscation” and says it is one of a few that Oculus employs to keep what is, in essence, a camera from behaving like one.
According to Kreylos:
“I then noticed that Oculus added a second layer of obfuscation. The camera’s real video image format is greyscale, with 8 bits per pixel. In uvc, image formats are selected by four-character so-called fourcc values. The fourcc for greyscale is ‘Y8’ or ‘GREY’, but the Rift camera’s firmware lies and sends ‘YUYV’, which is an interleaved 16-bit color format that’s used by most webcams. The firmware makes up for the fact that the pixels are twice as big by lying about the camera resolution. For example, instead of advertising 1280×960 Y8, it advertises 640×960 YUYV. That way, the actual raw image data has the same overall layout (1280 bytes per image row).”
This is a detailed way of saying that even if you can pull a still from your Oculus camera, there are still adjustments required in order to deliver a coherent picture. The Rift Sensor is only meant to capture one thing: the infrared trackers. In fact, the Oculus Sensors include a physical filter to remove visible light since that is “just noise, making it harder for it to operate,” as iFixit wrote in its teardown of the hardware.
However, just because this is what the Sensors were designed to do does not mean that an enterprising mind cannot use the hardware to produce something recognizable. It may not even be that difficult. According to Kreylos, “To get the real picture, I had to nothing but ignore the advertised fake video format, and treat the incoming raw data as the actual format.”
So…is Facebook Spying on Me?
The Oculus Sensor is designed to capture a specially formatted set of coordinates and immediately discard any visual data, with Facebook telling us “we do not store any frames captured by the sensor, so there is no way for someone to access this information from our server.” While it was designed with a very specific usage in mind — to track the headset and controllers — it is nonetheless technically possible for a hacker who has gained access to your PC to use the hardware to spy just like any other webcam.
Kreylos:
“My point is that Oculus’ driver doesn’t retain or store camera images. They get consumed and then destroyed immediately after arriving over USB, and only the extracted (x, y) LED positions survive and get fed into the pose estimation algorithm.
Now, it is conceivable that the driver could retain images anyway, and even send them up to Facebook headquarters for analysis. But that’s tinfoil hat territory. The potential payoff would be minimal (better ad targeting?), but the potential risk — if this were to come out, and it would sooner or later — is enormous. Not just from a PR disaster perspective, but from the point of severe legal repercussions. So no, they’re not doing that”.
Oculus is adamant on this point as well:
“The Rift Sensor doesn’t operate like a typical camera. We’ve specifically designed this sensor to detect infrared signals on the Rift and Touch controllers. This is how we make sure the experience in VR mirrors how a person is moving in real life. Frames captured by the sensor are processed to reduce things in the background so our infrared signals are clearly highlighted. Then, we immediately discard the frames. The sensor isn’t connected directly to the internet and we do not store any frames captured by the sensor, so there is no way for someone to access this information from our servers.”
What Should I Do?
Despite its intended purpose, Kreylos shows the Rift sensor can capture a recognizable image of anything or anyone in front of it. This does mean that the device is vulnerable to hackers and malware. As Kreylos puts it, “What might be an actual potential worry is other, non-Facebook actors, turning the cameras into spy cameras via malware. I could actually see that happening at some point.”
Facebook agrees on that point, stating, “Like other Windows or USB peripherals, the Sensor could be accessed if a person’s PC is compromised.”
The message here is that your Oculus Rift Sensor is perfectly capable of transmitting images of you and your home to any hacker or agency skilled enough to overcome Windows security or any extra security you may have installed on your PC. Until we hear otherwise, we believe the Oculus Sensor to be no more or less vulnerable than your PC as a whole.
It is important, therefore, for any user of an Oculus Rift to be aware that a potential for invasion does exist with their Sensors, and to take whatever steps he or she deems appropriate for protection.
To put this all very simply, if you are the type of person that feels a Post-It Note or tape over the webcam on your laptop is an important security precaution, you should probably cover the Oculus Sensor as well.
Clarification: A section above discussing Facebook’s intended use of the sensor was updated with a clearer description.
