Facebook shared some details about its experimental Passthrough API to enable new kinds of mixed reality apps for Oculus Quest 2.
The feature may also serve as the foundation for the company’s long-term efforts in augmented reality, effectively turning Quest 2 into a $299 AR developer kit. When asked if the feature is coming to the original Oculus Quest, a Facebook representative replied “today, this is only available for Quest 2.”
The new feature will be available to Unity developers in an upcoming software development kit release “with support for other development platforms coming in the future.”
Facebook says apps using the API “cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors” and raw images from the four on-board cameras “are processed on-device.”
The following capabilities will be available with the passthrough API, according to Facebook:
Composition: You can composite Passthrough layers with other VR layers via existing blending techniques like hole punching and alpha blending.
Styling: You’ll be able to apply styles and tint to layers from a predefined list, including applying a color overlay to the feed, rendering edges, customizing opacity, and posterizing.
Custom Geometry: You can render Passthrough images to a custom mesh instead of relying on the default style mesh—for example, to project Passthrough on a planar surface.
Facebook sees the Passthrough API as enabling more robust examples of productivity apps — remote work solution Spatial was one of the first apps to access the API — alongside the ability to interact with people or pets in your physical room at the same time as VR content, and it could open up new kinds of games that could incorporate more of your physical environment.
