Apple is working on headset versions of its core apps as well as the ability to view a Mac’s display, Bloomberg’s Mark Gurman reports.
Last year Gurman, The Information, and supply chain analyst Ming-Chi Kuo released reports claiming Apple is working on a premium headset for VR and AR with high resolution color passthrough. Kuo claimed this headset will weigh less than Meta’s Quest 2, feature dual 4K OLED microdisplays, and use a new chip with “similar computing power as the M1 for Mac”.
In May Gurman reported Apple recently ramped up development of realityOS (rOS), the operating system the headset will run, and previewed it to the board of directors. References to realityOS were found in App Store upload logs and Apple code earlier this year, and an Apple linked shell company trademarked RealityOS last week. At Apple’s WWDC conference today a number of features tangentially related to AR were announced, but the company may be waiting for the headset reveal to show its full AR/VR strategy.
In a newsletter published over the weekend Gurman wrote:
Apple’s headset initiative isn’t simply the device and its operating system. It’s an entire set of new VR- and AR-powered Apple apps and experiences, a slew of input paradigms never seen before on the company’s products and a completely new platform for third-party developers.
Apple is reportedly working on a VR client for FaceTime with face tracking driving Memoji avatars, as well as a VR version of Maps (similar to Google Earth VR?) and spatial versions of Notes and Calendar. You’ll also apparently be able to extend a Mac’s display into the headset. Leveraging its existing suite of first party cross-platform applications, and making them work seamlessly across devices, could be one of Apple’s key advantages competing with Meta’s Project Cambria.
Gurman also says Apple is “leveraging its entertainment arm” and acquisition of live events livestreaming startup NextVR to produce content for the headset. Over the weekend The New York Times also reported Apple is enlisting Hollywood directors like Jon Favreau to create immersive VR content, such as a version of Prehistoric Planet.
For developers, Apple will reportedly ship a spatial version of Apple’s SwiftUI user interface framework, which should make it much easier to build AR and VR applications compared to using a game engine like Unity or Unreal. This UI framework would integrate into RealityKit, Apple’s existing high level AR framework that handles physics, spatial audio, all aspects of rendering including materials, shadows, reflections, and even camera motion blur. RealityKit also handles networking for multiplayer AR apps, meaning developers won’t need to be network engineers to develop shared AR experiences.
Developers will also apparently get access to a Mac based headset simulator so they can develop before having the device in-hand.
Earlier this year Gurman reported Apple’s headset was delayed to 2023 due to “challenges related to overheating, cameras and software”. The New York Times also now reports the headset is delayed to 2023, with its sources saying this is due to “continuing challenges with its battery power”. Both Gurman and The Information report the product is set to be priced north of $2000.
