GDC 2017: OTOY Is Bringing Mixed Reality Light Field Streaming To Unity

by Jamie Feltham • March 1st, 2017

When we last heard from OTOY, the company revealed it was bringing light field baking and more to its Unity integration. At GDC today, OTOY is announcing a heck of a lot more.

Company CEO Jules Urbach is here to run through a list of updates in the video below but you best prepare yourself: there’s a lot. On display at Unity’s booth is the first light field video created in OTOY’s Octane renderer using the engine. According to the company this is both a “video light field movie” and also a “huge game level size navigable volume”. That essentially means you can move through the photorealistic space.

The piece uses the stereo cube map render shown back at the most recent Unite Conference. It apparently streams without the need for buffering.

Perhaps even more exciting is a demonstration of the company’s ORBX Media Player (OMP) running on Google’s Tango 3D capture system. This brings the company’s light field streaming into mixed reality, allowing you to view realistic-looking objects in the real world through a phone’s screen.

OTOY is also working with ODG to incorporate the tech into its AR smartglasses once they’re capable of 6 degrees of freedom (6DOF) movement. The stereo polarized cameras being built into the company’s glasses are also going to support full light stage capture with OTOY’s help, using OMP to process frames. Items captured could be turned into a Unity asset to be brought into VR or MR experiences.

Finally, moving back to VR, Unity and OTOY are showcasing a preview of Samsung’s VR Internet Browser app for Gear VR with native ORBX file and URL playback support. This means you’ll be able to immerse yourself in realistic environments as you browse the web. New samples of this content have been posted to the web too, showcasing “experimental support for exporting and embedding interactive Unity apps inside of an ORBX media file” bringing navigable UX, screens and dynamic objects into them. The company says this is currently working on mobile.

What's your reaction?
  • Downgrade

    Anyone knows if the light field stream they’re talking about is narrow enough to fit internet bandwidth? Or it’s been streamed to that smartphone via local high bandwidth network?

    • Richard Servello

      It is extremely light. There are plenty of articles and videos on the technology.

  • Jules Urbach

    We’ve set up live demos at GDC this week showing it running on various devices. The mobile stream is 1-5 Mbps, including the AOV layers.

    In giving a talk on how to build this content tomorrow at the Unity booth at 2 PM.

    • FreDre

      I remember that AMD Cinema 2.0 demo almost 10 years ago.
      What happened with that?
      I also don’t understand what are you trying to accomplish with light-field technology
      You want to do 360º stereoscopic VR videos with head-tracking? Or is it only to do AR?

    • John Locke

      Hi Jules, was this presentation recorded/online?

    • hellol11

      “The mobile stream is 1-5 Mbps” talk about accessible, it takes a shit internet connection to make that download in over a second.

  • unreal_ed

    Maybe I’m just tired right now, but am I the only one not understanding what the announcements are? I’d love to hear it again in plain(er) English