Max Planck, from Oculus, Shares Details About Internal Prototype and Oculus Story Studio, Hints at Consumer Release Date [UPDATED: Palmer's Comments]

In Oculus’ Story Studio’s launch party was one of the highlights of my time at Sundance, which was filled with highlights. Tucked up on the mountain, a gondola ride away from the resort, the night was filled with fun and demoing. I managed to find some time during all the craziness to sit down with Max Planck, supervision technical director at Oculus Story Studio, who worked at Pixar for 10 years previously.

During our conversation, Planck told me the story of how Story Studio came to be, and what his plans are for it in the coming months, what his top five challenges they have overcome through the process, and what he sees as the future of VR cinema in the next five years. Beyond the new Story Studio, Planck also revealed some details about the internal prototype, as well as hinting that “May is getting close,” in relation to preparing a consumer model.

Full Transcript:

——

WILL: All right, Max, so I just saw Lost. Total Iron Giant fan boy moment. Tell me, how did this whole thing get started?

MAX: It’s felt like a long journey, but it’s been more in the last six months, I—let me back up a little bit. So in 2004 I graduated from MIT and went to Pixar. I worked on Cars, I worked on Wall-E, Up, then on Brave, Monsters University, and finally The Good Dinosaur. And I loved it, it was awesome. But at 10 years at Pixar, I was ready for something new. I wanted to find what our generation’s interesting new creative problems were. I’m a futurist, I’m an inventor, I’m an engineer, and I left Pixar to find that thing. I actually didn’t know it was virtual reality at the time.

I was going more towards games, because I think games are great. I’m a gamer myself and I really wanted to explore storytelling and interactive. And then I saw the Oculus internal prototype, and I told myself, this is it. This is what I was looking for. This is the type of problem that’s going to have so many interesting things to solve for so many years to come that I’m like, this could be the rest of my career. This is it. This is what I want to do.

“Great art is made because you’re inspired by the people around you.”

And so I started talking with Saschka Unseld and Edward Saatchi, and we wanted to start a real time computer animated studio. And at the time, we were talking to Oculus, and Oculus said, you know what, we have this idea. How about you come to Oculus and do this. And this was in July of 2014. And we said, you know what, we want to be in a space where we can experiment, where we can build a community. Because I want peers. Great art is made because you’re inspired by the people around you.

It’s a little hard to be a creator in virtual reality right now because there is no consumer model. When a consumer model comes out, there may be a business model there. But to have the resources to support a career to play in the space that aren’t that many options out there. And Oculus was this pretty opportunity for us. And so we said, yes, let’s start story studio together. We will experiment, create shorts, show that real time immersive cinema is an interesting use of VR. I’m a total believer in gaming and VR, but I also think VR can be more than that, it can be this new medium.

And so it was a great opportunity and we starting working on our first experience, which was inspired very much by The Iron Giant. I think I’ve heard this from many people who get into VR making. We love scale. I remember seeing Titans of Space on the DK1, and it’s a story. It’s a very, very simple story, but it has this idea of scale progression. Let’s introduce you to the scale of the universe, and then it sort of blows your mind as it keeps on adding scale on top it.

“VR can be more than that, it can be this new medium.”

So we said it would be cool to create an animated narrative with scale in mind, so we went with what it would be like to have you in a forest, and we introduced elements that were small at first and then bigger, and finally this awe inspiring bigness that really takes advantage of virtual reality. And that’s the journey, and that’s where we are now. And our mission is, all right, we wanted, we’ve been operating in stealth mode. We want to create our first experience and say we really believe in this. This is our thoughts, this is our first attempt, our first step. And now that we’re out there, we want to share as much as possible.

“I want to share this experience, I want to share the source code of this experience. I want to share our assets, I want to share what we didn’t do. The failures, we said, hey, we tried cuts or teleports, and this is why we didn’t’ use them because they failed in this way.”

So I want to share this experience, I want to share the source code of this experience. I want to share our assets, I want to share what we didn’t do. The failures, we said, hey, we tried cuts or teleports, and this is why we didn’t’ use them because they failed in this way. And as we’re developing, we now have three other productions in the pipe. These are all short experiences. We don’t want to go for a long form experience, until we feel like we have more experience. We still feel like—I only have 6 months experience in this, but it’s surprising to see [what] my experience in computer animation has brought over, and that’s what we want to do. We want to tell computer animation artists and game artists, there’s this new medium you can be a part of, and the tools are out there. We’re using Unreal, we’re using Maya, we’re using Photoshop, we’re using Substance Designer. Those are the kind of tools and they’re out there.

“The talent is out there, the tools are out there, and we’re trying to de-mystify, hey, this is easy to do, and it’s really powerful, and it’s a new experience beyond cinema.”

When Pixar was first starting, they had to write all that stuff. They had to make the hardware to enter this stuff. But the talent is out there, the tools are out there, and we’re trying to de-mystify, hey, this is easy to do, and it’s really powerful, and it’s a new experience beyond cinema.

WILL: So what was the moment that clicked for you with virtual reality? When were you like, hey, I’ve really got to get into this?

MAX: I was actually a skeptic. I had a DK1, and I was playing with it, and I was like this is cool, this is neat. It was a hobby. And then I got the DK2, and that—the leap from the DK1 to the DK2 was enough to convince me that if the leaps continue like that—I actually still think the DK2 is pretty bad. It has a lot of flaws to it. Even working with the Crescent Bay unit, it’s yet another leap beyond the DK2, but it is also flawed. I don’t like that I have a wire tethered to the computer. The resolution needs to get better. There is still distortion and enigmatic issues happening with the lens. That all needs to get better. The machines need to get better to deal with anti-aliasing.

I was like, wow, DK2, this is cool! Maybe this is what I was looking for. And then I went to Oculus, and saw their internal prototype, the thing that is the target that they’re shooting for. So they had a prototype that blew everything out of the water, and Brendon has talked about this. They’re hoping to create a 300, 400 dollar version that delivers that experience. Plus, May is getting close.

I still think it has a little bit of room to go.

“They’re hoping to create a 300, 400 dollar version that delivers that experience. Plus, May is getting close.”

That day when I saw that and it was a very simple demo. This was done by a programmer, programmer art in this experience, but that was enough, right? Floating cubes in front of my face and a manger sponge in space, and I’m like, oh wow, I can do a lot with this.

[EDITORS NOTE: See Update]

WILL: Obviously you can’t describe too much detail, but what were some of the big differences between the internal prototype and the Crescent Bay and DK2 and stuff that you’ve seen before? 

MAX: There were a few things that I saw that needed to happen. Low latency is very important in terms of display. The two millisecond flash is important. You could argue that it needs to be done in one millisecond, but then it would look too transparent. You would lose the solidity. Positional tracking is really important. I love the stuff that’s happening in live action with 360 video, but I’m a real believer that if you’re going to be present in a space, you need to feel parallax. We got all of our depth cues more from parallax than we do from stereoscopic vision, and so even the microscopic view is important. The stuff that we love is stuff that’s close.

“I’m a real believer that if you’re going to be present in a space, you need to feel parallax.”

There’s this level of proximity enjoyment, is what we call it. It’s as big as your hand radius. The stuff that you can touch within your bubble. We love that—and that stuff is up close, you want to look around. You want to be able to see it from as many angles as you can where you’re standing. And that was the big leap. I think the DK1, DK2, that’s when I started to think, I think this could be my career. And then right now, resolution. Resolution is the next thing that needs to get much better.

[EDITORS NOTE: See Update]

WILL: What was the big difference between the Crescent Bay and what the internal model that you’ve seen?

MAX: The resolution. The resolution is better. The internal model is a great target. I think they’re trying to hit it but they also need to make a consumer model price, so there may be things that they fall short on, but resolution was key. Also, the tracking used a different technology. I don’t think I can go into detail about that. But the tracking was spot on, while the camera they’re using is still good. But I noticed there is a little bit of jitter, and Brandon knows this too. If you get close up to objects, there is a bit of wobbling in the experience and that needs to get better.

“The internal model is a great target. I think they’re trying to hit it but they also need to make a consumer model price, so there may be things that they fall short on, but resolution was key.”

And then, yeah, the internal prototype is still tethered. The internal prototype also has a massive machine attached to it, so that’s not practical. I think the internal prototype is a promise, and we might not get there with the first consume model, because it’ll mean you have to buy a 10,000 dollar PC. That’s just not practical—that’s fine. I’m okay with that kind of creative restraint. I will work with it. I will create very stylized character stuff that is polygonal, and maybe embraces tessellation. You see that in games that are sort of going for this 8 bit retro style, that it’s awesome. That can be a great experience. So you don’t need to go photo real for a cool experience.

[Editor’s Note: See Update]

WILL: Talking beyond technical aspects, what is the new narrative structure that is going to take place in this new medium? Because obviously, you can’t tell a story quite the same way as you do in flat cinema.

MAX: The biggest for me is we say we’re storytellers. We often say that we’re making immersive cinema or these are VR films. It’s okay we’re saying that. We’re saying we’re using film as a jumping off point, but this is more than that. This is a mix of film, a mix of game, a mix of immersive theater, and I actually think this is a very fun version of storytelling. And storytelling—when an orator tells you a story, he or she adjusts to the audience. They read the body language. They say, all right, when I usually tell this story, I begin a certain way. Certain events happen in a certain way, and I end a certain way. But depending on who I’m talking to, I might speed up a section, I might change the order of events.

“This is a mix of film, a mix of game, a mix of immersive theater, and I actually think this is a very fun version of storytelling.”

And I think because in the VR, it’s a very intimate experience, I know who you are as a viewer because I know where you’re looking. There’s such accuracy in the headset that I can tell when you’re kinda slumping or getting bored, and I can use that to change pacing. So I think that VR is a way to tailor storytelling to the individual, and I think that’s what makes it cool.

[EDITORS NOTE: See Update]

WILL: So, when you’re telling a narrative in VR, obviously there’s going to be different structural techniques, some different things you have to do. What are some of the top 5 challenges that you guys have managed to accomplish to succeed in VR?

MAX: Top 5. I’ll just start talking. Because we’re doing real time immersive cinema, we’re doing it in a game engine. So it means that if we’re trying to hit 90 frames per second, we have 11 milliseconds to throw something interesting up that is very creatively constraining. And so we made decisions on Lost to have a night time scene that allows us to hide the fact that we might not be able to fit as much detail as we want. And so that was a cool, creative constraint.

“If we’re trying to hit 90 frames per second, we have 11 milliseconds to throw something interesting up that is very creatively constraining.”

Two, we discovered that people who are new to VR are often worried that there’s something happening behind them. I think a lot of people do have stuff happen because it’s a fun novelty. We decided not to. We said, we’re going to—we want you to be comfortable in this space. Clearly this is a virtual stage. Clearly we’ve decided so that the most interesting stuff is within maybe 150 degrees around you. And so we made the action happen so that you’re looking right, left, but you’re not turning around. We don’t want to violate this trust of—you put this headset on, I don’t want to come up behind you and scare you. And I think that was a good approach. At the same time, I think we have action that’s too far away from you. We kind of set you off and you’re watching a stage or a diorama when it would be a lot of fun to take advantage of that proximity bubble I was talking about.

“We’ve decided so that the most interesting stuff is within maybe 150 degrees around you.”

We made some decisions to not put as much interactivity in. and so the experience does flex a little bit based on where you’re looking, and we call this gaze triggers. There are moments, depending on where you look, we wait until that moment happens so it feels like you discovered it. At first, we put that all over the place. People began to realize, oh, I see, this is kind of a game. I have to look in the right spot for the game to progress. And we didn’t want that. We wanted it to feel like it was story being told and it was natural, and so we actually cut that mechanic down so it’s only in a very few places. This is a cool technique, but overused, it gets tired.

I think we also did a good job of staying away from the uncanny valley. One of the problems, I think, in VR, is people are trying to go towards photo real. I think that you know you’re in virtual reality. You know this isn’t real. And yet if you try to go photo real, it feels creepy because you’re present in the space. And so I actually think we need to go towards cartoony stylized. I don’t want to go into virtual reality to experience reality, because I do that. So I’d rather go in there to experience something I cannot do in the real life, something fantastic. And so that’s why we wanted something that’s like, I could never experience in real life, and it think that was a wise choice.

WILL: In your ideation stages I’m sure you had some dreams as to what the future might hold for VR and VR cinema. So in a completely idealistic world, in the next 5 years, what is your ultimate VR cinematic experience?

MAX: I think where we’re leaning towards is episodic content. I think in the long run, VR is not as I said, I’m a believer, but at the same time I’m kind of a perfectionist, and I still think VR as a technology has a long way to go. I don’t think we’re ready to be in an experience for an hour, two hours. I think it’s mentally taxing for us because it touches so much of your brain, that I think a 10 to 20 minute experience is great. If you establish a world and characters you love, I want to keep going back there. I think that’s a good business model. It’s sort of a—try out this first episode for five dollars, and if you love it, you’ll keep paying for episodes and you don’t mind that six episodes later you paid a certain amount of money. But I think if you start charging something for 30 dollars, an hour, two hour long experience, I think that’s exhausting.

“I don’t think we’re ready to be in an experience for an hour, two hours. I think it’s mentally taxing for us because it touches so much of your brain, that I think a 10 to 20 minute experience is great.”

I want to have experiences—I think 5 years from now… everyone talks HoloDeck. I think the HoloDeck is many, many years down the road. Maybe 100 years. We’re talking about artificial intelligence and storytelling with branching narrative that is insane to think about. But I think that we will have narratives that the user colors. We’re big fans of Telltale Games, in terms of there’s a story experience happening and you can kind of color the story. You kind of know it’s not this rich, branching narrative that you can go in many different directions, but the choices you make do have impact later on, and we like that model as well. Plus presence, plus the fact that you do not have a controller in your hand and your controller is your body and your head, I think that’ll be magical.

“I don’t think I want people stuck in virtual reality for four hours on end.”

So I think that’s where we’re heading. I think that’s what people will want to continually go back to. If it’s an episode, I will watch it once every day of the week, or maybe every weekend a new one comes out. I think that s a great healthy use of virtual reality. I don’t think I want people stuck in virtual reality for four hours on end.

—–

Update 2/2/15 @ 3:32PM PST:

Palmer Luckey’s comment on reddit:

“There is a lot of incorrect information in this article – Max is a new hire, and he was definitely off on many things. I can’t clarify with concrete info on a lot of these yet, but suffice to say that May is no special month for the Rift, nor do we have price locked down for CV1.

As a more concrete example, we run everything (including internal prototypes) off a regular Falcon Northwest Tiki with a regular GTX 980, which is our preferred rig for shows as well. They cost ~$2,000, not $10,000, and are one of the smallest gaming PCs you can buy.

Same goes for wires and tracking technology – we are always researching new things internally, but we are a long ways away from wireless video transmission and alternative tracking technologies getting good enough from VR. I doubt anyone wants CV1 held up for tech that may never come!”

Update 2/2/15 @ 3:11PM PST:

 From Palmer Luckey on Twitter. 

@UploadVR @sundancefest @RealBenGilbert @benz145 @BenKuchera A lot of this is incorrect. New hire making several mistakes.

— Palmer Luckey (@PalmerLuckey) February 2, 2015

@UploadVR Too many things for Twitter, but release date, price point, PC size/cost are all incorrect. We use @FalconNW Tikis, not $10k rigs

— Palmer Luckey (@PalmerLuckey) February 2, 2015

@UploadVR @FalconNW Just normal form factors gaming PCs, which is very much our target for launch. Same goes for tracking tech.

— Palmer Luckey (@PalmerLuckey) February 2, 2015