We talk to PolyArc (Moss) and Schell Games (I Expect You to Die) on the technical limitations of the Oculus Quest and what untethered control means for virtual reality.
[This feature originally published in July, 2019. We’re re-publishing it now.]
With the launch of the Oculus Quest several games that were previously on the company’s Rift headset were ported to the new hardware. Unlike games that emphasize standing or moving around such as Beat Saber and Superhot VR, several of these ports were simply stationary, sit-down experiences.
I spoke to the developers behind two of the more critically-acclaimed sit-down games — Polyarc who developed Moss and Schell Games who developed I Expect You to Die (IEYTD) — about what the Oculus Quest brings for their stationary titles and the challenge of facing the Quest’s technical limitations.
The Limits Of Oculus Quest
Schell Games had to deal with a variety of specific issues in porting IEYTD to the Quest’s mobile hardware.
“We had a number of issues, GPU issues and CPU issues,” says Mike Lew, the Engineering Director for I Expect You To Die ports. “On the GPU side, the larger issue is something called overdraw. Overdraw is really expensive on a mobile architecture. Overdraw occurs when you have two objects overlap and maybe one is transparent. And so you’re writing to the same pixel more than once. Some of the things we did to reduce this was we had to downsize a lot our particle systems.”
Another issue was the number of draw calls going to the CPU, the number of times in a scene you are instructing the CPU to create something in the game. A typical level in the PC version of IEYTD had 400 to 500 draw calls. For the Quest version, so the game would not bottleneck at the CPU, the team had to reduce this to 130.
To achieve this reduction, they had to use a technique called Atlasing, to combine assets in the game, to reduce the sheer number of things in each puzzling level of the game.
“When you are building an environment, you are building things one piece at a time,” says Lew. “When you look at the Submarine Castle [a level in the game], the whole thing isn’t one piece. There’s the castle walls, and the pipes, and the valves, all the little intricate pieces. When they are created, they are separate. So one of the things, we did is called Atlasing. We can take a lot of these meshes that exist in a scene and combine the textures into a single texture. So they end up resolving in one draw call instead of X number, where X is how many items there are. That really saved us a ton.”
To reduce the load on the GPU, they used some software assistance that Oculus built into the Quest: Fixed Foveated Rendering (FFR). This is when there is a set pattern of areas along the edges of the screen where the game’s resolution is reduced. Less pixels to push means less for the GPU to worry about.
“FFR has a few settings, where it goes from low to high. I think we did medium,” says John Kolencheryl, Project Director of I Expect You To Die ports. “What that gives us is a lesser number of pixels to draw and that ended up being a big win for us. We started off with a High setting and then you can see when you moved up and down you could see the pixelation that’s happening. After several optimizations, we asked ourselves how low can we get with Fixed Foveated Rendering. We were super happy with the result.”
FFR is especially important considering the way VR headsets project an image into your eyes. On the screen for each eye will be an image that has to be warped, to have a bulging center and curved edges. This is called “barreling.” This is done so that once the image goes through the lenses, which warps them the opposite way as they enter your vision, you have a flat image for each eye.
“The pixels don’t have the same density throughout your vision, because the lenses are curved,” says Lew. “So you see the pixels in the center of the screen as being larger than the pixels in your periphery. This means that to produce a high quality image, you want to be pushing more pixels to the screen, because the ones you look at the most, in the center, are larger. So FFR is the counter to that. You can shave off a lot of the really dense pixels in your periphery. It actually ended up affording us about 10 frames per second in our performance. It made a huge difference,” said Lew.
On top of all the CPU and GPU issues Schell Games found, there had to face the memory limitations of the Oculus Quest too.
“Oculus reports that you have around 3.75 GB of memory to work with,” says Lew. “Realistically, it’s closer to 2 GB and that’s because there’s still the operating system and other stuff running in the background. If you use up too much memory in the game, you will start running into issues with the things running in the background, which includes the software for tracking the controllers. ‘Why is my tracking degrading?’ Because you are hitting a memory cap that came up faster than you’re expecting. We reduced selectively some texture sizes and things like that to make up for those differences. Once you learn the rules of thumb, it’s not too bad.”
The engineers at PolyArc had their own specific issues with the limited performance of the Quest when porting third-person adventure game, Moss.
“We didn’t want to compromise on the design of the game,” says Josh Stiksma, principal designer at PolyArc. “We really wanted it to be the same core experience, the same connection with Quill, the same immersion in the world. But we hit that limitation when we hit the graphic-side of things. We couldn’t push as many verts, we couldn’t have as complex shaders. If you look at the game that Moss is, it’s quite a large world, with a lot of different diorama rooms. We got over 80 of them, by the time we released on Quest with the new content.”
While Moss is known for its lush art featuring a quaint forest world, the developers had to scale back on the graphics of the game for it to work on Oculus Quest. This not only meant simplifying the shaders, much like Superhot and others had to do, but it also meant redrawing all the texture art.
“We had to recreate a second level of all of our artwork to make sure it worked on the Quest. It was a significant amount of work,” says Stiksma. “One of our dedicated technical artists started early on doing the transition, to crack open all the levels and figure out the right way to optimize them. It took maybe 6 to 9 man-months for the art, a considerable amount of time so that we had multiple people working on it. It was a large task.”
But Stiksma stands by the resulting version of Moss that came to Oculus Quest.
“One of the things we are super proud of, that a lot of players feels that we still hit a really high quality bar,” says Stiksma. “For us, that’s really important to the experience. There are similar objects, but they’re running much simpler textures. We were able to stay true to the design of Moss in nearly all the aspects.”
Oculus Quest And Standalone VR
At some point in my discussions with developers about the technical and graphical hurdles to overcome when developing for Quest, the conversations turn to the virtues of the platform.
“We definitely believe in playing to the strength of the hardware,” says Stiksma. That’s how we are going to continue to make amazing VR experiences. I think the Quest is definitely the one pushing that to the forefront.”
One of the big components of how Quest switches up the developers’ games is the freedom to move, even if the original game was not developed with 360-degree movement and room-scale space in mind.
“We initially envisioned Moss as a seated experience, where you can sit down and play with this diorama, go on this meaningful adventure with Quill, and be invited into this world,” says Stiksma. “Where 360 might be more attuned to more room scale, moving around the room, see everything that’s behind you, experience. That’s something for us to explore.”
It was a similar issue with Schell Games, as they brought a seated experience to room-scale hardware.
“We talked about how on the Quest that players will be able to get up and just walk about and go through geometry,” says Kolencheryl. “We decided not to do anything about it. Keeping feature parity [between VR platforms] is something that we want to maintain. There are ways to break our game if you walk around. But in our playtests we have not seen that happen much. And usually it is done by players who are curious to what goes on beyond the level.”
Both companies designed additional content for their games in recent months. Moss includes several new levels called the “Twilight Garden” when it launched on Quest, with those new levels recently moving to other platforms. The team kept Oculus Quest in mind when it came to those new levels.
“What affected how we crafted Twilight Garden, we approached it with a more of an optimized mind set when we were designing the spaces: what would work really well with the hardware of the Quest,” says Stiksma. “We were constrained by other things like space and your range of motion, your freedom of movement, 180-degrees on the PlayStation VR, for example. We can’t go super crazy with some of the things that we think are amazing advantages of having the Quest, but we did really want to focus on stuff that would showcase what was possible with the hardware.”
Similarly, Schell Games recently released a new level for I Expect You to Die, called “Seat of Power.” This level, too, required extra considerations with Oculus Quest finally launched.
“In ‘Seat of Power,’ — this is a minor spoiler — there is a moment where you have X-Ray vision,” says Lew. “That had some challenges to meet our performance. We ended up reducing the resolution of the game to 80%. But you are in X-ray vision mode; you don’t get all the colors. It’s very abstract from a post-processing effect. And so it was really hard to notice the change in resolution. So it was a really hard to notice, so that ended up being a pretty good solve for us without impacting our design of the level too much.”
Schell Games, on the other hand, has already announced sword fighting game Until You Fall. This game, unlike IEYTD, is a new title in a post-Quest world and supports players being on their feet.
“For Until You Fall, we are supporting sitting and standing,” says said Dave Bennett, Project Director of Until You Fall. “Because Until You Fall is an action game where you are fighting for your life against these dudes with swords, a big thing is, because we need to support the front-facing cameras, is to build reminders and safety features so that if you turn away from the camera for so long, it will let you reset. So you turn in the game world, but not in the real world. So it’s a little weird. You can quick turn toward the enemies without having to turn your physical body. It’s finding ways to let you move within the game environment comfortably,”
And the developer’s process of adapting IEYTD to the less powerful Quest helped them create Until You Fall.
“Even for higher-end VR, performance is still super important,” says Bennet. “The way you go about building the levels, the assets, everything is super intentional. Trying to reduce the draw calls, reduce the strain on CPUs, because you drawing everything twice, once for each eyeball. And then if you are going to increase the post-processing, that’s a multiplier on top of that. Especially with Until You Fall, where we are trying to go for this crazy neon, 80s hellscape where we want to get this cool glowing effect in this really specific style, we have to make a bunch of concessions in certain areas to make sure to preserve these post effects. It is really about making choices.”
These decisions made for Until You Fall, and the very existence of both the Moss and IEYTD ports for Quest, offer a view into the strange position developers find themselves in. Do you make games for the innovative new headsets, like Oculus Quest or Index, that have smaller audiences? Or do you go where the players are on other headsets, despite the movement and play area limitations?
“The Quest gives us way more freedom than maybe the other platforms in some areas, but you can’t design for the best part of every platform,” says Lew. “You got to maintain platform parity. So—“
“—the worst part of every platform,” finished Bennett with a laugh.
But in all seriousness, developers have this awkward push-pull situation while VR goes through its growing pains: make cool games that capitalize on the latest tech, and yet be beholden to legacy platforms.
“Until more headsets out there have the capability of going 360, we are probably going to be a little bit beholden, just because we want to reach as many people as we can with our games,” says Stiksma. “We’re going to start looking at its little bit more, with the Quest pushing that. I think other studios are going to take more notice of it. It’s something for us to evaluate.”
Regardless of where these developers approach these considerations in their games, the way they look at it, it is one step toward the future of VR.
“I was quite impressed with how well it performs,” says Lew. “I know we’ve talked about performance concerns, but it easily is the best standalone device I’ve personally ever tried. I was really impressed with the hardware when we got it…So as a developer we are really excited for this new hardware. It’s really rewarding to find new methods of input or new ways to push presence and immersion in VR.”
