Ultimate Beginners Guide to Virtual Reality Storytelling

What is VR?

Virtual Reality [VR] is a magical alternate universe where all of your wishes come true and where pain and suffering don’t exist. Just kidding. Ok, seriously VR is the use of computer technology to create a simulated environment. Unlike traditional user interfaces, VR places the user inside an immersive experience. Instead of viewing a screen in front of them, users are “immersed” and able to interact with 3D worlds.

Short and sweet: in VR we create worlds and invite individuals to experience them. This means that in VR you’re not just an observer but a participant in the story with an option to influence the story itself. That’s pretty rad!

Who is this for?

This guide is for everyone who is dipping their toes into Virtual Reality storytelling and filmmaking. This includes VR game developers, filmmakers and amateur enthusiasts who are discovering the freedom of the new medium.

I won’t go into the theoretical aspects of filming because that’s not the domain or the scope of this eBook. I also won’t talk about how to choose the right equipment, camera system, editing processes or which software to use for post-production. Instead, I’ve gathered the best practices and tips that the most successful filmmakers used on their projects. I’ll strictly focus on creating a good story.

It’s important to understand that we are all experimenting and learning about VR. For that reason, you won’t find many books on the subject. All the progress and learning is currently happening in the real world examples.

This eBook’s goal is to give you a boost and understanding what VR can bring to the table. It’s a new medium that gives you as much or more power and creativity than traditional filmmaking. But as Spidey said: “With great power comes great responsibility.” I wish you can avoid the pitfalls that can set you back in your next awesome VR feature film.

I’d also like to thank you for taking the time to read this. VR has been making waves since the start of 2016, but for people to grasp it and have a positive experience from it, they need good content. That’s where you come in. By reading this you set yourself on a path of storytellers in the new era. You’re making history already!

Virtual reality was once the dream of science fiction; but the internet was also once a dream, and so were computers and smartphones. The future is coming”. — Mark Zuckerberg

Types of VR

Virtual Reality comes in different forms. Either Computer Generated Images (CGIs) and display live images from the physical or real world. There are Heads Up Displays [HUDs], or Heads Mounted Displays [HMDs] that can superimpose CGIs onto the real-world . This function is often referred as mixed or Augmented Reality.

The 360-degree video, also known as 3DVR and Stereoscopic VR, use multiple cameras that capture the image from 360 degrees. There are some that would argue that 360 video is not really a VR. Technically they are correct. A standard 360 video is just a flat equirectangular video displayed on a sphere. Think of it like the face of a world map on a globe, but with VR, your head is on the inside of the globe looking at the inner surface. As you move, the head tracking on your device moves with you, giving you that feeling like you are inside the scene.

360 video is usually augmented with stereoscopic 3d which adds another level of immersion by adding depth between the foreground and background. This is achieved by shooting the scene with two lenses side by side, that give you a feeling of different vantage point per eye. It can appear strange if not done correctly, but great if done right. With stereoscopic 3D in VR, that depth information has to be overlaid and mapped to the sphere. Because of parallax between cameras, this can be tough to achieve. Any minor flaws or “stitch seams” in the footage are magnified in 3D. These anomalies occur in different places per eye — which makes it uncomfortable to watch or even promote headaches, eye strain, and motion sickness.

The goal of 360 and VR video is the same — total immersion of the viewer in the created world. Once the viewer feels comfortable in the created scene, we can serve him with a story.

What do I need to know about VR Storytelling vs Traditional Storytelling?

Virtual Reality is now firmly established. The technology that we have been craving for is now at our fingertips; We have the ability to spread the word and show the people the freedom of the new medium, and we can make them comfortable with VR to use the right way. This means establishing new routines and especially showing them what they can do in the world you have created for them.

In the old days, we only had a pantomime and a black and white camera standing in front of the theater. Then we learned to use the camera, panning it around, trying out different shooting techniques and points of view [POVs], zooming into people’s faces, cross-cutting, multiple locations, etc. Thus, the cinematography was born.

VR to a film is what film is to pantomime. It will completely change the game when we figure it out. The POV and cinematography break the 4th wall that separates the actors, and the audience goes away. That’s why in VR you have to find creative solutions on how to guide your audience.

The viewers of VR are not just passively looking at a flat screen. Suddenly, your viewer is inside that “frame”, freely looking around and within the next few years, they’ll be able to move around too. You can get an idea how impactful these connections are.

This unique sensation of “being inside ” is called presence.

Chris Milk, founder of VRSE and VRSE.Works calls VR the ultimate empathy machine.

“VR is difficult to explain because it’s a very experiential medium. You feel your way inside of it. It’s a machine, but inside of it, it feels like real life, it feels like truth. And you feel present in the world that you’re inside and you feel present with the people that you’re inside of it with.”

For storytellers, your first thoughts might be cramming all the traditional knowledge you learned in film school into a new medium. This can lead to negative consequences because you’ll soon find out that VR goes beyond traditional filmmaking rules and techniques. With that said, it’s important to remember that the stories don’t change, but the way we tell them in VR will change.

Before you start filming… I mean creating stories

There seems to be a confusion if VR filmmaking is possible. Most of the confusion comes from not understanding the difference between 360 videos and VR. The simple differential factor is one element — Interactivity.

In reality, your actions (and inactions) have consequences. The decisions you make change the world around — or your experience of it — sometimes in profound, unexpected ways.

These changes are not scripted unless you believe in some kind of outcome.

Virtual Reality should then conform to the same rules. The decisions in VR should change the constructed world in an unexpected way. This means the VR world should change without predetermined scripts.

Principles:

Hundreds of VR projects are being developed by various artists and studios; I’ve collected few core principles, or should we say axioms, that you will have to take into consideration before you even start shooting.

They are:

Creating a great story is going to be a battle of compromises.
Think of yourself more as an influencer then a director. It’s all you can do, because you’re not showing your audience just a frame but a world.
The core of VR is experience and presence. As a director, you’ve got the huge responsibility to keep your viewer engaged. Your viewer can’t switch the channel or look the other way — the VR device is mounted onto his or her head, and something major has to occur for him or her to pull the HMD off.
VR Storytelling is a lot like being in a video game.

Tip 1: Ease it in

The first goal we have to achieve with the viewer is making him feel comfortable. Oculus Story Studio recommends 30-second introduction video. This will give the viewer time to adapt to the headset first and familiarize themselves with the new medium.

The Ultimate Beginners Guide to Virtual Reality Storytelling — Lost

This “settling in” period is enough for most people to adapt to the environment and relax. In Lost, you start in a darkened space. A pinpoint of light appears: it’s a Fi the Firefly. The Firefly buzzes around from left to right. It’s placed there with an intention for the viewer to follow it around and get used to the movement in VR. You soon realize you’re looking at a forest, and then the movie’s opening credits start as light strings swell in the background.

First-time viewers are used to watching flat screens; most likely they will just stare straight forward. You can use narrative voice or visual cues to encourage them to look around. Give your viewer a permission and encourage their curiosity.

Tip 2: Guide the Viewer’s Attention… Or not!

You will have to decide beforehand how do you want to treat your viewer. Would you like to guide him through the story and lead her where to look? Should we discourage a viewer from looking somewhere else?

Give the viewer complete control of the situation? For directors, relinquishing control is a completely new concept. If a viewer really wants to spend the entire time looking up at the sky, even though the narrative is happening in front of him, the director of a VR movie has to accept this. Cinema has plenty tricks in the bag to tell stories and guide viewers attention.

You can use different lighting or sound cues, a focal point of a character’s POV onscreen, or even verbal/action cues. The important fact to remember is that our gaze is attracted to two things: movement and faces.

The double edged-sword of control is it may feel inauthentic. Since VR is the medium about your viewer’s freedom, I suggest you embrace it. Instead of spending a considerable amount of time and energy attempting to control something that you fundamentally can’t focus on delivering a world of wonders — a unique experience that makes people feel.

Tip 3: Presence

Total immersion is often the end goal of VR storytelling. The first obstacle you need to overcome is a functional technology. This means you have to prevent or repair the stitching issues; use the highest quality picture possible and the right sound equipment. The last thing you want is to cause headaches or nausea to you viewers and thus ruining the chance to show him your world.

Second, you have to trick the viewer’s brain the VR world is “real” by figuring out a way to trick the mind to accept things that are clearly not real. Remember Kevin Spacey’s line from The Usual Suspects?

“The greatest trick the Devil ever pulled was convincing the world he didn’t exist”.

Montreal-based studio Felix & Paul had an “a-ha” moment on one of their first projects. They brought a non-professional actor, an older Chinese woman with gentle personality, the lady was walked into the church and sat down next to the camera. After few seconds, she looked at the camera as if was a person. That 20-second scene blew people away. They couldn’t stop thinking about the woman!

Use characters that look directly at the viewer. The intimate connection evokes authentic emotions of the actor and the viewer. This is a perfect example how VR has the power to break through the 4th wall.

Henry

There are situations where you don’t want the viewer to be the center of attention; especially action scenes like high-speed car chases or martial art performances, where it makes sense to use different perspectives.

Tip 4: Pacing

A sudden shift from flat screen frame to a 3D immersive world can be confusing and overwhelming at first. If you’ve done a good job with settling in your viewer and making him or she feel comfortable you’ve completed the first goal.

Oculus Story Studio is using the ritual of “Settling in & Setting the Scene”. In “Lost”, they used a firefly which flies a bit to left and right, just so the audience get used to the concept of looking around. The firefly is the only thing they see before the mysterious forest appears.

Your viewers, most likely, aren’t used to VR films and for most of them, this could be their first VR experience. Because of such a huge jump in information density from tv screen to immersive environment, think about how you’d gently ease them into the story.

You may decide to move the camera in your story but be very careful with an undulating movement, since it can cause sickness.

There’s a thin line where the story is interesting because there is a lot of things going on at once, beware of overwhelming from sheer information overkill. If there’s just one thing going on, the scene will feel empty and inauthentic. VR should emulate a real world environment. Oculus calls this “Spatial Story Density” — a three-dimensional storytelling with the space around us.

Remember our first principle, we will have to make compromises.

Tip 5: Conditional Storytelling

In “Mass Effect”, a popular Sci-Fi Role Playing game, we are put in the role of Commander Shepard’s point of view. We are responsible for our spaceship, the SSV Normandy, its crew members, and choosing which mission to tackle. As we progress through the game, we are forced to make difficult decisions. The decisions carried huge consequences — a death of a crew member, genocide of a certain race, even the destruction of the whole planet.

Mass Effect. Make a choice and face the consequences…

In VR, we have the ability to give the same power of decision making to our viewers. How much power we delegate is entirely our decision. These decisions can be thought through and mediated. For example, our viewers come to a checkpoint where the action stops and are forced to make a decision to continue. Choose a path — will our viewers take the red pill or the blue pill?

But we can go a step further and predetermine an expected reaction. VR already have the capability to go beyond the usual content experience; It can simultaneously engage visual, auditory, and haptic (touch) senses. Virtual space within the story can respond to speech and gesture inputs from the viewer. This might be technically much harder to achieve, however, as it simulates reality much better than thought breaks in the previous case.

Let’s assume you are in a scene where the actor and our viewers are in a passenger seat driving in the a car. The street seems empty and the driver speeds up. As the car approaches a pedestrian crossing, we notice a child running down the street. If the driver doesn’t hit the brakes, an accident will occur. If we scream, the driver will try to stop the car, but if we fail to notice the child or decide not to react, the outcome will be different.

Tip 6: Experiment

VR is an exciting sandbox where we find endless tools and tricks to play with. The tips so far are not the rules of filmmaking, but rather suggestions. However if our tips were the rules, we should break or bend them any way you can.

Henry Ford’s famous statement was: “If I’d asked my customers what they wanted, they’d have said a faster horse.” Instead of making the horse faster, he built a car. Something that hadn’t even existed in people’s minds.

I dare you to be as bold in VR storytelling in the same way. Make mistakes because we will either learn something that doesn’t work or we’ll find an ingenious solution. Charles Goodyear accidentally invented vulcanized rubber when the combination of rubber and sulfur dripped on the hot stove. Much to Goodyear’s surprise, the rubber didn’t melt but actually hardened. This accidental product is used on almost every vehicle today — it’s your tires.

Tip 7: Test, Test, Test

Product and services are always tested before release into the public. So should your VR creation. Since we have established a developmental l relationship with our work, we are not the right person for testing our design. We might be too critical or forgiving of our creation.

Ideally, we should show our work to random people that we don’t know since they have no vested interest in outcomes and won’t stroke our ego.Their observations, feelings, and especially reactions watching our creation are pure gold.