Virtual Reality has been talked about so much that it’s sometimes easy to forget how new the whole thing is. It’s also easy to miss out the fact that at its core, this isn’t a story about machines, but about people. Among all the new releases and the full-blown arms race around the latest hardware, we’re forgetting that what makes VR so exciting has as much to do with art as it does with science and tech.
Virtual worlds have been around for ages, as has this type of immersive technology. The real game changer is that now we’ve reached a place where the technology can be used to tell compelling stories that we can all consume.
Pioneering the New Age of Storytelling
“It’s important to remember that this is the equivalent of the 1900s in film,” says Luke Ritchie, Executive Producer at Nexus VR Studio. Based in London, they’re a division of independent creative studio Nexus, which has produced a host of award-winning works over the past 20 years and is now betting on VR content as the next big thing. He believes that we will look back at this time much like we now study the Lumiere Brothers’ Arrival of a Train at La Ciotat reel, which had audiences jumping out of their seats to get out of the way of the incoming train.
We’re only just beginning to figure out the potential of VR as a storytelling medium, so many of the rules, techniques and conventions that we take for granted in filmmaking and theatre, for example, haven’t yet been tried, tested, or codified. In sum, we need to get busy writing the VR rulebook.
Read More: VR Has a Storytelling Problem and Theater Will Save It
But do we do this from scratch, or do we try and adapt existing techniques from familiar media? Both, believes Ritchie. VR is a unique medium with its own set of parameters, but a lot of the craft skills that visual arts professionals have developed over the years can be adapted or reinvented for it.
“Everyone is still learning,” says Nexus Founder and Creative Director Chris O’Reilly. “You have to rethink staging, art directing, lighting, and there’s a very different production pipeline so we had to do a lot of specific R&D around that. It is really important that things are challenged in that space. Some of the less interesting experiences in VR film have been merely where you have the ability to look around and there has been no creative development around the idea of telling a story in space.”
“Film is quite easy to check, you just play back. But in 360, as you’re looking in one direction something can be happening in another, so doing post checking on something like that can be quite tricky.” adds Ritchie “I think we’ll end up writing coding scripts that automatically check for things such as pixel alignments and things like that. The processes aren’t that dissimilar though, the main learning curve is in how to tell that story in 360.”
360 film scripting, for example, merges traditional film structure with those gamification mechanisms. O’Reilly shows me what this looked like for their newest film Rain or Shine, which is a Google Spotlight short being released later in the summer. There’s a grid showing a “golden thread” with each element of the core story surrounded by variable paths that allow for deviation and exploration by the viewer.
The key to that type of scripting is shaping the narrative in such a way as to always guide the viewer/participant back to that “yellow brick road” even if they want to take the time to go down all the different paths available, exploring the peripheral elements and smelling the virtual roses to their hearts content.
“You might have a scripted story, but how do you know which characters to follow? You have to think outside the box.” Explains Ritchie. “In Rain or Shine we used this very directive soundscape where when you move away from the main action the music changes and starts to give you clues that you’re in a less critical story space, and characters can each have a different theme.”
Advantages of the Medium
As I watched a preview of the film, prompts guided me through the story without making me feel “herded”. At certain points in the narrative where the main storyline was supposed to unfold, a pigeon showed up which naturally made you want to follow it (although you could do so in your own time, or not at all) It reminded me of the Hansel and Gretel faerie tale where the children follow a trail of breadcrumbs to try and find their path through the woods (although in that particular story pigeons actually gets them lost by eating those breadcrumbs). Ritchie calls these “subconscious navigation devices,” and they are familiar to most gamers. Video game designers routinely use such mechanisms to give players a sense of mastery and agency where there are, in fact, only a finite number of options available to them. “The whole objective of this type of narrative is to get you from A to B without you ever thinking you were headed for B in the first place,” concludes Ritchie.
Another thing Nexus did when designing that experience flow was speak to architects, adds O’Reilly, as they’re very good at corralling people through expansive open spaces. Just think about how an airport naturally funnels you through the various stages of check-in, security, shopping, waiting and boarding, or how difficult it is to “go against the flow” when wandering around IKEA. That is essentially human experience flow by design, and it works just as well in the virtual world as the real one.
“We also spent a lot of time talking to people who worked on immersive theatre,” adds Ritchie. “Immersive theatre works in spaces, not scenes, and they already told stories in 360 space where they didn’t know where you were looking and you might be encountering the story in a different order than someone else, but they were still emotional experiences.”
The point he makes is that although new media and digital formats might seem closer to the heart of VR at first glance, once you delve into the production side it becomes clear that it has many more synergies with traditional storytelling mediums such as film and even theatre. Even in the inventing of new techniques, that physicality of the stage comes into play, as O’Reilly explains: “When dealing with a 360 space you come up against problems that you don’t have in traditional film such as “where do you put the crew?” So you have this crazy-sounding solution called “stacking” where you try and position people in a row with the tallest one in front and all the others behind, so you only have to paint one of them out of the scene.” He laughs as I suggest that they might start recruiting crew for VR productions taking into account their relative height and “stackability”.
Another recent example of this theatrical overlap is fabulous wonder.land, a Virtual Reality experience produced by the National Theatre in London to accompany their musical production inspired by Alice in Wonderland:
““In the play, Alice suffers from bullying at school and hugely lacks self-confidence, and the Internet becomes the rabbit-hole down which she escapes,” explains Toby Coffey, the National Theatre’s Head of Digital Development. “She deals with her self-esteem issues by creating this Barbie-esque avatar for herself, and enters this alternate reality by locking herself in the school toilet and going on her mobile phone, which becomes this sort of portal through which she can embody that alternative self.”
Over 90,000 people viewed the experience, which was also nominated for the Cannes Festival’s first-ever VR stream. It was enough to convince Coffey of the enormous potential of the medium to engage with audiences. It also led to the development of their Immersive Storytelling Studio, which during the Autumn 2016 season will collaborate with the National Film Board Canada to explore and advance the language of creative non-fiction VR storytelling.
While the potential to translate Lewis Carroll’s classic story into Virtual Reality was always apparent to Coffey, the question soon became how to execute it effectively on a relatively small budget. Their core remit, as far as he was concerned, was to enable people to experience something they wouldn’t normally be able to: “the key was for them to be able to fall down that rabbit hole, “ he explains.
Read More: Ultimate Beginner’s Guide to VR Storytelling
They landed on the idea of a music video as a short, ephemeral and self-contained experience that tied into the production’s existing musical theme. The designers for the show had already worked on 2D projections for the stage production, so the next step was for them to collaborate with a developer agency which ported these onto VR using Unity.
The Unity VR developer platform can, in fact, be effectively be described as a kind of VR staging software. It replicates many of the same mechanisms as you’d encounter when placing and lighting objects on a stage. You’re effectively building an environment from scratch, and although creators might have different tools at their disposal to manipulate variables such as texture, colour, reflections and so forth, it is not difficult to see how the skill-set of a stage manager would be transferrable to the creation of evocative virtual worlds.
“A lot of the practice of theatre goes into 360 film,” agrees Coffey. “Elements such as staging, positioning of objects to create the right perspective, the use of lighting and textures are all part-and-parcel of stagecraft, and integral to the creation of immersive VR experiences.”
Award-winning multimedia artist Oscar Raby recently undertook a two-week residency at the NT to research this dialogue and synergy between the languages of VR and theatre: “Framing in film works as a threshold between the story – the world which the characters inhabit – and our experience of it, the audience’s realm.” Virtual reality, he explains, turns that on its head, so that the user becomes a performer in their own narrative, shifting that frame of reference to a point where it becomes difficult – if not downright impossible – to identify where that so-called “fourth wall” between the two worlds lies.
Down the Rabbit Hole
To deliver the fabulous wonder.land experience to pre-theatre audiences, the NT also staged an installation that re-created the school toilet cubicle to which Alice escapes, and users sat on the loos wearing the VR gear as they plunged down that proverbial rabbit hole after her.
“We actually missed a trick there,” Coffey tells me. “People were pulling the water closet chain thinking that was the trigger to start the experience. We underestimated how immersed people would be in the idea.”
The kind of interaction that Coffey describes is interesting because it shows how there is a natural expectation with VR that the virtual and real worlds should effortlessly blend into one another. This is why we’ve seen so much excitement around the development of Mixed Reality, which overlays elements from Virtual and Augmented Reality, weaving them together into one experience. It’s about enabling social and emotional experiences that will really connect with the viewer, and to do that, says Ritchie, the skills that they’ve been working with for the past decades are essential.
“If you go into that world without the creative craft skills, you can get your head around the technical side, perhaps, but you will be sorely lacking in the storytelling department. All of those skills we have from filmmaking and production transpose but have to be re-thought through. I think that London has a huge amount to bring to that conversation. We have the tech ecosystem, the creative talent, the brands and the indie subculture all coming together. That’s the sort confluence we need if we’re going to invent a whole new way of telling, and re-telling these stories in VR.”
“It’s complicated though because VR is not one platform,” says Ritchie. “Making a project for VR is not like making a project for film or TV where there is a standard definition about what that format is. So VR in itself is a moving target between incredibly immersive interactive 360 walk around experiences. There’s quite a big difference between the kind of experiences you get on a HTC Vive and those where you have a pre-baked 360 video where your interaction boils down the fact you’re able to look around that environment.”
Read More: How to Direct Audience Attention in VR
There is quite a lot of room for innovation even at the simpler end of the spectrum, however. O’Reilly shows me another of their recent projects, a 360 music video called Chapita along those “pre-baked” lines, but which director Eran Amir used to pilot a host of innovative techniques.
Instead of traditional “stitching” between multiple cameras, the video uses a single, extremely high-resolution camera to repeatedly capture dancer Mimi Jeong in different outfits and choreographies within the same space. To create this “army of clones” they needed to eliminate variables such as camera movement to such an extent that it was necessary to screw the rig on which it was mounted to a concrete floor. There is little doubt, hearing them describe the process, that producing innovative VR content involves not only a lot of creative thinking and know-how, but a huge amount of meticulous hard work. It is not for the faint hearted, but Ritchie and O’Reilly stress that the extra effort is worth it.
“As we see 360 content starting to flood onto YouTube, it will be the ones who grasp this that will stand out. There’s a huge opportunity here to retell all these classic stories in a new medium, just like what we saw in the inception of film, and that’s the exciting thing for me,” concludes O’Reilly
But are we moving towards a future where all our films will be in 360? Not necessarily, says Ritchie.
“A lot of directors are very authorial, and their craft hinges around directing the gaze. To give up that control freaks a lot of them out. You’re going into a very different world in terms of how you tell a story, and while you’re gaining new elements, a huge chunk of the director’s toolkit has to be given up.”
In terms of technology, it will be interesting to see how major new entrants in the hardware space such as PlayStation and HoloLens will enable different types of content creation and delivery, but also how far and how fast Google can push mobile VR to the masses. The hardware arms race will undoubtedly influence the type of content that creators make, but speaking to those creatives it’s clear that it’s in no one’s interest for that to be a one-way conversation dictated by technologists.
If we’re all going to go down that yellow brick road, it’s the storytellers that will need to give a heart to that VR Tin Man.
—
Alice Bonasio is a freelance writer with work appearing in well-known publications such as The Huffington Post, Newsweek, Playboy, and more. You can follow her on Twitter: @alicebonasio.
Featured Image: Shutterstock via Wayne0216
