McKenzie Stubbert is a resident composer for original VR films from Within, and is responsible for the scores behind more than 20 VR films so far, including The Displaced and Clouds Over Sidra.
With McKenzie’s extensive experience, we thought it might be helpful to explore some of the things that have been learned about sound in VR over the course of all those projects, and asked Stubbert some questions over email.
Upload: Are there any significant things you note others frequently do wrong when it comes to music in VR?
MS: We are in such an experimental age that I believe the only thing you can “do wrong” is skip the opportunity to incorporate original music. As we try to figure out the current and future role of music in VR, every opportunity to incorporate it – as a creative participant in interactive experiences – is one that gets us closer to knowing what does and doesn’t work. Thinking about things in terms of ‘right and wrong’ inhibits breakthroughs during experimentation.
Upload: Can you point me to 2-3 projects that display different advances in the way music is integrated into an experience?
MS: My favorite piece of music by far is Jonny Greenwood’s score for Mike Tucker’s beautiful VR piece “Tana Pura”. It’s one of those marvelous experiences that will no doubt seem very simplistic one day, but conveys a sense of what being “inside” of music feels like. “La Peri” is obviously an older piece of music (it’s from a ballet by the great French composer, Paul Dukas), but it pulls you inside of the ballet as both a participant and viewer in what I would consider a new way.
What’s great about VR is that we have an opportunity to return to a less literal artistic experience. Cinema enabled us to tell stories that resembled real life so well that mainstream culture moved away from things like ballet. I’m looking forward to seeing how people take older, more abstract art forms and explore them in this new context.
UploadVR: What sort of study do you do, or inspiration do you pull from, when thinking about a music project for VR? Are there other works that have inspired you?
MS: My life is full of studying and looking for inspiration. I continue to practice instruments, learn new recording/mixing techniques, study traditional orchestration while applying programming knowledge gleaned from years of making samples sound exactly as I want them. I also read a lot– and watch films/TV/VR like anyone. As my work deals in the conversion of emotion into music, the fact that it may or may not be in VR doesn’t change the inspiration of that. It’s the needs of the medium that is different with VR. And that is ever evolving and changing as we move forward.
UploadVR: What about things like branching stories or responsive storytelling that activate elements only when you look at them — is that something you are working with in your music in any way? Can you provide examples if so?
MS: This a pretty secretive business, so I’m not usually allowed to speak specifically about projects until they are released, but yes, absolutely. In the same way 360° pieces are inspired by traditional cinema, interactive experiences are naturally inspired by video games. It can be hard to think of solutions outside of those two giant creative mediums, but everyone is trying. Getting composers involved as early in the process as possible allows for the existing techniques to be experimented with, just like visuals and sound design. This is the great hallway we are all going down as we incorporate the 360° with real-time rendering and gaze-based sequences and I can’t wait to share with you what I’ve been working on. The future is happening – slowly, but surely.
