One of the coolest things we’ve seen made with VR is a music video filmed entirely in Google’s creativity software Tilt Brush. If you have an HTC Vive headset, you can now download the world shown in the video and experience it for yourself. If you don’t have a headset, just watch the video above.
Jaymis Loveday worked with the Australian band Ball Park Music to create the experimental project for their song Whipping Boy. The album the song is from, Every Night The Same Dream, is available for pre-order now, but if you want to experience the song in VR you should be able to download the world (4.75 MB .tilt file), open it with Tilt Brush, hit play on the above video and then put on the headset. You’ll need the latest version of Tilt Brush with audio-reactive brushes.
Loveday explained on Reddit how it came together, shedding light on the project but also hinting at some truly transformative things that might be possible soon if you combine VR with filmmaking:
We were super lucky with the timing, I woke up on the 2nd morning of filming and loaded up Steam to be greeted with the 6.0 update of Tilt Brush. The clip was suddenly audio-reactive! We’d only done the first section (before the whipping demon guy) by then, so the rest of the piece was made with new brushes.
It took 3 days all up to create. We did a bunch of different tests on the first day, figured out how big we could go, and how we’d fit the different sections of the song together.
I’ve worked with this band for a while, and we decided to do a bunch of experimental pieces for this album, rather than just a single “traditional” music video. After we finished the last video (Nihilist Party Anthem) I gave them a go of my Vive, and they quickly agreed that we should do the next single in Tilt Brush.
My original plan was to “film” by putting the headset on a motion-control robot, and having it do smooth dolly/pan moves through the 3D space, but once we saw how cool it looked just holding the headset and “flying” it through the artwork, we ditched that idea. The spectator window has excellent motion smoothing, so you can get quite usable camera moves handheld. This also has the advantage of looking less “computer-generated” so it fits the hand drawn style.
As someone who’s spent a lot of time in VR and Tilt Brush, the piece is kind of exciting to me, but not mind-blowing. For people who don’t VR, their brains have been unilaterally melted.
The robots Loveday is talking about can make very fast, smooth or precise movements. If connected to a camera and/or a Vive controller, robots could offer a lower cost way of capturing the kind of footage typically reserved for high-end TV commercials or music videos. Here’s an example high-end robot paired with a super slow-motion camera and the breathtaking footage it produces:
As Loveday explains, you could theoretically hand-hold a camera with the Vive controller attached to it and film a scene with performers. Then you could feed the tracking data from the Vive controller to the robot and have it film the exact same movements as before, but this time filming the background of the scene. Imagine a foreground moving in normal speed but a background that’s timelapsed. This is just a theory right now as Loveday is still trying to get his hands on a third Vive controller in Australia that would let him try this kind of experiment.
We’ve seen apps like Tvori and Mindshow approaching new ways of storytelling entirely through VR. We also know how powerful mixed reality capture is, made possible by the SteamVR Tracking technology that’s part of the Vive system and being licensed for free by Valve Software. In that context, this music video could be a hint of very big things on the way.
