‘Mindshow’ Is VR’s Missing Character Animation And Storytelling Tool

by Ian Hamilton • August 7th, 2016

Mindshow from Visionary VR is the virtual reality software I didn’t know my life was missing. Shown publicly for the first time at the massive VRLA meetup this weekend, I found Mindshow accessed a part of my personality that’s been hidden below a veneer of seriousness for too long.

Back in high school I performed in a number of plays as an actor (including one called The Veldt I find myself thinking about from time to time). If I think back deeper into my childhood there are memories of endless summers in the back yard inahbiting a variety of heroic figures. What I’d forgotten in the years since these performances is what it felt like to loosen up a little bit and embody someone else. While becoming who I am, I’ve forgotten what it’s like to be anyone else.

Mindshow reminded me of this lost part of myself. What I saw when I stepped inside the HTC Vive were the basics of character animation software making use of head tracking, hand tracking and the Vive’s microphone. Altogether, the tools let me animate the characters and props in a scene.

mirror-mindshow

Checking out an alien avatar in a virtual mirror can let you get a feel for your character. Pressing a button on the controller changes the character’s expression.

“You put on a puppet show and you’re wearing a costume,” said Gil Baron, Visionary VR CEO. “It’s about taking the risk and the judgement out of it and then it’s liberating right? You can do what you want and find joy. . . I feel like when you see your character’s hands first-person then you look over and see a mirror and make a smile — that’s magical.”

Toybox

What avatars look like when you only represent the points that can be tracked with first-generation consumer VR — the head and each of the hands.

The problems Mindshow tackles are very difficult to solve. Current VR equipment generally can’t track facial, knee or elbow movements. Guessing at the correct locations of these body parts is just too difficult. As a result, when someone is brought into VR they are often represented as a ghostly disembodied head and hands.

“We’re…starting from a place of three inputs and your voice,” said Baron. “It only gets better from here.”

Still, software like Raw Data (hands-on) and Mindshow do try and make those guesses. The approach isn’t perfect yet, but in Mindshow, for example, it can let me inhabit an alien and slowly sneak up on a human. Using my real-life hand and head movements only, the software guides the full-bodied alien to take a few steps forward. As I speak into the microphone my voice becomes the alien’s, and a press of my thumb on the controller turns its expression into a fierce snarl. A few arm swipes knock virtual crates out of the way to surprise the human standing on the other side.

mindshow-alien

An example scene from Mindshow. The software enables a kind of “asynchronous improv” which lets you act out each character’s part one at a time.

After animating the alien it is time to animate the human. I switch perspective to the human and the scene I just recorded of an alien attack is playing out around me. When the boxes go flying (not depicted above) all I have to do is scream and cower in fear to complete the scene.

The idea is that this piece of software is the core of a creation platform Visionary VR is developing focused on capturing live performance. The trailer for the software shows the ability to toy with the scale of avatars — one of the most compelling things I’ve ever done inside VR. Ultimately, Mindshow also feels like the missing piece of Tvori too

Read More: Tvori’s Impressive VR Creation Platform

It’s easy to imagine a multiplayer app that merges the best parts of scene-setting in Tvori with the best parts of character animation in Mindshow for a fully realized VR creation platform. You could set up and animate a scene on a table in front of you with Tvori and then teleport inside the scene to animate the characters with Mindshow. You’d have a completely hand-made world others could visit and modify in VR, or you could pull out a virtual camera and record a cartoon set inside the same world you’ve just animated.

The part of Mindshow I saw was clearly a small sliver of what Visionary VR hopes to build. The startup was co-founded in 2014 when Jonnie Ross, Adam Levin and Cosmo Scharf (also co-founders of the VRLA conference) teamed up with Baron (a visual effects veteran) and game designer Luke Patterson. In March this year they raised $6 million.

“For us the most important moment is the moment when you actually bring a character to life,” said Ross, the company’s chief creative officer. Then layer in reactions and other elements and “you’re bulding scenes and putting on a puppet show just like when you were a kid.”

“That has epic value.”

Visionary VR is planning to give more people access to the software later this year.

Tagged with: , , ,

What's your reaction?
Like
Wow
0%
LOL
0%
Dislike
0%
  • Can one view the content cross platform (Oculus, Vive, Gear, SMARTvr, Cardboard)?

    • Ian Hamilton

      What I saw was a very small part of the overall platform they are building, and they didn’t want to talk on the record about much beyond what I saw and described in the story. So I can’t answer your question definitively. That said, it seems like it would make sense for Mindshow to pursue multiple ways of sharing the story.