Google’s Tilt Brush Is Going Places You (and Google) Might Not Expect

It’s Saturday morning. I’ve both cooked and successfully eaten french toast. Wife and kids are away for the morning. Cats are fed.

Check, check, check. The core essentials are in place.

Let’s go.

—

Google’s Tilt Brush team gave us a peek into what’s coming down the pipe last night and my mind has been buzzing ever since. If you’ve not seen their teaser video, I strongly recommend you take a peek:

There are a bunch of things here that I’m going to try to unpack. I might be writing this more for me than you dear reader, so buckle in or jump out as you see fit.

Before we talk about the future, let’s talk about the past and something been clawing for attention in my mind since May of this year.

Tucked away in a corner of the internet is a YouTube user who immediately stood out like a blazing sun to me for two reasons:

He was posting HTC Vive Tilt Brush content.
He has the same last name as me.

Both VR and the McGregor clan have taken more than their fair share of battles over the 30–500 years, so it is a rare thing to catch them together in the wild

John Sterling McGregor is a student.

John Sterling McGregor is a student learning calculus.

John Sterling McGregor is a student learning calculus and he’s learning it using Tilt Brush in VR with the HTC Vive.

If I were talking to you directly, I’d now turn to you fully and start leaning forward while saying, “No really, he’s doing this, I think this is a big, important thing, very much worth your time and attention. YOU NEED TO LOOK AT THIS. LOOK AT IT!”
( You then would take a step back and glance at the watch you stopped wearing 9 years ago. As one does.)

Night after night he’s been getting together with friends/tutors to learn math while immersed in VR.

Why this has got me so very excited?

It’s because John does not appear to be out to “prove anything” to the world and I just happened across him.

The scientist places technology amongst the fauna for a year, steps back and watches what nibbles on it. -Me, just now.

John isn’t a VR researcher. He isn’t an educator trying to blaze an experimental path or prove a paper. He’s just a student trying to learn something and get through his next exam, just like all the rest of students around the world…

Let me just write that for dramatic effect “…just like the rest of students around the world.” Um… Did it work? Did you feel the drama?

There is a big difference between talking theory about using VR for learning VR actually seeing it applied in the wild by the learner. — me, just now

More impressive, is this doesn’t appear to be just a casual, “hey, let’s try this out.” type of one shot experiment.

No…

John has spent roughly 16 hours (so far) studying calculus in VR.

A lot of virtual reality game developers would be VERY happy to have a user spend that much time in their game.

Something here is clearly working…

Here’s are a few examples of a few sessions, some in excess of 3 hours.[below].

I find it a to be a fascinating first look into what might be the beginning of something that just might redefine a lot of our traditional learning over the next decade. I encourage you to browse through the lot. You won’t find any drama, just lots and lots of very focused mathematics learning.

In VR…

With Tilt Brush…

I feel like I could just stop writing now, that pointing at these videos is enough to make at least a few people out there [hello Google] pause and say “Huh. That’s interesting.” Frankly, I feel study of the videos themselves are a hell of a lot more interesting than anything I might be able to write…

But I’ll press on and try to elaborate a bit.

In case you are reading this and have missed the last few year’s of VR, here are a few building blocks to play with at home:

Here are the tools he is using to make this happen:

HTC Vive VR headset (HTC)
Tilt Brush (Google)
Skype (Microsoft) (to stream his sessions so that other can see what he’s doing.)

While most people reading this will recognize Skype from the above list, if you aren’t familiar with the HTC Vive or Tilt Brush, the below clip does a pretty good job at illustrating (har, har) what it does:

“Put on the VR gear, grab the controllers and Tilt Brush allows you to walk around and paint in a 3D virtual space.”

Now, a very key, clever thing about what John is doing here, is that he is streaming his calculus session in real time with his friends/tutors via Skype. John isn’t floating alone in the void, he has plenty of company. They see everything he is doing and whisper help and advice in his ear any time he needs it.

“Only what you take with you…” — Yoda, not me

When John puts on his headset and earphones, he finds himself surrounded by the inky blackness of the VR / Tilt Brush environment. It is as close to a pure, 100% distraction free environment as I would ever wish on a person. It is a place where John is free to noodle on equations and notes anywhere he likes in the space around him. It’s like taking notes in class and then having them rise up and swallow your mind whole.

calculus — Image stolen from the depths of Pinterest. I don’t know who to credit but “Hey you, nice notes! I mean, wow. Look at ‘em!”

I struggle a bit to find the proper words to describe how conducive to learning this kind of pure environment might be to the right person.

They see everything he is doing and whisper help and advice in his ear any time he needs it.

You have all the advantages of discussion, collaboration, gesturing, active body movement (the list goes on…) and none of the distraction that comes with the real world. A unfettered exploration of concept.

Now, I have to stress that, as forward thinking as the designers are, I’m fairly certain that the Tilt Brush team didn’t set out to build tool for learning math… and yet here we are.

Tilt Brush does contain a surprising number of the essential tools needed…

For instance, a user can bring in any image they like into the environment. Math notes/assignments/reference materials can be added to the space:

And the straight edge / mirroring tools allow you to handily sketch out graphs:

You can bring all the reference materials you need and place them exactly where you need them in this space.

You end up surrounding yourself with the subject you are studying and your own thoughts. Nothing else exists beyond the subject at hand.

_ _ _

I’m going to stop right here and throw out the idea that maybe, in the long run, Tilt Brush isn’t in the business of providing tools for “painting” but rather in the business of giving VR users a place and the tools to express ideas. This seems about right for the level of scope that Google is in the habit of taking on.

“Um, yes, what did you think this was? Did you think we were simply gunning for MSPaint?”- Google

OK. So let’s skip forward back to last night and Google’s preview of the upcoming Tilt Brush features. Here are the biggies:

Multiple Users sharing the same environment.

So, one thing missing from John’s studies above is a way for his tutors to write things down in front of him, or gesture to a particular point in an equation. You can put down your hand if your were planning on pointing that out.

Google will soon be rolling out the ability for users to join together and do just this. They could all waltz around the equations together, working out calculations together in real time from while happily residing in different countries.

Google uses an volcano in their multi user example which is neat… but I feel we are rapidly leaping past the “let’s build a sand castle together” stage.

2. Painting on models and then… manipulating both model and paint.

I’m honestly still trying to wrap my mind around this addition. I’m not going to do a good job of articulating how impressive this is. I won’t even try. I’m not trying.

This seemingly opens the door to a time where you could add annotations to a 3D model in space and have the individual parts of that model respect those notes / designs as they are re-positioned in space.

I’m pretty sure the Tilt Brush team might step in here and say “Wait, no, just the wood mannequin.. um, that’s… well, wait, what you are saying here…. that’s a lot of work.”

But, too late, the potential for it has been shown and I fully believe we will will it into existence. (+10 points awarded for using “will will” in a sentence.

Imagine the kinds of notes and diagrams you might add to a book or paper [ above ] and now imagine being able to do the same thing to a 3D model of a architectural model of the house you are building, parts of an engine design or battlefield map.

3. Animation

Animation is coming. They are hinting at it here and I feel it is only a matter of time before we see further support.

You can then imagine working with animated source materials / models, video references used for rotoscoping in 3D (and by 3D I mean… you are walking around the thing you are working on), animated scene lighting, color. The mind goes a bit nuts contemplating what could be done. All of this would be collaborative and worked on in a single shared space and subject to the annotation and communication made possible in the “Painting on models” section above.

So, enough chatter, time to wrap this up.

Summary:

I think there’s an inkling of something here that represents something important not only for VR but for our education system as well. Something that has tendrils that might extend into core business and design. Something tremendously important.
Keep your eyes closely on the VR tools Google is making but keep an even closer eye on how users are applying those tools out in the field.
We are still at the starting line for all of this…

If you found one of the many typos that most surely exists above, give me a shout on Twitter @ID_R_McGregor. This post originally appeared on Medium and is reprinted here with permission from the author.