Ask New York University researcher Ken Perlin why he is interested in virtual reality and he says that he isn’t. That’s a remarkable answer from a man about to show the world one of the most impressive demonstrations of the technology to date at SIGGRAPH in Los Angeles on Monday, August 10.
Beginning at 8:30 a.m. and running until 5 p.m. four people at a time will wear untethered Gear VR headsets that can be tracked in a 32 foot by 34 foot space. Each of their arms and legs will be tracked too. Inside an alternate reality, not unlike a cartoon storybook, participants will be able to see avatars of one another represented as a kind of stick figure. Unencumbered by the tangle of wires usually associated with VR, group after group will be armed with magic wands allowing them to paint or sculpt an evolving work of art in the shared space. Everything will be recorded.
“We believe that this will be an historical first,” Perlin said. “The first community-created collaborative 4D space-time sculpture.”
While “4D” is the kind of marketing jargon an amusement park might slap onto a new ride, here it is used to concisely describe the “Holojam” demonstration. It was put together in the NYU Media Research Lab by nine people, including Perlin, and will allow participants to see a “time slice of the emerging space-time sculpture.”
“Participants will collectively be creating a four dimensional (three dimensions over time) sculpture, as colorful line drawings in space,” says a description of the project Perlin shared. “After a few minutes, earlier portions of the sculpture appear to fade away, as those portions recede into the past.”
If you fast forward human history a couple hundred years what Perlin built with the NYU students may represent an important milestone that could be revisited by untold generations. In the same way you can see a photograph of Abraham Lincoln from the 1800s or, in the 1900s, a movie of soldiers running across the field of battle to their doom, future generations can re-live a day from the 2000s when humanity began learning how to capture their nuanced experience.
“My interest is not really in VR. It’s in the future of reality, or rather, what reality will become after everybody is ‘wearing’,” said Perlin. “That is, when all children grow up in a world where everyone either has cyber-contact lenses or cyber-lens implants.”
Echoing sentiments shared by researchers like Mark Bolas, who runs a lab at USC that helped give rise to the Oculus Rift, Perlin has an immeasurably deeper view than simple video games for how technologies like “Holojam” will change humanity. To them, the shift taking place is no less significant than the advent of speech or the written word.
“But my real interest is not in all of that for its own sake, but rather in the impact that these things will have for the future evolution of natural language,” said Perlin. “When children grow up in a world in which they can literally draw in the air, and can show each other visual ideas with a wave of their hand, then those children will spontaneously evolve natural language itself to incorporate these new and greater powers of expression. After all, natural language is really the great superpower of our species. All of our other powers are outgrowths of that one. So evolving natural language to becoming even more powerful is perhaps the most important thing we can do.”
If you need to re-read that quotation again to fully process it, please do. I re-read it several times when he sent the words to me in an e-mail.
“But such an evolution in language is not something that we can invent,” he continued. “As evolutionary linguists have discovered, natural language can only be evolved by children, through their use of it when communicating with other children. What we can do — and what our group at NYU is starting to do through the technology and content experiments that we are working on now — is start to create the conditions within which such an evolution can occur.”
The people who share the merged space will be empowered to run around, talk, laugh, look at each other and work together to create their evolving sculpture. Those outside can observe this behavior as the participants interact with one another in the real world, or use a monitor to look into the space the participants are seeing. You at home, or you in the far-off future, could one day see this first-of-its-kind interaction play out around you.
Perlin was inspired to create the experience the first time he tried Gear VR in September last year.
“My very first thought, upon seeing the high quality, low latency and impressive rotation tracking, was that I must purchase a Motion Capture system for our lab, because now we could finally prototype a truly social virtual reality experience,” he said. “That is, one with no latency or other distracting artifacts, and no need for people to be attached to a wire or even a backpack.”
The virtual avatars are built using strap-on wrist and ankle markers tracked by expensive OptiTrack cameras positioned around the space. (I, for one, desperately want to see a similar solution built using the less expensive Lighthouse tracking technology invented by Valve.)
“Our long term research goals are to understand, experientially, the sort of future that Vernor Vinge wrote about in his speculative fiction novel ‘Rainbows End‘ — when everybody will have cyber-enhanced vision, and walking around with the power of VR will be just an ordinary part of our daily lives.”
The demonstration will be in the VR Village area of SIGGRAPH. “Exhibits Plus” tickets are available for $75
