During a visit to SIGGRAPH 2016, I was surprised to encounter one of my all-time favorite PlayStation 2 games. Rez is a synesthesia-themed on-rails shooter that was released in 2001 at the height of the collision of videogaming and electronic dance music culture. It combined abstract, post-cyberpunk visuals with a pounding soundtrack, and was unique in that all player actions were snapped to the beat, making your gameplay part of the music.
At the time, in Japan at least, you could buy an official ‘trance vibrator’ peripheral to put on your chair while you played. Given the size of the device, and the fact that both your hands would be occupied by the controller, most people joked it was some kind of bedroom toy designed to keep your partner entertained while you flew headfirst through abstract glowing neon environments, tagging mysterious geometric shapes for destruction, and releasing missiles perfectly in time to the zeitgeistey soundtrack.
Fifteen years on, Rez evolved into Rez Infinite for PSVR, and the trance vibrator evolved into a full body ‘synesthesia suit’. This time around, there’s no possibility to mistake the device’s intent. The suit covers the entire body, and contains 24 voice-coil actuators, all of which kick like a mule. It took a team of three volunteers to strap me in to it – by the time they were finished, I looked like an extra for TRON, albeit with a lot more Velcro.
Once suited up, the PSVR and headphones were slipped over my head, I was handed a DualShock 4 controller, and I was inside the game.
Full body haptic feedback
Rez is an interesting prospect for a haptic suit. It’s reasonably old-school in that you have a limited number of lives (‘evolutions’), and a single hit from a projectile will devolve you. Because of this, projectiles are slow, and easily destructible. Essentially, if you’re good at the game, you don’t get hit very often. So the developers of the suit (including the original producer, Tetsuya Mizuguchi) have concentrated on producing haptics for outgoing events: locking on targets, releasing missiles, and progressing from stage to stage.
I couldn’t tell you exactly how they’ve mapped the events, except that it tangibly, physically connected me to the action of locking on and releasing missiles in a remarkably intense way. The actuators aren’t just kicking out simple impulses – they’re playing back a range of different modulated real world sounds as vibrations, so the sensations range from kicks and fizzes to tingles. Alongside the game events, the beat of the music pulses through the suit, giving you the impression you’re standing next to the ultimate subwoofer. Everything combined to enhance my feelings of presence and power, my connection to both the virtual world and the music, in a way that was far more complete than any connection I’ve experienced with a vibrating handheld controller.
My favorite moment came as I progressed from one stage to the next: as my avatar performed a flying pirouette and the world shifted and transformed, waves of vibrations rippled up and down the suit, perfectly complementing the psychedelic transformation visuals.
One minor disappointment with the experience was that I didn’t really experience any directional haptics. Rez has an excuse for this: projectiles are huge with respect to your avatar, so it would make no sense to experience being hit in the lower leg. But what about other titles? We don’t really know how the first person shooter is going to evolve in VR — would a suit like this be able to go beyond HUD-based directional damage indicators, and provide physical, locational damage cues?
Right now, the answer is no. Most VR systems track three points — your head and each of your hands. This information isn’t enough to reconstitute the position of your lower and upper arms and elbows — hence most hand-tracked VR experiences only show a representation of your hands. And there’s no information regarding your legs. Sure, you could strap a number of tracked controllers to the suit and roll your own ad-hoc solution, as many developers are doing for weird and wonderful HTC Vive experiences. But there are already several suit-based tracking solutions out there, such as the Perception Neuron, that integrate penny-sized Inertial Measurement Units (IMUs) into a suit, and use the IMU data to pose a full body skeleton.
So, if you’re prepared to don a full haptic suit to enter The Matrix, it would seem to make sense to combine that with some IMU-based technology to provide both full-body tracking and locational haptics, for the ultimate in online paintball — without the sting. Just remember to invite three friends around to strap you in to it.
Further details on the suit can be found here.
Matthew Brooks is a senior engineer working in user experience research and development for the BBC. He is particularly interested in the creative application of technology to storytelling, with 15 years of experience in the videogames industry.
