When the UK-based VR studio, Rewind, asked me if I wanted to come and see a demo of its eye-tracking tech demo for the upcoming FOVE 0 headset, I was intrigued. Not just because it was a device and experience I’d yet to get my head into, but also because Rewind wasn’t a company I’d previously associated with helping to pioneer such tech.
These developers make impressive 360 videos like their Strictly Come Dancing footage, immersive experiences like the enjoyable Home, and artistic experiments like the one it created in partnership with popular singer, Björk. Project Falcon, as this new demo is called, is an on-rails first-person shooter that uses eye-tracking as an aiming system for noisy machine guns and explosive rockets. Its war-torn city wouldn’t be out of place in a Gears of War game (it’s built inside developer Epic’s Unreal Engine 4), and it’s quite unlike anything else the studio has made thus far.
So how did this unexpected partnership come about? “I met the guys from FOVE, Yuka [Kojima, CEO], and Scott [Harper, Creative Director] at South by Southwest 2014 in Austin, where they were demoing a very early prototype of the FOVE [headset],” Head of Special Projects Oliver Kibblewhite tells me. “And we got chatting about a variety of things, including Yuka is a massive Björk fan. At the time we were just finishing the Stonemilker video player, which we built for them and released on the various app stores. So I was able to show her a copy ‘Stonemilker’, which she was super excited about.”
From that meeting a relationship was born and, years later, Rewind agreed to make this showcase, which debuted at Tokyo Game Show (TGS) back in September. It went down “fantastically well”, according to Kibblewhite, and even a certain Oculus Rift inventor saw it and liked it.
Having gone hands-on with it myself, it’s easy to see why: Project Falcon is a simple but enjoyable demo that effectively demonstrates eye-tracking as an input mechanism, if also highlighting some of its weaknesses. At this point it’s important to note I’m not here to assess the FOVE itself; we’ll be doing that later on. I’m playing Project Falcon on battered and beaten developer hardware, but it’s still in good enough shape to let me run through the 10 – 20 minute experience.
As with so many other VR games, everything begins in a cockpit. Over a radio, I hear gritty and desperate military voices informing me or the urgency of the situation. A gate opens and I begin to move forward on a set path, blasting spider-like robots that surround me.
To do this, I’m simply looking at them and pulling triggers on a controller. A circular reticle follows closely behind me eyes, with machine gun fire spouting out from where it’s aiming. A larger, triangular reticle also lags behind a little further, firing rockets at enemies when it finally catches up with my vision (a deliberate mechanic). I found I would have a bit of trouble aiming at enemies off in the distance with my machine gun, as focusing on them could be a challenge, though the area effects of the rockets made short work of them. When enemies were closer, taking care of them was effortless.
If I was struggling with the eye-tracking, I could quickly hold a button and stare at an icon in the center of the screen to recalibrate my vision, which I needed to do a few times throughout. The process was quick and simple enough that it didn’t break the immersion of the demo itself.
As I said earlier, this was my first time using eye-tracking in a VR game and, while I was interested to see how it enhanced the experience, I was just as curious as to what new issues it might present, just as any breakthrough in the VR industry presents. Rewind is happy to offer up some examples of this. “Your eyes and your tracking kind of get into a little bit of a feedback loop,” Kibblewhite explains. “Because we have a targeting reticle that follows your eyes’ position around the screen it can often distract you from the thing that you’re intending to target.”
I see what he means as I’m playing. Eye-tracked aiming requires a surprising amount of concentration, because your eyes will naturally start to drift towards the aiming reticles if you don’t focus. Think about when you’re looking out of the window of a fast-moving train and your eyes constantly fix to different objects as they pass. It’s a similar effect here, though it can be avoided if you keep yourself aware of it at all times.
“That’s something we had to work quite carefully with our devs and the guys from FOVE to overcome,” Kibblewhite explains. “And it’s all about finding a balance between the speed of the marker and the speed of the tracking so your eyes don’t get too distracted.”
VR developers are not strangers to new tech creating new sets of problems, but at least they’re backed up by engines that have pretty much fully integrated headset support by this point. As it’s still early, eye-tracking tech doesn’t have that luxury, and Kibblewhite describes developing eye-tracking in Unreal Engine as “exceptionally complicated.”
“In the early stages of development, plug-ins were very much beta, even alpha,” he says. It’s become much more integrated since, and you can expect Unreal and Unity to place a greater focus on native support as more headsets grow to adopt the tech.
Ultimately, I’m not convinced eye-tracking will be as useful an addition for aiming in VR as much as it will UI interaction, foveated rendering and avatar replication. Kibblewhite is also interested in what it means for NPC interaction, where virtual characters can tell if you’re paying attention to them or not.
Still, it was interesting to see it implemented into what would otherwise be a fairly standard VR experience, but it’s just the tip of the iceberg as to what this vital new tech can do for VR. Project Falcon itself is a great way to showcase that tip, but I’m hoping it’s just the start of what developers like Rewind will do with FOVE when it gets into more people’s hands.
