Virtual reality took over the Game Developers Conference (GDC) this year as companies like Sony, Valve, Oculus, Samsung, AMD, and HTC actively tried to persuade developers to create content for their VR platforms. Practically around every corner was a VR headset; whether it was wired to a PC or were mobile versions. This sparked continuous conversations about the directions that virtual reality is heading during the conference and on session floors. Obviously, head mounted displays (HMDs) like HTC’s Vive VR, Sony’s Morpheus, and Oculus’s Crescent Bay prototype were a hot topic. This primarily put the focus on the HMDs at the event, but this also allowed for peripheral systems to emerge along the edges of GDC. One of the little-known areas of discussion centered around eye-tracking as a Swedish high-tech company named Tobii demoed a gaming device that picked up eye movements through a bar-like piece of hardware. They also had a pair of eye-tracking glasses hidden away underneath piles of merchandise as well.
If you walked along the floor of the South Hall of the Moscone Center in San Francisco, you might have noticed the Tobii booth. They had several PC computers lined up that allowed people to play a variety of games, including Assassin’s Creed Rouge edition. The eye-tracking bar sits in front of a person while they play. However, after additional research into what Tobii was working on, it was discovered that they created these eye-tracking glasses too. After a couple of minutes talking with them, it is clear that their hardware will most likely be added into VR and AR headsets later on.
Tobii’s eye-tracking bar sits on top of players’ computers and has been integrated into Assassin’s Creed Rogue. The technique they are using is called ‘Infinite Screen’ which can pan the camera based on the individual’s gaze. The goal is to sync up the vision of the player with the vision of the protagonist in the game. This eliminates the need to keep one’s hand on the mouse and provides more of an immersive experience to engage people even further. Aiming is easy with a system like this because it includes natural eye movements and brings it into the game.
When I first looked at the glasses, the first thing that popped into my mind was how fashionable they looked. They were less dorky in design compared to other types of glasses that I have seen in the past (especially AR headsets like Google Glass). They resembled glasses that people would wear on a regular basis. Not to mention the device was extremely light; so much so that I almost forgot I was wearing it at times.
For the most part, the glasses are being used for market research and analysis in order to produce ‘heat maps’ depending on what people are looking at. Tobii has teamed up with larger corporations to perform experiments. This is where the majority of the revenue is coming from. French cosmetics and beauty product manufacturer L’Oréal has used Tobii eye-tracking systems in the past. L’Oréal set up a display for market research to see where people were looking based on what items were shown on the screen.
In regards to the hardware, the lenses are nothing special. In fact, they are just like a regular pair of glasses, and prescription lenses can be added if needed. There is an HD camera embedded into the front of the frame. Matt Clyne, Tobii’s Technology Evangelist/Community Manager, stated that the camera is mostly likely 1080p. There is a microphone to record audio as well. Ion-lithium batteries go into a battery pack that hooks to someone’s belt. The setup that I saw was wired, but Tobii is looking into making a wireless version down the line. The circuit has built-in wifi-capabilities to transmit data, however the functionality has not yet been integrated. Reportedly, the belt attachment can stream the information and the battery life lasts approximately 4 hours. Each headset comes with a set of nose clippings that allow people to customize the attachment depending on the shape of their nose. The circuits are all developed internally by Tobii. The only thing they don’t make is the PCI board. They do most of their manufacturing in Taiwan and China which keeps the cost down.
The magic comes from the internal camera that looks towards the user’s eye. As Clyne told us in an interview, they are “the world’s smallest” IR cameras. After the glasses identify where the user is looking, there are a couple different ways to get data out of the processor. As mentioned above, the circuit has built-in capabilities to transmit wifi. The system can also save the information on an SD card for backup processes if needed.
Now, since the virtual reality community has actively been seeking an eye-tracking solution, I asked Clyne what Tobii’s plans were for integrating their hardware into VR and AR headsets. He started off by saying that they are looking for partners to bring eye-tracking to the virtual and augmented reality worlds.
With the recent announcement that OSVR has partnered with Ubisoft, and because Ubisoft is working with Tobii to bring eye-tracking to games like Assassin’s Creed Rogue, I was curious if Tobii was talking directly with OSVR. Clyne stated that “we are talking with a lot of people.” He went on to say that he couldn’t discuss the specifics just yet.
Ubisoft hops on the VR train as OSVR reveals updated HMD
Tobii is absolutely interested in the augmented reality side of the spectrum. From the sounds of it, Tobii is farther along with VR, but they are now approaching AR as demand is starting to increase during the explosion of virtual reality. As Matt Clyne discussed with us, Tobii is “working on both AR and VR”, but they “haven’t really locked down any partners yet for anyone long term right now. [They] are doing some market research with a bunch of people.” Eventually, this could be built into a head-mounted display in a way that doesn’t interfere with looking at a screen. Clyne goes on to say that “we are working with different people to find what we think is the best solution to move forward. For VR, it’s huge.”
In addition to chatting about VR and AR, Clyne talked with us about the benefits of eye-tracking in terms of artificial intelligence (AI). If the system can determine where someone is gazing, avatars can react more intuitively once a person looks their way. This provides more interactivity, but will blur the lines between what people perceive as real because the AI will be able to act more like a real human. In turn, it can determine the direction the eyes are facing adding more immersion to the game making it feel way more real.
Despite the demand for eye-tracking on a consumer level, Tobii is relatively far away from bringing their products to VR and AR for the general public. This is because the cost of their system is currently quite high. There is no information about the current price tag for the glasses, but Clyne hinted that it is around “5 digits.” Indeed, this is way too much for mass adoption. However, this opens up the door for their eye-tracking package to be used in industrial realms including construction, advertising, and even sports. For example, one of Tobii’s previous clients works with large mining trucks that transport materials on long boring roads. Drivers get tired and their focus strays off the path they are taking, which can lead to dangerous drifting to the side. The glasses have the ability to detect if the wearer is getting tired, notifying the user to take a break before they fall asleep. Awareness is critical in the automotive and transportation industries. Having a system like this could significantly reduce unnecessary accidents.
Sports training is another interesting application for eye-tracking. Tobii has previously worked with a basketball coach (whose team will remain nameless) that rented a bunch of the headsets to train his players on where to look when taking shots or passing the ball. Having them focus on specific spots dramatically improved scoring accuracy percentages. This type of system can be brought to other sports like football, soccer, hockey, and more. Matt Clyne was particularly excited about the possibilities of football integration because they could add their eye-tracking system directly into the helmets of the players.
Despite amount of work Tobii is doing in industrial markets, they are trying to go towards a consumer route eventually, although it will take some time to accomplish. Despite this, it sounds like Tobii is not looking to make a huge profit in eye-tracking on a consumer level, but would rather try and saturate the market increasing the overall demand for eye-tracking. They want to get as many people using the devices as they can to see if the functionality will spread to many industries all across the board. Tobii understands that there are a lot of potential revenue streams out there one top of just selling the devices.
Being that Tobii has been around for 14 years, they now have a firm grasp on eye-tracking. Their team consists of approximately 700 people and they have offices all around the globe. They are working actively to bring eye-tracking to VR and AR. This is good for the VR community because it will add more immersion to the experience. Hopefully we get more information about Tobii’s plans for virtual and augmented reality soon. In the meantime, you can learn more about the company on their website.
