SMI – the company that blew our minds previously with its eye tracking technology – is now capable of running on mobile VR platforms.
SMI released a video showing a Samsung Gear VR functioning with full eye tracking and frame-rate boosting foveated rendering capabilities.
I talked over the phone with SMI’s Christian Villwock and he was able to shed some light on the what, why and how of this apparent breakthrough.
The first thing Villwock clarifies for me is that eye tracking and foveated rendering are not going to suddenly be a downloadable feature on Gear VR headsets. The video is essentially a proof-of concept. SMI is hoping it will encourage content creators to purchase development kits and manufacturers to take a hard look at eye tracking for future product generations.
“This is a launch and it will be available in about 4 to 6 weeks,” Villwock said. “But at SMI we are targeting more the high-end researcher/developer market with this solution.”
SMI’s eye tracking solution is a dual hardware and software platform that needs to be manually retrofitted into a headset like the Gear VR in order to function. Villwock tells me that right now any person who owns a Gear VR or Oculus DK2 can send them in to be modified for eye tracking by his company. Villwock says this would cost thousands of dollars, however, and is therefore not something the average consumer will be able to use.
Villwock says the goal for SMI is not to create a consumer product and sell it in Best Buy for users to hack into their headsets, but rather to convince the major headset manufacturers to make eye tracking and foveated rendering standard features in whatever systems they produce next.
Villwock says SMI is actively talking with “all” of the big headset companies, and that ultimately the decision to take advantage of this new technology is up to them.
“We have contact with all the big manufacturers,” he said. “We don’t have any control of the timeline that guys like Samsung and Oculus or whatever may be using. What we can do is show the advantages operating this technology and be ready when they are ready.”
While some may be disappointed SMI eye tracking is still a bit out of reach for most VR users, the fact that the technology is looking more viable on a mobile platform is a big step forward for the entire industry.
“We have retrofitted our eye tracking component, which is a camera for each eye into the headset,” Villwock said. “Then everything sort of connects. If you snap in your smartphone it connects to the cameras and the smartphone is running all of the software and doing all of the work.”
When an HMD can follow your eyes it opens the door to huge leaps in frame rate and performance through foveated rendering. It also allows for more immersive social and environmental interactions inside of a virtual space. It is at once one of the most exciting and frustrating capabilities of VR since it will not be available in the first round of headsets. Villwock, however, remains optimistic concerning future adoption.
“I am personally convinced that next generation headsets will have eye tracking,” he said. “Foveated rendering is also a must to have.”
By bringing its system into the world of mobile VR, SMI is sending a clear message to manufacturers that this technology can be pervasive, simple, and powerful.
