I’ve used my eyes to interact with a virtual world before, but startup Neurable just enhanced that experience by reading my thoughts too.
At SIGGRAPH this week the Boston-based startup is showing its modified HTC Vive which include EEG (Electroencephalography) sensors along the interior of the headstrap. This is combined with eye-tracking technology from German firm SMI, which may have just been acquired by Apple. The EEG sensors’ comb-like structure dug through my hair to subtly make contact against my scalp where they detected brain activity. It is definitely alarming to hear someone outside VR say my brain is looking good.
What followed was a brief training session where a group of objects floated in front of me — a train, ball and block among them. Each time one of them rotated I was told to focus on that object and think “grab” in my mind. I did so a number of times for several of the objects, all successful.
Afterward there was a test. I was told to just think of the object I wanted. I tried not to stare directly at the object I wanted but five out of five times the correct object was picked as I thought about it. A sixth time the wrong object was selected but it occurred as someone was talking to me and I was distracted. As I refocused, almost immediately the correct object moved toward me.
In the video above you can see each of the objects flash. Neurable CEO and President Ramses Alcaide says they are able to detect these flashes in my brain even though they register subconsciously. He said the eye tracking inside the headset wasn’t active during the training and test portion of the demonstration. It became active during the next portion of the demo meant to show the potential of the system in a game environment. Here’s how Neurable describes it:
Neurable is debuting Awakening, a VR game preview made in partnership with eStudiofuture, at SIGGRAPH 2017 in Los Angeles. Awakening is a futuristic story reminiscent of Stranger Things: you are a child held prisoner in a government science laboratory. You discover that experiments have endowed you with telekinetic powers. You must use those powers to escape your cell, defeat the robotic prison guards, and free yourself from the lab. The game allows you to manipulate objects and battle foes with your mind, and is played entirely without handheld controllers.
According to Neurable, this works using machine learning to interpret “your brain activity in real time to afford virtual powers of telekinesis.” The company offers an SDK so Unity developers can integrate the system into a game.
I was able to select a group of objects on the ground of my holding cell just by thinking about them and then use them to try and escape. I was offered some hints from outside VR to escape the room but the selection with my mind worked to grab the objects I wanted. As I moved into a lab, I looked around at the counter tops and thought about the objects to toss at a robot approaching me. One of them was a keyboard. As I thought the word “grab” it floated toward me. Object after object I tossed at the incoming robots until I progressed through to the end of the level.
“We have two modes. Pure EEG mode, which just determines the object you want and brings it to you directly, and we have a mode that is a hybrid BCI [brain-computer interface] mode, and in that mode we can use the eyes as a type of mouse where you can move your eyes near…the object you want to select,” said Alcaide. “From there your brain tells us which one you clicked on.”
In the video above you can see me sort of covering my face as a kind of surprised reaction each time the system correctly identified which object I wanted. I was frankly in shock — I really didn’t expect it to work as well as it did. Both this brain-computer interface and the earlier eye tracking demo I tried felt like true super powers.
According to Alcaide, Neurable raised around $2 million and has 13 employees.
“I think the future of mixed reality interactions is an ecosystem of solutions that incorporates voice, gesture control, eye tracking and the missing link to that entire puzzle which is brain-computer interfaces…we need some sort of system that prevents the action from happening until the user wants it to happen, and that’s where brain-computer interfaces come in,” said Alcaide. “In my opinion mixed reality cannot become a ubiquitous computing platform like the iPhone, or like the computer, until we have brain-computer interfaces as part of the solution.”
