As the virtual reality industry explodes into the market, increased experimentation is manifesting additional tracking mechanisms along the way. A Los Angeles-based firm called Reach Bionics, for example, developed a unique controller system called Conjure by embedding electromyography (EMG) sensors into a Gear VR headset to analyze facial muscle movements of the user.
The beauty of the Conjure system is the lightweight nature of it, both in heft and processing power. Because the tiny sensors are sewed directly into the fabric of the headset, they are comfortable enough to remain undetectable while wearing a modified VR headset. The EMG sensors also don’t require much processing power either, according to Michael Rollin, CTO of Reach Bionics, keeping the computational load to a minimum.
Currently, Reach Bionics’ goal with the Conjure controller is to create a platform for virtual reality that limits “taking users out of the immersive experience.” In order for virtual reality technology to become effective in maintaining a sense of presence, it has to track as many sections of the body as possible. The Conjure controller is adding another important part of us to the list of interactive features – our facial muscles.
Conjure works by analyzing electrical signals generated by facial muscle movements, which is different from the RGB-D camera recognition approach we’ve seen before. It’s surprisingly intuitive. When a person wiggles their nose, the data is sent to a computer nearby via Bluetooth from an open-source circuit powered by a 3.7 Volt lithium-ion battery attached on the side of the headset. That information is messaged back allowing the user to select items with their nose. The next iteration will be a standalone attachment, eliminating the need to transmit data over the air.
Although EMG sensors are not being shipped by the leading VR headset manufacturers at this time, EMG-based facial detection does have a chance to make its way into future versions down the line.
The logical progression of the next generation of head mounted displays (HMDs) like the Rift and Vive will obviously include better screens and updated lenses. Subtle changes to form factor will also be present. That’s just how device updates go. Improve what you can, then start experimenting on the side to see what makes sense later on.
Integrating an external camera before adding EMGs is a good bet for PC-based headsets as well. HTC and Valve are already demoing their latest version of the Vive Pre headset with early tests of computer vision features. Not to mention smartphones include front-facing cameras, too. Adding external cameras is a no brainer, as long as the cost of integration remains low. That’s a short-term addition, but what are we going to see in a few years?
Tracking the human body is more important than ever for virtual reality content developers. The more human features available for content creators to interact with, the better the experiences can be. Glimpses of eye-tracking with foveated rendering are already appearing as premium virtual reality headsets begin to ship.
SMI Showcases Foveated Rendering With New 250Hz Eye Tracking Kit
Depicting facial gestures in virtual environments is a great feature too, especially for social interactions, but the immediate need to reduce the processing load of VR experiences if far more important. After that happens we should get things like EMG sensors manufactured in large quantities for VR. Sure, companies like Oculus, HTC and Valve, or more predictably mobile manufactures like Samsung, could add facial detection before eye-tracking due to the low cost of the sensors. But in order to see a giant leap in innovation, foveated rendering needs to come first.
The good news is we will see items like the Conjure controller make their way into VR headsets over time. Wiggling your nose to select options brings out a magic-like quality in VR. Imagine being able to cast spells like a character from Bewitched, or smiling in a social environment and your friends and family can see how happy you are. Since EMG sensors detect the electrical signals of your muscles, features like that are indeed possible. So sit back, relax, and know that the future of virtual reality will include all that – and more.
