Meta shared details of its long term research towards haptic gloves with force feedback for VR and AR.
Most VR headsets today come with a pair of tracked hand controllers with basic vibration. Some headsets like Meta’s Quest 2 also support controller-free hand tracking, but the lack of any form of haptic feedback limits its usefulness. Many VR glove startups and crowdfunding campaigns have emerged over the years, but most only provide per-finger vibration. Gloves which actually restrict your fingers in response to a virtual object – called force feedback – currently start at $5000, and that’s actually significantly lower than previous devices.
Meta says its goal is to develop “soft, lightweight, stylish, comfortable, affordable, durable, and customizable” gloves capable of simulating “a range of complex, nuanced sensations for the wearer such as pressure, texture, and vibration to create the effect of feeling a virtual object with your hands”.
The company says this goal requires “groundbreaking developments across many scientific and engineering disciplines” from its Reality Labs research team:
Microfluidic Actuators
To simulate the texture of virtual objects, gloves today use an array of tiny vibration motors. Like with pixels on screens, more motors means a higher fidelity recreation of the feel of virtual objects. Meta says traditional mechanical motors are “too big, expensive, and power-hungry to render realistic haptic sensations”, so four years ago it tasked its researchers to start work on soft, pliable actuators made of “entirely new materials”.
The researchers looked to emerging technologies like the soft materials on prosthetic limbs and microfluidics used in cutting edge medical diagnostics devices. Meta claims they have made breakthroughs in pneumatic actuators using air to create force feedback, and electroactive actuators which change shape or size in an electric field.
“What makes our work different from the broader field of microfluidics in general is that we have this emphasis on making things very lightweight, wearable, and fast”
To control these novel actuators, Meta claims it’s building the world’s first high speed microfluidic processor – “a tiny microfluidic chip on the glove that controls the air flow that moves the actuators, by telling the valves when and how far to open and close” with sub-second latency.
Haptic Rendering
To know which actuators to activate at a given time, highly accurate hand tracking is needed, but so is a system to interpret the shape, texture, weight, and stiffness of the virtual objects you’re currently touching.
The researchers are working on “haptic rendering” algorithms which interface with physics engines (present in game engines) to send the right instructions to the gloves. This entails “modelling the physics of touch” while accommodating the limitations of the hardware and the inability to prevent the user from passing their hands through virtual walls and tables. This, Meta says, requires an advanced understanding of multisensory integration – the science of how your brain merges input from multiple senses to understand the world.
Meta claims VR experiences can exploit multisensory integration with precise timing to trick your brain into experiencing the sensation of holding weight, by “gently pulling on the skin of the wearer’s fingers with the actuators to mimic the tug of gravity” alongside visual and auditory cues.
“Our brains are really good at taking a little bit of haptic signal, a little bit of visual signal, a little bit of auditory signal, and fusing it all together to really feel the sensation and be convinced that there’s an object that exists in your hand”
In future, the researchers envision tools to create haptic materials just as visual and audio content are created for games today.
Smart Textiles & New Manufacturing
Meta wants haptic gloves to be lightweight, soft, and highly durable – not rigid heavy and uncomfortable. Its materials scientists worked on new comfortable and flexible materials “customized at the molecular level to yield new functionalities” – a field called smart textiles.
The researchers are exploring “how to build multiple functions — including conductive, capacitive, and sensing functions — into the same fiber or fabric and enable a much slimmer, more wearable form factor”.
The prototype gloves are currently made using a semi-automated process requiring “skilled engineers and technicians who manufacture the subsystems and assemble the gloves largely by hand”. Producing gloves like this at scale though will require entirely new automated manufacturing techniques, Meta cautions. And ideally, for “maximum haptic precision and comfort”, gloves should be custom manufactured for the buyer’s specific hand shape and size.
Decade-Scale Horizon
Meta describes this research as a “moonshot” currently “casting a vision for what our digital worlds will look like in 10 to 15 years”.
“When we started the haptic glove project, we asked ourselves whether we could build a mass-producible, affordable consumer device that lets people experience any tangible interface anywhere.
We couldn’t do it — not without inventing new materials, new sensors and actuators, new methods of integration and systems, new rendering algorithms, new physics engines, the list goes on. It just wasn’t possible, but we’ve forged a path that is plausible and could allow us to get there.”
In other words, don’t expect this in a product any time soon.
As with most of Meta’s research showcases this is likely intended to attract world class researchers to the Reality Labs team, with the aim of recruiting the people capable of making breakthroughs necessary to one day deliver technologies like these to consumers.
