Multiple patents awarded to Facebook this year suggest the company is researching a range of technologies which could enable force feedback gloves for VR.
Gloves may be the ultimate goal for VR input. The term “haptic gloves” usually refers to finger tracking gloves with vibration motors on each finger. Force feedback gloves, though, go further by restricting the movement of fingers in response to a simulated object or surface.
Facebook previously showed research involving a haptic glove, but not force feedback gloves. These three patents, however, provide insight into some of the company’s research behind the scenes.
Note these are not applications, but actual awarded patents.
Microfluidics
In March of this year, Facebook was awarded a patent titled Switchable fluidic device.
The patent describes a glove with “soft materials that use millimeter or smaller channels filled with fluid“. By controlling the flow of fluid through these tiny channels, the system adapts the pressure it applies to the finger joints.
When the pressure is high, it “prevents or enables a physical movement of a portion of a user in contact with the virtual object in the virtual space“, according to the patent description. “For example, if a user’s finger is in contact with a virtual object (e.g., a virtual wall) in a virtual space, the haptic assembly prevents a physical movement of the user finger to move in a direction through the virtual object in the virtual space. Accordingly, the user can receive a perception of contacting the virtual object.”
One “embodiment” of the gloves is described as covered with infrared LEDs for positional tracking from cameras, just like an Oculus Touch controller.
Hard/Soft Touch Simulation
In June, Facebook was awarded another patent related to haptic gloves, titled Haptic devices that simulate rigidity of virtual objects.
Almost all gloves to date use vibration motors to mimic the feeling of touching virtual objects, however, this is not particularly realistic. The glove in the patent instead uses an array of plates which dynamically actuate to touch the user’s finger with a force simulating the object the user’s finger is touching.
When pressing against a hard virtual object such as a button, the plates would resist the pressure of the finger. When pressing a less rigid virtual object the plates can “give” much more easily to pressure, providing the feeling of softness.
Pneumatic Bladder
Last week, Facebook was awarded yet another patent. This time the idea covers an alternative to microfluidics for force feedback: Pneumatically controlled haptic mechanisms with nested internal structures for haptic feedback.
The glove is covered in an array of stacked “pods” which each contain a pneumatic “bladder” made of “a durable, puncture resistance material, such as thermoplastic polyurethane (TPU)”. By adding and removing air from each bladder, the pressure against the user’s hand at that position can be altered: “Depending on a posture of the user’s finger when the pressure inside the bladder is increased, the user may experience his or her finger becoming stiff and rigid, bending downwards, or bending upwards (e.g., pushing and pulling sensations).”
This pneumatic technology, because of the higher pressure it can generate, could simulate more intense events, even pushing back on the fingers rather than simply restricting them. The patent describes how this could be used in VR games:
“just prior to releasing an arrow from a bow in real life, a tremendous force is applied to the pads of the fingers drawing the bow string. Therefore, in virtual reality, the haptic stimulation created by the wearable device would need to be intense to provide some realism to the virtual reality experience (e.g., one or more pods on each string-contacting finger push against the string-contacting fingers and attempt to straighten these fingers, as would the bow string in real life)”
While such a system may not be reliable for hours of usage per week, the patent describes designing with this in mind: “Due to the ever-changing nature of virtual and augmented reality, the pods may be required to transition between the two states hundreds, or perhaps thousands of times, during a single use. Thus, the pods described herein are durable and designed to quickly transition from state to state.”
How Far Away Is All This?
At Oculus Connect 3 in 2016, Oculus Chief Scientist Michael Abrash made a series of predictions about the future of VR technology. Among these was the claim that VR controllers similar to today’s would be the state of the art “for at least 5 years”, and “maybe” much longer. Abrash postulated that controllers could potentially be “the mouse of VR”, and remain so even “40 years from now”. He followed this by saying:
“The only thing I can see replacing Touch-like controllers is the ability to use your hands as direct physical manipulators as you do in the real world, and I don’t see that happening in the next 5 years because it requires haptic and kinematic technology that isn’t even on the distant horizon.”
At Oculus Connect 5 in 2018, however, when revisiting these predictions, Abrash changed his outlook:
“I still don’t think it’ll happen in the next 4 years, but something interesting may in fact be on the distant horizon. […] I believe we’ll have useful haptic hands in some form within 10 years”
It is possible research related to these patented ideas — or perhaps some similar research underway at Facebook — may have informed Abrash’s updated guidance. Of course, organizations of Facebook’s size file patents all the time on lots of ideas that never see actual use.
Other force feedback gloves exist already such as HaptX and Dexmo, but these are aimed at the enterprise market with five figure price tags. Given Facebook’s focus on the consumer market and affordability, it’s likely the company is looking for a solution that can be manufactured affordably at scale.
Facebook Reality Lab, the company’s VR/AR research division led by Abrash, has been on a hiring spree for years now. In Facebook earning calls, the company mentioned increasing investment in VR/AR research.
