Oculus Video Shows Advanced Hand Tracking Gloves

by Jamie Feltham • July 25th, 2017

Update: This story has been updated with additional information from a recent Oculus blog post.

Earlier this year Facebook CEO Mark Zuckerberg teased a new form of input being developed over at Oculus: hand tracking gloves. We only saw them through a picture at first, but now a video has surfaced online showing them in action.

An Oculus video posted last week, and featured in a recent blog post from Michael Abrash, shows what looks like the same gloves in action. It reveals a marker-based solution that uses external tracking cameras. “Unfortunately, hands have about 25 degrees of freedom and lots of self-occlusion,” Abrash said. “Right now, retroreflector-covered gloves and lots of cameras are needed to get to this level of tracking quality.” Perhaps one day the company might be able to fit all of that into sensors like those used for the Rift and Touch, but right now it requires that elaborate rig constructed around Zuckerberg in the picture he teased. Not very consumer-friendly.

Given the timing of the video, we wonder if Oculus will be showcasing these gloves at SIGGRAPH when it gets underway next week.

You shouldn’t expect an actual product to come out of this anytime soon; Oculus won’t be releasing any new hardware in 2017 and the Touch controllers are still the primary form of input for its VR experiences having only released late last year. It’s more than likely that this is still an R&D project that’s not even confirmed for a consumer release. We’d love to know if these gloves have any kind of haptic feedback to react to actions like pressing buttons.

Accurate finger tracking is another important step towards fully VR immersion, but actually replicating the feel and resistance of surfaces and objects in VR is another challenge entirely.

It’s exciting to think, though, about what might be possible with a combination of glove and Touch controller, giving us unprecedented hand-presence in the virtual world. That said the finger tracking on Valve’s new Knuckles controllers for SteamVR offers a similar sort of experience already.

What's your reaction?
Like
Wow
0%
LOL
0%
Dislike
0%
  • Adrian Meredith

    We’d love to know if these gloves have any kind of haptic feedback to react to actions like pressing buttons.

    We’d love to know if these gloves have any kind of haptic feedback to react to actions like pressing buttons.

    errrrrr…

    • braylien

      well, we would love to know if these gloves have any kind of haptic feedback to react to actions like pressing buttons 🙂

  • Alorwin

    These gloves, and all VR gloves, are worthless without restrictive grasping.

    • TDUBS

      restrictive gripping is too bulky anyway so idk. There’s got to be another way that doesn’t involve strapping some huge skeleton on the outside of your hands.

  • Joan Villora Jofré

    These gloves are using optitrack tecnology, too expensive. Those little balls need very expensive cameras.

    • Robbie Cartwright

      Um….. they said that….

      • Joan Villora Jofré

        What’s the point in this article, then? And why the title?

        • TDUBS

          The point Is that they released a video, and its kinda cool.

          • Joan Villora Jofré

            But you are aware that it is only a normal glove, with no circuitry, with reflective balls glued, right?

          • NooYawker

            Nope, I didn’t realize that. but it’s genius on Facebooks part. good excuse to use get high resolution cameras into your home.

  • Justos

    We’d love to know if these gloves have any kind of haptic feedback to react to actions like pressing buttons.

    • Robbie Cartwright

      We’d love to know if these gloves have any kind of haptic feedback to react to actions like presing buttons.

  • CarlosTSG

    Seriously Oculus? Gloves? What happen with your purchase of Nimble VR with the Nimble Sense Camera? Ha yeah, you’d absorbed them into a black hole like every other Silicon Valley startup. I would to have preferred that instead of the Touch controllers.

    • SomeGuyorAnother

      This could potentially be based off Nimble VR’s tech, just using markers on the gloves to increase precision.

      • cartweet

        It is based of that. If you look at the picture with zuckerberg using them you’ll see the previous head of Nimble VR standing beside him 🙂

  • Ombra Alberto

    Fantastic. Go Oculus.

  • NooYawker

    If there’s no haptic feedback it’s pointless. Leap Motion doesn’t need gloves.

    • Eric Nevala

      It’s all about that FOV, baby! If the gloves can be tracked behind your back, this is way ahead.

      • NooYawker

        That’s interesting… behind the back. Pulling a gun tucked in your belt behind you for games like GTA…. and… bondage games??? I can’t think of that many reasons to reach behind you, but I lack imagination I guess. but yea, that is interesting to see what can be done.

        • Flikr

          You definitely don’t need to have your hands behind you, as long as you never turn your neck. Take The Climb, for example. You’re holding on to a ledge, reach out towards another one close to your whole wingspan away, and look at that second ledge. With leap motion, the hand on the first ledge wigs out the moment you turn your head.

        • Caven

          Keep in mind that behind the back and behind the headset are two different things. As an example, let’s say you’re standing still in VR. You’re holding an object at your side with your right hand. Maybe it’s a sword, maybe it’s a duffel bag, maybe it’s a gun, maybe it’s a briefcase. It could be just about anything. You hear a sound to the left, so you turn to look. In VR, the object would technically still be at your side because your hand and body are still in the same positions. However, you’ve turned your head to the left, so the object is now behind the headset. A similar situation might involve aiming a weapon at a door in the expectation that you’ll be shooting at emerging enemies, but you glance over your shoulder to avoid being flanked.

          This sort of problem can happen any time your head is moving around. Once the Leap Motion can’t see your hands, how can it tell if you’re holding something? In Superhot VR, I can pick up an object even if I’m not looking at it, and I can also do things like shoot a gun from the hip. I wouldn’t be able to do either of those with something like Leap Motion, since it has no idea what my hands are doing when out of view of the sensor. I wouldn’t be able to pull a lever or trigger a detonator without looking at it, which could be a problem if I’m timing an action based on approaching enemies. In Windlands, I wouldn’t be able to grapple an object with Leap Motion unless I’m looking at it, a problem I don’t have with normal motion controls.

          And this doesn’t even begin to get into the difficulties Leap Motion has tracking fingers reliably. Leap Motion unfortunately has way too many limitations to be an effective means for reliable hand and finger tracking. I can see value in it for mobile, where controllers aren’t always practical, but in any situation where some form of controller is viable, Leap Motion just isn’t an adequate substitute.

    • care package

      Leap Motion is still flawed, mostly because it still needs direct view of a camera.

      • New hardware has an enormous FOV (180* 180)

        • care package

          Camera tracking and FOV are two different things.

  • Robert1592

    Haptic feedback in gloves have been the holy grail of VR for decades. Yet we don’t see them available today which tells you a lot about the limitations of trying to build mechanical structures around the fingers. Even just trying to track the fingers has proven very difficult. Oculus are running into issues and its probably dawning on them now that they’ll never be able to prevent occlusion and other issues with this approach.

  • Cool research project… but I don’t see gloves as the primary input for VR… apart from some enterprise applications

  • bobzdar

    Simple, mount a leap motion to each wrist for hand tracking and keep the touch controllers for object manipulation. Then you have both: accurate hand tracking (without occlusion) and a physical object with haptic feedback to manipulate.

  • Mk.82

    This is the reason why the Oculus decision was better as you don’t need complex hardware (CPU, RAM, Wireless connectors etc) to the tracking parts at all as all you need is just a IR LED and then program the pattern to Oculus software so it knows what direction it is pointing. Basically all could do is buy a plastic toy weapon, drill a few holes to it and insert IR LED’s in to it and add a AA battery for power. And then if Oculus would allow program one by pointing it and then calibrating the X,Y, Z axis rotations it would be usable in any VR game as a weapon.

    • Caven

      It’s really not that simple. For starters, my understanding is that the Rift pulses its LEDs in unique patterns so that it can positively identify individual LEDs, even without being able to see the neighboring LEDs. Not only does this allow Constellation to positively determine that it is looking at a particular point on a device, it can also positively identify which device the LED belongs to, which is important when waving around multiple devices, such as the headset itself and two touch controllers.

      Beyond that, despite the use of Constellation, the Rift and Touch controllers also have IMUs in them. Like with Vive’s Lighthouse system, Constellation by itself isn’t really up to the task of rock-solid positioning. Because of this, the Rift and Touch use IMUs for tracking, with the IR cameras being used for drift correction. There would be no need for IMUs if tracking IR points by itself were good enough.

      So ultimately, your toy weapon would still need some way of uniquely pulsing the LEDs, as well as an IMU to ensure precise, high-refresh tracking, and a transmitter for sending IMU data to the computer. And that’s all assuming something completely inert like a sword that has no need for buttons and switches. Make it an object with actual controls on it like a gun, and now you have even more data to send back to the computer.

      Sure, it would be possible to hack in support for inert objects with LEDs that don’t pulse, but you lose the tracking benefits of the IMU, and with no way for the object to identify itself, multiple identical devices could get mixed up. As one example, suppose you’re using two swords, and one is much more powerful than the other. You try to switch them between your hands so that the more powerful one is in your dominant hand, or you do something else that temporarily obscures the view of one or the other sword. Once they’re both visible again, how will Constellation tell which one is which? You could probably work around that by giving each sword unique LED positions, but that would be a big problem for mass manufacture. At that point it seems like it would make more sense for Oculus to go the route of HTC and make a tracker that can be attached to other items. It wouldn’t be as cheap as people would want them to be, but the trackers would be far more accurate and reliable.

  • Pretty cool idea, definitely a step in the right direction! Would love to see if it actually improves anything during gameplay.