Meta today announced “significant improvements” to the Quest 2’s controllerless hand tracking. The “redesigned approach to computer vision and machine learning” is said to specifically improve the reliability of overlapping or fast-moving hands and certain gestures. SDK and OS updates to support these improvements will begin rolling out today.
Meta first brought controllerless hand tracking to the original Quest in late 2019, and it remained an “experimental” feature until mid-2020 when it began allowing developers to use new features in its apps.
Since then, we’ve seen a handful of games incorporate hand tracking into their apps, and even some that rely entirely on hand tracking, such as Hand Physics Lab (2021) and Unplugged: Air Guitar (2021).
Now, less than a year later, Meta says it’s bringing “significant improvements” to the Quest 2’s hand tracking capabilities (the company confirmed that the original Quest won’t receive these improvements).
These improvements are attributed to “redesigned computer vision and machine learning methods” that improve the robustness of hand tracking in key ways.
In version 1.0 of the hand tracking system on the Quest 2, the system had particular difficulty recognizing the user’s hands when they were blocking or touching each other and when they were moving quickly. From the user’s perspective, their virtual hands temporarily disappear during these lost tracking moments, then reappear when the system detects them again.
With version 2.0 of hand tracking on the Quest 2, Meta says the system will better handle those blocked and fast-moving scenarios, resulting in fewer instances of hands disappearing. The company calls it a “step-by-step improvement in tracking continuity.”
The update is also said to improve gesture recognition in the hand tracking system. Gesture recognition looks for a specific gesture that the system detects as unique and therefore can be used as input. For example, pinch is one such gesture used to allow the user to “tap” on elements in the Quest interface.
In the demo below, the “grab” gesture is used to hold a virtual object, and the improvement in applause robustness is also shown.
– – – – –
Meta says that the Quest 2’s Hand Tracking 2.0 update will be rolling out via SDK and OS updates starting today. Developers who already have hand tracking built into their apps won’t need to change any API calls to use the upgraded system, although it won’t be enabled automatically, the company said. The company says developers can refer to “documentation to be released soon” to enable it in their apps.
The move should bring the Quest 2’s hand tracking closer to Ultraleap, which has maintained the best hand tracking in the industry to date, although it’s unclear how the two systems will stack up.
