Continuous 3D Hand Pose Tracking Using Machine Learning & Monado OpenXR

As part of a project backed by INVEST-AI, Collabora has developed a multi-stage neural network-based solution that accurately locates and tracks hands despite complex background noise and occlusion between hands.

The system estimates 2D and 3D joint locations without any depth information. Collabora is currently working on integrating it into the Monado XR codebase, so it can be used out-of-the-box with different devices.