Research Scientist, Hand Tracking & ManipulationLocation: New York, Toronto or Vancouver (on-site)
Type: Full-time · Openings: 1
Salary: US$170K – $210K + equity
The role
Architect and train proprietary foundation models from scratch for 3D hand tracking and articulated pose estimation. This role targets a critical bottleneck in embodied AI: dexterous manipulation and hand-object interaction. You'll build in-house equivalents to leading 3D hand and mesh recovery architectures, plus robust interaction models for heavily occluded egocentric manipulation. You'll train on a large, continuous stream of proprietary ground-truth manipulation data.
What you'll do
- Design, implement and train state-of-the-art networks for 3D hand pose estimation, dense mesh recovery and kinematic tracking
- Scale multi-view and temporal architectures across multi-GPU clusters on massive datasets of hands in action
- Develop novel loss functions that enforce biomechanical constraints, temporal smoothness and physical plausibility
- Build models robust to motion blur, severe self-occlusion and rapid rotation in first-person manipulation
- Track objects through complex grasps, segment tools from hands, and map contact points and forces for downstream robotics models
- Rapidly prototype models for tactile-visual fusion, fine-grained action segmentation and new hardware sensors
- Feed tracking outputs into pipelines that reason about contact surfaces and object affordances, bridging human video and robot control policies
What you bring
- Deep expertise in deep learning, 3D computer vision and articulated tracking / hand pose estimation
- Proven experience training large-scale vision models from scratch, beyond inference or fine-tuning existing checkpoints
- Strong grasp of parametric hand models (e.g., MANO), inverse kinematics and dense mesh estimation
- Mastery of PyTorch and deep learning scaling frameworks
- Experience curating multi-terabyte image and video training datasets
- Comfort in a fast-paced environment where priorities shift with new research or hardware
Strong signals
- First-author publications at CVPR, ICCV, ECCV or NeurIPS on 3D hand tracking, hand-object interaction, dexterous manipulation or human mesh recovery
- Work with egocentric manipulation datasets (e.g., Ego4D, Ego-Exo4D, DexYCB, EPIC-KITCHENS)
- Custom CUDA kernels for 3D operations, differentiable mesh rendering or collision/contact computation
Please note: Research Scientist roles require highly specific expertise. Apply to only one research role; you may also apply to non-research roles.
