At NVIDIA, we are closing the "embodiment gap." We don’t just build robots; we build digital and physical nervous systems that allow humans to teach robots. You will lead the development of DexUMI (Dexterous Universal Manipulation Interface), a framework that leverages human-worn hardware and advanced computer vision to transfer complex skills from human hands to robotic actuators.
This is a True Full-Stack role in Solutions Architecture Team: you will touch everything from the tactile sensor firmware on a wearable exoskeleton to the cloud-based data pipelines that train our diffusion policies.
What you'll be doing:
- Hardware-Software Co-Design: Maintain and iterate on the DexUMI wearable exoskeleton. You will bridge the kinematics gap between human hands and diverse robot end-effectors (e.g., XHand, Inspire Hand).
- Sensor Fusion & Integration: Integrate high-fidelity tactile sensors and IMUs into wearable interfaces. Ensure low-latency data streaming.
- Vision & Perception Pipelines: Implement and optimize the "software adaptation" layer—using tools to segment human operators out of training data and robot embodiments.
- Data Engineering for AI: Build robust pipelines to collect, clean, and replay dexterous manipulation data for Imitation Learning and Diffusion Policies.
- Optimization: Solve bi-level optimization problems to parameterize exoskeleton designs that maximize human wearability while preserving robot-equivalent fingertip workspaces.
What we need to see:
- A MS/PhD in Robotics, Machine Learning, Computer Science, Electrical Engineering, Mechanical Engineering, or a related field (or equivalent experience) with at least 1 years of research and engineering experience.
- The "Body" (Hardware/Embedded): Proficiency in C/C++ for embedded systems and ROS2, with hands-on experience in tactile sensing, force-feedback (haptics), and motor control.
- Mechatronics & Prototyping: Experience with CAD (SolidWorks/Fusion360) and rapid prototyping, including 3D printing and PCB design.
- The "Brain" (Software/AI): Expertise in Python and deep learning frameworks (PyTorch), with familiarity in computer vision (CV) models.
- Advanced AI Techniques (Plus): Understanding of Imitation Learning or Reinforcement Learning (RL) is a strong plus.
- The "Bridge" (Integration): Experience with Record3D or iPhone-based spatial tracking, enabling integration between perception and physical systems.
- Systems & Infrastructure: Experience working in Ubuntu/Linux environments with high-performance data serialization tools such as Protobuf and MQTT.