A new control framework called ZEST enables humanoid and quadruped robots to perform complex, agile movements by learning from human motion capture, video, and animation data, reducing the need for task-specific engineering
Researchers at the RAI Institute and Boston Dynamics have introduced Zero-shot Embodied Skill Transfer (ZEST), a reinforcement learning framework designed to teach legged robots a broad range of agile, whole-body movements using human motion data. Unlike conventional approaches that require separate training or manual controller tuning for each new skill, ZEST enables robots to acquire diverse movements-including crawling, breakdancing, and backflips-through a single training phase conducted entirely in simulation. The system is intended to reduce the engineering burden associated with deploying new robotic behaviors and to expand the repertoire of actions available to humanoid and quadruped robots.
ZEST is capable of learning from three primary sources: high-fidelity motion-capture recordings, single-camera video footage, and keyframe animation. Human movements are converted into robot-compatible trajectories using kinematic retargeting, while animation data can supply skills that are difficult or unsafe for humans to demonstrate. The framework does not require explicit labeling of ground contact points in the demonstration data, which simplifies the preparation of training material and allows for a wider variety of input sources.
Simulation and Real-World Transfer
The core of ZEST is a feedforward reinforcement learning policy trained in simulation, relying solely on onboard proprioceptive sensors during deployment. The policy is transferred to physical robots without additional fine-tuning, avoiding the need for state estimators, long observation histories, or complex reward engineering. To address the challenge of sim-to-real transfer, the researchers incorporated simplified models of closed-chain actuators and accounted for actuator effects such as power limits, motor saturation, and friction. This approach aims to preserve essential dynamics while reducing simulation complexity.
During training, ZEST uses adaptive sampling to focus learning on the most challenging segments of each motion trajectory. The system tracks failure rates for fixed-duration segments and increases sampling frequency for those with higher failure rates, while maintaining a minimum sampling rate for easier segments to prevent skill loss. An automatic curriculum is implemented using a virtual assistive force, which is gradually reduced as the policy improves, allowing the robot to attempt more difficult movements with less external support.
Evaluation and Limitations
ZEST was evaluated on three commercial robots: Boston Dynamics' Atlas humanoid, Unitree's G1 humanoid, and Boston Dynamics' Spot quadruped. Each policy required approximately 10 hours of training-about 7,000 iterations-on a single NVIDIA L4 GPU. On Atlas, the system reproduced a range of complex movements, including crawling, forward rolls, cartwheels, army crawling, and breakdancing. Video-derived skills included dancing, soccer kicking, and box climbing, while animation data enabled Spot to perform continuous backflips and barrel rolls. The demonstration was conducted in controlled, flat, and non-slippery environments, and the system has not yet shown generalization to entirely novel movements or unstructured terrain.
While ZEST reduces the need for task-specific engineering, it remains limited by the quality and diversity of available motion data and by the constraints of the physical robots. The framework does not currently support adaptation to new environments or tasks without retraining, and its performance in the presence of obstacles, uneven surfaces, or unexpected disturbances has not been established. The researchers note that future work will focus on enabling zero- and few-shot adaptation, continual learning, and higher-level control interfaces, such as language or keyframe commands.
Context and Related Work
The development of ZEST reflects a broader trend in robotics toward more flexible, data-driven skill acquisition. Previous research has often relied on extensive manual engineering or teleoperation to achieve complex behaviors, limiting scalability and adaptability. Recent advances in reinforcement learning and motion retargeting have enabled robots to learn from a wider range of human demonstrations, but challenges remain in transferring these skills to real hardware and in ensuring reliable performance outside controlled settings. For example, other research efforts, such as the LUMO robot, have explored adaptive mobility and interaction in varied environments, as discussed in this recent report on all-terrain humanoid navigation.
As robots are increasingly deployed in dynamic and human-centered environments, the ability to acquire and execute complex movements safely and reliably will depend not only on advances in learning algorithms but also on robust sensing, actuation, and safety engineering. The ZEST framework represents a step toward more generalizable skill transfer, but its current limitations highlight the need for continued research on adaptation, robustness, and human oversight in real-world deployment.
Reinforcement learning is a machine learning approach in which an agent learns to make decisions by interacting with an environment and receiving feedback in the form of rewards or penalties. In robotics, reinforcement learning can be used to train control policies that map sensor inputs to motor commands, enabling robots to acquire new skills through trial and error. However, transferring policies trained in simulation to physical robots-known as sim-to-real transfer-remains challenging due to differences between simulated and real-world dynamics. Techniques such as domain randomization, actuator modeling, and adaptive curricula are often used to bridge this gap, but reliable deployment still requires careful engineering and validation.
