about
Twist: Teleoperated Whole-Body Imitation System (arxiv.org)
2 points by the-mitr on May 16, 2025 | hide | past | pdf | discuss on HN

In plain words: A humanoid robot copies a person's full-body movements by retargeting motion-capture clips onto its own body and learning one controller that imitates them. Unlike systems that handle only walking or only arm tasks, it performs both together plus expressive motion with a single network.

Abstract · TWIST: Teleoperated Whole-Body Imitation System

Teleoperating humanoid robots in a whole-body manner marks a fundamental step toward developing general-purpose robotic intelligence, with human motion providing an ideal interface for controlling all degrees of freedom. Yet, most current humanoid teleoperation systems fall short of enabling coordinated whole-body behavior, typically limiting themselves to isolated locomotion or manipulation tasks. We present the Teleoperated Whole-Body Imitation System (TWIST), a system for humanoid teleoperation through whole-body motion imitation. We first generate reference motion clips by retargeting human motion capture data to the humanoid robot. We then develop a robust, adaptive, and responsive whole-body controller using a combination of reinforcement learning and behavior cloning (RL+BC). Through systematic analysis, we demonstrate how incorporating privileged future motion frames and real-world motion capture (MoCap) data improves tracking accuracy. TWIST enables real-world humanoid robots to achieve unprecedented, versatile, and coordinated whole-body motor skills--spanning whole-body manipulation, legged manipulation, locomotion, and expressive movement--using a single unified neural network controller. Our project website: https://humanoid-teleop.github.io

Yanjie Ze, Zixuan Chen, João Pedro Araújo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu, C. Karen Liu
arXiv:2505.02833 · cs.RO, cs.CV, cs.LG · submitted May 5, 2025
abstract · pdf · html · Project website: https://humanoid-teleop.github.io

add comment on HN