Peking University and Qiyuan Robotics introduce HIL-UMI for robot-free post-training data collection

What happened

Dong Hao’s team at Peking University and Qiyuan Robotics introduced HIL-UMI, a research system that collects targeted robot post-training corrections through handheld UMI demonstrations without a robot present. It detects differences between human actions and model predictions, then uses an advantage estimator to prioritize actions that advance tasks. Across four physical robot tasks, reported average task progress reached 94.8 versus 58.0 for continued SFT; effective data collection on table organization was 5.63 times faster than HG-DAgger.

The story so far

Dong Hao’s team at Peking University and Qiyuan Robotics introduced HIL-UMI, a research system that collects targeted corrections for robot post-training through handheld UMI demonstrations without a robot present, and reported improved task progress and faster effective data collection in testing. HIL-UMI detects differences between human actions and model predictions, then uses an advantage estimator to prioritize actions that advance a task. Across four physical robot tasks, the team reported average task progress of 94.8, compared with 58.0 for continued supervised fine-tuning (SFT). On table organization, effective data collection was reported to be 5.63 times faster than HG-DAgger. The robot-free capability applies to collecting post-training demonstrations; the task-performance results came from physical robot tests.

Keep reading