In plain words: Simitate pairs 1,938 videos of people doing daily tasks with a 3D simulator, so a robot can learn by copying and be tested in it. It records video and exact hand and object positions, scoring imitations on how well their path and effect match.
Abstract
We present Simitate --- a hybrid benchmarking suite targeting the evaluation of approaches for imitation learning. A dataset containing 1938 sequences where humans perform daily activities in a realistic environment is presented. The dataset is strongly coupled with an integration into a simulator. RGB and depth streams with a resolution of 960$\mathbb{\times}$540 at 30Hz and accurate ground truth poses for the demonstrator's hand, as well as the object in 6 DOF at 120Hz are provided. Along with our dataset we provide the 3D model of the used environment, labeled object images and pre-trained models. A benchmarking suite that aims at fostering comparability and reproducibility supports the development of imitation learning approaches. Further, we propose and integrate evaluation metrics on assessing the quality of effect and trajectory of the imitation performed in simulation. Simitate is available on our project website: \url{https://agas.uni-koblenz.de/data/simitate/}.
Raphael Memmesheimer, Ivanna Mykhalchyshyna, Viktor Seib, Dietrich Paulus
arXiv:1905.06002 · cs.LG, cs.RO, stat.ML · submitted May 15, 2019
abstract · pdf · html · 6 figures, 2 tables, submitted to IROS 2019