about
iPhone Deployment of End-to-End Perception via Auto-Labeled Synthetic Data (arxiv.org)
1 point by PaulHoule 120 days ago | hide | past | pdf | discuss on HN

In plain words: An iPhone app turns a video of an object into a detector and pose estimator: it builds a 3D copy, pastes it into varied scenes for auto-labeled training images, and trains on them. It beat the usual geometry-based pose method on four of five objects.

Abstract · FalconApp: Rapid iPhone Deployment of End-to-End Perception via Automatically Labeled Synthetic Data

Reliable perception for robotics depends on large-scale labeled data, yet real-world datasets rely on heavy manual annotation and are time-consuming to produce. We present FalconApp, an iPhone app with an end-to-end frontend-backend pipeline that turns a short handheld capture of a rigid object into a perception module for mask detection and 6-DoF pose estimation. Our core contribution is a rapid mobile deployment pipeline paired with a photorealistic auto-labeling workflow: from a user-captured video of an object, FalconApp reconstructs an editable GSplat asset, composites it with diverse photorealistic backgrounds, renders synthetic images with ground-truth masks and poses, trains the perception module, and deploys it back to the iPhone frontend. Experiments across five rigid objects with diverse geometry and appearance show that FalconApp produces usable perception models with about 20 minutes of synthetic-data generation and training per object on average, around 30 ms end-to-end on-device latency on iPhone, and better overall pose accuracy than a PnP baseline on 4 / 5 objects in both simulation and real-world evaluation.

Yan Miao, Will Shen, Sayan Mitra
arXiv:2604.25949 · cs.RO · submitted Apr 21, 2026 · updated Jun 30, 2026
abstract · pdf · html

add comment on HN