about
Object Wake-Up: 3-D Object Reconstruction, Animation from a Single Image (arxiv.org)
30 points by lnyan on Aug 6, 2021 | hide | past | pdf | 4 comments on HN

In plain words: From one photo, it rebuilds an object in 3D and predicts where its joints and parts belong, like a skeleton inside a toy, so it can move. It beat earlier methods by a noticeable margin at both shaping the object and finding its joints.

Abstract · Object Wake-up: 3D Object Rigging from a Single Image

Given a single image of a general object such as a chair, could we also restore its articulated 3D shape similar to human modeling, so as to animate its plausible articulations and diverse motions? This is an interesting new question that may have numerous downstream augmented reality and virtual reality applications. Comparing with previous efforts on object manipulation, our work goes beyond 2D manipulation and rigid deformation, and involves articulated manipulation. To achieve this goal, we propose an automated approach to build such 3D generic objects from single images and embed articulated skeletons in them. Specifically, our framework starts by reconstructing the 3D object from an input image. Afterwards, to extract skeletons for generic 3D objects, we develop a novel skeleton prediction method with a multi-head structure for skeleton probability field estimation by utilizing the deep implicit functions. A dataset of generic 3D objects with ground-truth annotated skeletons is collected. Empirically our approach is demonstrated with satisfactory performance on public datasets as well as our in-house dataset; our results surpass those of the state-of-the-arts by a noticeable margin on both 3D reconstruction and skeleton prediction.

Ji Yang, Xinxin Zuo, Sen Wang, Zhenbo Yu, Xingyu Li, Bingbing Ni, Minglun Gong, Li Cheng
arXiv:2108.02708 · cs.CV · submitted Aug 5, 2021 · updated Jul 6, 2022
abstract · pdf · html · ECCV 2022

add comment on HN

Hold on to your models?

Here is the 2D version. https://youtu.be/BpApq2EPDXE

AI-enhanced object reconstruction would be great for 3D printing enthusiasts. The entire object scanning ecosystem is kind of a shitshow right now.
What is the object scanning ecosystem?

As a hobbyist I can grab a bunch of photos of an object, feed them to photogrammetry software such as Agisoft Metashape, export the generated 3D mesh and convert into watertight printable mesh using Meshlab.

Is there anything else out there?

Try scanning finer features or complex setups, you will hit the limits of photogrammetry fast.

3d scanners and photogrammetry didn’t solve hair scanning yet, to name one example. And I am not talking up to each strands, just ending up with somewhat of a model without having to postprod based on hair type, etc. that’s just one example of many.