about
Generating GUI code from screenshots using deep learning (arxiv.org)
1 point by thevivekpandey on Sep 22, 2017 | hide | past | pdf | discuss on HN

In plain words: A system turns a screenshot of an app or website design straight into code, learning from paired examples so it writes the layout itself. It produced correct code over 77% of the time on iPhone, Android, and web, where people write that code by hand.

Abstract · pix2code: Generating Code from a Graphical User Interface Screenshot

Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications. In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).

Tony Beltramelli
arXiv:1705.07962 · cs.LG, cs.AI, cs.CL, cs.CV, cs.NE · submitted May 22, 2017 · updated Sep 19, 2017
abstract · pdf · html

add comment on HN
Also discussed: May 2017 (2 points, 0 comments) · May 2017 (4 points, 1 comment) · May 2017 (2 points, 0 comments)