about
Transform video into comics (arxiv.org)
1 point by godelmachine on Dec 12, 2018 | hide | past | pdf | discuss on HN

In plain words: It picks the most informative frames from a video, keeps the best-looking ones, and redraws them in a comic style to build a page. The result is a working website that turns any video into a comic automatically.

Abstract · Comixify: Transform video into a comics

In this paper, we propose a solution to transform a video into a comics. We approach this task using a neural style algorithm based on Generative Adversarial Networks (GANs). Several recent works in the field of Neural Style Transfer showed that producing an image in the style of another image is feasible. In this paper, we build up on these works and extend the existing set of style transfer use cases with a working application of video comixification. To that end, we train an end-to-end solution that transforms input video into a comics in two stages. In the first stage, we propose a state-of-the-art keyframes extraction algorithm that selects a subset of frames from the video to provide the most comprehensive video context and we filter those frames using image aesthetic estimation engine. In the second stage, the style of selected keyframes is transferred into a comics. To provide the most aesthetically compelling results, we selected the most state-of-the art style transfer solution and based on that implement our own ComixGAN framework. The final contribution of our work is a Web-based working application of video comixification available at http://comixify.ii.pw.edu.pl.

Maciej Pęśko, Adam Svystun, Paweł Andruszkiewicz, Przemysław Rokita, Tomasz Trzciński
arXiv:1812.03473 · cs.CV · submitted Dec 9, 2018
abstract · pdf · html · 14 pages. arXiv admin note: substantial text overlap with arXiv:1809.01726

add comment on HN
Also discussed: Jan 2019 (1 point, 0 comments) · Dec 2018 (4 points, 1 comment) · Dec 2018 (5 points, 0 comments)