about
Image Inpainting for Irregular Holes Using Partial Convolutions (arxiv.org)
2 points by lelf on Mar 24, 2019 | hide | past | pdf | discuss on HN

In plain words: To fill in missing parts of a picture, this system ignores the blank areas when reading each layer of pixels, instead of treating filler values as real data. It produced cleaner, more accurate fills than other approaches on ragged, irregular holes.

Abstract

Existing deep learning based image inpainting methods use a standard convolutional network over the corrupted image, using convolutional filter responses conditioned on both valid pixels as well as the substitute values in the masked holes (typically the mean value). This often leads to artifacts such as color discrepancy and blurriness. Post-processing is usually used to reduce such artifacts, but are expensive and may fail. We propose the use of partial convolutions, where the convolution is masked and renormalized to be conditioned on only valid pixels. We further include a mechanism to automatically generate an updated mask for the next layer as part of the forward pass. Our model outperforms other methods for irregular masks. We show qualitative and quantitative comparisons with other methods to validate our approach.

Guilin Liu, Fitsum A. Reda, Kevin J. Shih, Ting-Chun Wang, Andrew Tao, Bryan Catanzaro
arXiv:1804.07723 · cs.CV · submitted Apr 20, 2018 · updated Dec 15, 2018
abstract · pdf · html · Update: camera-ready; L1 loss is size-averaged; code of partial conv layer: https://github.com/NVIDIA/partialconv. Published at ECCV 2018

add comment on HN
Also discussed: Apr 2018 (3 points, 0 comments)