about
The Unreasonable Effectiveness of Linear Prediction as a Perceptual Metric (arxiv.org)
1 point by E-Reverance 38 days ago | hide | past | pdf | discuss on HN

In plain words: It scores how similar two images look by solving a simple pixel-level fitting problem on the fly, with no training or deep network features. On image quality tests it matched learned deep-feature scores while running about as cheaply as simple hand-crafted measures.

Abstract

We show how perceptual embeddings of the visual system can be constructed at inference-time with no training data or deep neural network features. Our perceptual embeddings are solutions to a weighted least squares (WLS) problem, defined at the pixel-level, and solved at inference-time, that can capture global and local image characteristics. The distance in embedding space is used to define a perceptual similarity metric which we call LASI: Linear Autoregressive Similarity Index. Experiments on full-reference image quality assessment datasets show LASI performs competitively with learned deep feature based methods like LPIPS (Zhang et al., 2018) and PIM (Bhardwaj et al., 2020), at a similar computational cost to hand-crafted methods such as MS-SSIM (Wang et al., 2003). We found that increasing the dimensionality of the embedding space consistently reduces the WLS loss while increasing performance on perceptual tasks, at the cost of increasing the computational complexity. LASI is fully differentiable, scales cubically with the number of embedding dimensions, and can be parallelized at the pixel-level. A Maximum Differentiation (MAD) competition (Wang & Simoncelli, 2008) between LASI and LPIPS shows that both methods are capable of finding failure points for the other, suggesting these metrics can be combined.

Daniel Severo, Lucas Theis, Johannes Ballé
arXiv:2310.05986 · cs.CV · submitted Oct 6, 2023
abstract · pdf · html

add comment on HN