about
A Generative Cramér-Rao Bound (arxiv.org)
1 point by hasley 49 days ago | hide | past | pdf | 1 comment on HN

In plain words: Normally, finding the Cramér-Rao bound—the best accuracy any unbiased guess can reach—needs a hand-written model of the data. Here the distribution is learned from examples instead, and the bound is computed from it, working well on image denoising and edge detection.

Abstract · Learning to Bound: A Generative Cramér-Rao Bound

The Cramér-Rao bound (CRB), a well-known lower bound on the performance of any unbiased parameter estimator, has been used to study a wide variety of problems. However, to obtain the CRB, requires an analytical expression for the likelihood of the measurements given the parameters, or equivalently a precise and explicit statistical model for the data. In many applications, such a model is not available. Instead, this work introduces a novel approach to approximate the CRB using data-driven methods, which removes the requirement for an analytical statistical model. This approach is based on the recent success of deep generative models in modeling complex, high-dimensional distributions. Using a learned normalizing flow model, we model the distribution of the measurements and obtain an approximation of the CRB, which we call Generative Cramér-Rao Bound (GCRB). Numerical experiments on simple problems validate this approach, and experiments on two image processing tasks of image denoising and edge detection with a learned camera noise model demonstrate its power and benefits.

Hai Victor Habi, Hagit Messer, Yoram Bresler
arXiv:2203.03695 · cs.LG, eess.SP · submitted Mar 7, 2022 · updated Oct 9, 2022
abstract · pdf · html

add comment on HN

The Cramér-Rao Bound (CRB) lets one compute the minimal mean-squared error that any unbiased estimator (algorithm) can achieve. Unfortunately, you need to know the probability density function (PDF) to compute the CRB. This work proposes a way to overcome this drawback. Maybe with this, the CRB gains more attention in the machine learning community - or even as much attention as it gets in the parameter estimation community.