about
Introduction to Neural Network Verification [pdf] (arxiv.org)
65 points by belter on Sep 23, 2021 | hide | past | pdf | 12 comments on HN

In plain words: Formal verification proves mathematically that a neural network will behave safely for every input in a range, instead of just checking a sample of test cases. This book introduces the core ideas behind such proofs and shows how to adapt them to deep learning.

Abstract · Introduction to Neural Network Verification

Deep learning has transformed the way we think of software and what it can do. But deep neural networks are fragile and their behaviors are often surprising. In many settings, we need to provide formal guarantees on the safety, security, correctness, or robustness of neural networks. This book covers foundational ideas from formal verification and their adaptation to reasoning about neural networks and deep learning.

Aws Albarghouthi
arXiv:2109.10317 · cs.LG, cs.AI, cs.PL · submitted Sep 21, 2021 · updated Oct 4, 2021
abstract · pdf · html

add comment on HN
Also discussed: Sep 2021 (3 points, 0 comments)

A weird property of the described abstractions is that as you go tighter (interval -> zonotope -> polyhedra), the trained networks counterintuitively become less robust. Why does more precision in verification hurt training?

A recent work not mentioned in the last chapter "Adversarial Training with Abstraction" is [1], which kind of explains this issue using the notions of continuity and sensitivity of the abstractions.

[1]: https://arxiv.org/abs/2102.06700

Does anyone know some employers that are hiring for this stuff in industry? I can’t imagine many startups apply this stuff due to prohibitive costs. Research in this area is obviously orders of magnitude more computationally taxing than simply training neural networks.
Any proper selfdriving or other advanced robotics company should do. In my company (specialized autonomous vehicles) we’ll probably have such role soon.
Maybe Bosch? Prof. Zico Kolter from CMU is a chief scientist associated with them, and his group does a lot of really good work in the ml verification space (e.g. the first randomized smoothing and the Wong & Kolter certificates).
I agree, it's going to cost a lot of money to figure out if that shit actually works.
So, stupid question, is it possible to train neural networks using 3D CGI that will then be somewhat performant in real life?

Like, would a network be able to overcome the error due to switch between a 3d model and actual video?

I believe Tesla uses something similar for training. https://youtu.be/j0z4FweCy4M?t=5716
I’m sure there are videos around of people generating synthetic datasets using Blender 3D.
For anyone interested in elegant implementations of state of the art algorithms for verification, there is a nice library in Jax:

https://deepmind.com/research/open-source/efficient-and-tigh...

This is something I am very interested in, There’s lot of work to be done when it comes to building verified and explainable learning systems (not just neural networks).

I think the verification tools are finally getting better to the point of them being useful for this kind of stuff. L

Neural network verification is one of the most exciting research areas IMO, but is still little understood! The maths behind it are beautiful btw.

Good job putting this together :)

Are there any specific mathematical properties that you find beautiful that arise out of this field?

I personally find the reasoning on graphs(neurons) to make the verification tractable really beautiful. :)