about
Unsolved Problems in Machine Learning Safety (arxiv.org)
2 points by lnyan on Sep 29, 2021 | hide | past | pdf | discuss on HN

In plain words: A roadmap sorts machine learning safety research into four tasks: resisting hazards, spotting them, making models less harmful by design, and fixing risks built into the wider system. It sharpens the field's vague worries into concrete, ready-to-study problems with suggested research directions.

Abstract · Unsolved Problems in ML Safety

Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings. As with other powerful technologies, safety for ML should be a leading research priority. In response to emerging safety challenges in ML, such as those introduced by recent large-scale models, we provide a new roadmap for ML Safety and refine the technical problems that the field needs to address. We present four problems ready for research, namely withstanding hazards ("Robustness"), identifying hazards ("Monitoring"), reducing inherent model hazards ("Alignment"), and reducing systemic hazards ("Systemic Safety"). Throughout, we clarify each problem's motivation and provide concrete research directions.

Dan Hendrycks, Nicholas Carlini, John Schulman, Jacob Steinhardt
arXiv:2109.13916 · cs.LG, cs.AI, cs.CL, cs.CV · submitted Sep 28, 2021 · updated Jun 16, 2022
abstract · pdf · html · Position Paper

add comment on HN
Also discussed: Oct 2021 (61 points, 3 comments) · Oct 2021 (1 point, 0 comments)