In plain words: A roadmap sorts machine learning safety research into four tasks: resisting hazards, spotting them, making models less harmful by design, and fixing risks built into the wider system. It sharpens the field's vague worries into concrete, ready-to-study problems with suggested research directions.
Abstract
Machine learning (ML) systems are rapidly increasing in size, are acquiring new capabilities, and are increasingly deployed in high-stakes settings. As with other powerful technologies, safety for ML should be a leading research priority. In response to emerging safety challenges in ML, such as those introduced by recent large-scale models, we provide a new roadmap for ML Safety and refine the technical problems that the field needs to address. We present four problems ready for research, namely withstanding hazards ("Robustness"), identifying hazards ("Monitoring"), reducing inherent model hazards ("Alignment"), and reducing systemic hazards ("Systemic Safety"). Throughout, we clarify each problem's motivation and provide concrete research directions.
Dan Hendrycks, Nicholas Carlini, John Schulman, Jacob Steinhardt
arXiv:2109.13916 · cs.LG, cs.AI, cs.CL, cs.CV · submitted Sep 28, 2021 · updated Jun 16, 2022
abstract · pdf · html · Position Paper