In plain words: A tool questions a trained memory network and uses its wrong answers to build a state machine showing how it tracks input. It recovers accurate machines even when the network's internal state is large and finely detailed, which usually makes such extraction hard.

Abstract · Extracting Automata from Recurrent Neural Networks Using Queries and Counterexamples

We present a novel algorithm that uses exact learning and abstraction to extract a deterministic finite automaton describing the state dynamics of a given trained RNN. We do this using Angluin's L* algorithm as a learner and the trained RNN as an oracle. Our technique efficiently extracts accurate automata from trained RNNs, even when the state vectors are large and require fine differentiation.

Gail Weiss, Yoav Goldberg, Eran Yahav
arXiv:1711.09576 · cs.LG, cs.FL · submitted Nov 27, 2017 · updated Feb 27, 2020
abstract · pdf · html · Accepted in ICML 2018, (Feb 2020: added link to code, at https://github.com/tech-srl/lstar_extraction )