In plain words: A large neural network learns from banking event records by hiding parts of transactions and predicting them, turning raw sequences into reusable embeddings. Across credit, fraud, and customer-value tasks it beats task-specific models trained from scratch, even with a simple linear classifier on top.
Abstract · PRAGMA: Revolut Foundation Model
Modern financial systems generate vast quantities of transactional and event-level data that encode rich economic signals. This paper presents PRAGMA, a family of foundation models for banking event sequences. Our approach pre-trains a Transformer-based architecture with masked modelling on a large-scale, heterogeneous banking event corpus using a self-supervised objective tailored to the discrete, variable-length nature of financial records. The resulting model supports a wide range of downstream tasks such as credit scoring, fraud detection, and lifetime value prediction: strong performance can be achieved by training a simple linear model on top of the extracted embeddings and can be further improved with lightweight fine-tuning. Through extensive evaluation on downstream tasks, we demonstrate that PRAGMA achieves superior performance across multiple domains directly from raw event sequences, providing a general-purpose representation layer for financial applications.
Maxim Ostroukhov, Ruslan Mikhailov, Vladimir Iashin, Artem Sokolov, Andrei Akshonov, Vitaly Protasov, Andrey Goncharov, Dmitrii Beloborodov, Vince Mullin, Roman Yokunda Enzmann, Georgios Kolovos, Jason Renders, et al.
arXiv:2604.08649 · cs.LG, cs.CE, cs.CL, cs.IR, q-fin.CP · submitted Apr 9, 2026 · updated Aug 24, 2026
abstract · pdf · html · [v2]: adds extra ablations and results; related work improvements