In plain words: VaultGemma 1B is a 1-billion-parameter model trained with differential privacy, adding noise so no single person's data can be picked out. Trained on the same data mix as Gemma 2, it shows a 1B model can be trained this way and is openly released.
Abstract · VaultGemma: A Differentially Private Gemma Model
We introduce VaultGemma 1B, a 1 billion parameter model within the Gemma family, fully trained with differential privacy. Pretrained on the identical data mixture used for the Gemma 2 series, VaultGemma 1B represents a significant step forward in privacy-preserving large language models. We openly release this model to the community
Amer Sinha, Thomas Mesnard, Ryan McKenna, Daogao Liu, Christopher A. Choquette-Choo, Yangsibo Huang, Da Yu, George Kaissis, Zachary Charles, Ruibo Liu, Lynn Chua, Pritish Kamath, et al.
arXiv:2510.15001 · cs.CR, cs.AI · submitted Oct 15, 2025 · updated Oct 22, 2025
abstract · pdf · html