Google releases VaultGemma, its first privacy-preserving LLM (arstechnica.com)

Google Research has released VaultGemma, its first privacy-preserving large language model, and published an empirical study characterizing how differential privacy (DP) affects LLM performance. To reduce the risk that models “memorize” sensitive or copyrighted training data, VaultGemma’s training uses DP—injecting calibrated noise during optimization—which prevents verbatim leakage but traditionally harms accuracy and raises compute costs. The team focused on the noise-batch ratio (the volume of random noise relative to the training data) and ran experiments across model sizes to quantify those trade-offs. The paper establishes DP scaling laws that describe a three-way balance between privacy budget, compute budget (FLOPs), and data budget (tokens): more noise degrades output quality unless compensated by more compute or more training data. Practically, this gives model builders a roadmap for choosing an appropriate noise-batch ratio to meet privacy targets while managing accuracy and cost. For the AI/ML community, VaultGemma and the accompanying scaling-law analysis make privacy-preserving model design more predictable, enabling informed engineering trade-offs when protecting user data or avoiding copyrighted memorization.