Get Started

Differential Privacy for LLMs

DP-SGD remains the standard, noise-injected gradient updates provide formal (epsilon, delta) guarantees. Overhead in compute, memory, and utility is substantial; parameter-efficient DP fine-tuning (LoRA + DP-SGD) is the common 2025 configuration (ACM 2025). Two bodies of evidence in tension: canary-based membership inference (Duan et al., 2024) reports limited success against language models; training-data extraction (Carlini et al., 2021; Nasr et al., 2025) recovers substantial verbatim memorization. Resolution (arXiv:2503.06808, March 2025): the gap is in what is measured, inserted canaries vs "most extractable" data. New canary designs enable loss-based MIA without shadow models and yield the first nontrivial DP audit of an LLM trained on real data with realistic DP guarantees. Alternatives: PMixED (arXiv:2403.15638), private prediction at inference time via mixture of experts on next-token distributions; DP synthetic data generation (Google Research 2024). Emerging attack: Differential Privacy Reversal via LLM Feedback, confidence-score leakage.

60 minutesBuildPython (stdlib, DP-SGD noise-injection and ε-δ accountant demonstration)Members

Learning Objectives

  • Define (epsilon, delta)-differential privacy and state the DP-SGD recipe.
  • Explain the 2024-2025 tension: canary MIA vs training-data extraction give different pictures.
  • Describe PMixED and why inference-time private prediction is an alternative to DP training.
  • Describe the Differential Privacy Reversal via LLM Feedback attack.