Summary of Differentially Private Fine-tuning Of Diffusion Models, by Yu-lin Tsai et al.

Differentially Private Fine-Tuning of Diffusion Models

by Yu-Lin Tsai, Yizhe Li, Zekai Chen, Po-Yu Chen, Chia-Mu Yu, Xuebin Ren, Francois Buet-Golfouse

First submitted to arxiv on: 3 Jun 2024

GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty	Written by	Summary
High	Paper authors	High Difficulty Summary Read the original abstract here
Medium	GrooveSquid.com (original content)	Medium Difficulty Summary Medium Difficulty Summary: The integration of Differential Privacy (DP) with diffusion models (DMs) is a promising yet challenging frontier, particularly due to the substantial memorization capabilities of DMs that pose significant privacy risks. Our work proposes a parameter-efficient fine-tuning strategy optimized for private diffusion models, which minimizes the number of trainable parameters to enhance the privacy-utility trade-off. We empirically demonstrate that our method achieves state-of-the-art performance in DP synthesis, significantly surpassing previous benchmarks on widely studied datasets (e.g., with only 0.47M trainable parameters, achieving a more than 35% improvement over the previous state-of-the-art with a small privacy budget on the CelebA-64 dataset). Our approach leverages Differential Privacy Stochastic Gradient Descent (DP-SGD) and diffusion method decompositions to generate high-quality synthetic data by pre-training on public data (i.e., ImageNet) and fine-tuning on private data. We also explore the potential for generating synthetic data with improved privacy guarantees.
Low	GrooveSquid.com (original content)	Low Difficulty Summary Low Difficulty Summary: This paper combines two important ideas in computer science: keeping personal information private and making computers create realistic images. The first idea is called Differential Privacy, which helps keep personal information safe by adding random noise to the data. The second idea is diffusion models, which are special kinds of artificial intelligence that can generate realistic images. By combining these two ideas, researchers have developed a new way to create synthetic data while keeping it private. This paper proposes a new method for creating this synthetic data, which uses fewer computer resources than previous methods and still produces high-quality results.