Summary of Latent Dataset Distillation with Diffusion Models, by Brian B. Moser et al.
Latent Dataset Distillation with Diffusion Models
by Brian B. Moser, Federico Raue, Sebastian Palacio, Stanislav Frolov, Andreas Dengel
First submitted to arxiv on: 6 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach called Latent Dataset Distillation with Diffusion Models (LD3M) to address the challenges of dataset distillation in machine learning. LD3M combines diffusion in latent space with dataset distillation, using a ConvNet architecture to link the original and synthetic datasets. The approach is designed to improve the gradient flow for distillation and offers a trade-off between distillation speed and dataset quality by adjusting the number of diffusion steps. Experimental results show that LD3M consistently outperforms state-of-the-art methods on several ImageNet subsets and high resolutions (128×128 and 256×256) by up to 4.8 p.p. and 4.2 p.p. for 1 and 10 images per class, respectively. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about a new way to make machine learning models work better with smaller datasets. Right now, we need big datasets to train these models, but that’s not always possible or practical. So, scientists are working on “distilling” larger datasets into smaller, more useful ones. The problem is that the model used to create this distilled dataset matters a lot – if it’s different from the one you’re training with, the results won’t be as good. This paper introduces a new approach called Latent Dataset Distillation with Diffusion Models (LD3M) that solves this problem and produces better results. |
Keywords
* Artificial intelligence * Diffusion * Distillation * Latent space * Machine learning
