Summary of Fine-tuning Discrete Diffusion Models Via Reward Optimization with Applications to Dna and Protein Design, by Chenyu Wang et al.
Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design
by Chenyu Wang, Masatoshi Uehara, Yichun He, Amy Wang, Tommaso Biancalani, Avantika Lal, Tommi Jaakkola, Sergey Levine, Hanchen Wang, Aviv Regev
First submitted to arxiv on: 17 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this research paper, the authors propose a novel algorithm called DRAKES to optimize discrete diffusion models for task-oriented sequence generation. This approach leverages pre-trained discrete diffusion models that can generate natural-like sequences and reward models that map sequences to specific task objectives. The authors formulate the reward maximization problem within discrete diffusion models using reinforcement learning (RL) principles, while minimizing the KL divergence against pre-trained diffusion models to preserve naturalness. The DRAKES algorithm enables direct backpropagation of rewards through entire trajectories generated by diffusion models using the Gumbel-Softmax trick. The authors demonstrate the effectiveness of their approach in generating DNA and protein sequences that optimize enhancer activity and protein stability, respectively. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary In this paper, researchers develop a new method to create useful biological sequences like DNA or proteins. They use special computer models called diffusion models, which can generate natural-looking sequences. But they want these sequences to have specific properties, like being stable or having certain activities. To do this, they combine the diffusion models with “reward” systems that help guide the generation process. The authors create a new algorithm called DRAKES that lets them optimize their sequence generation for specific tasks. They test this approach on DNA and protein sequences and show it can be effective in creating useful biological sequences. |
Keywords
» Artificial intelligence » Backpropagation » Diffusion » Reinforcement learning » Softmax
