Summary of Dean: Deactivating the Coupled Neurons to Mitigate Fairness-privacy Conflicts in Large Language Models, by Chen Qian et al.
DEAN: Deactivating the Coupled Neurons to Mitigate Fairness-Privacy Conflicts in Large Language Models
by Chen Qian, Dongrui Liu, Jie Zhang, Yong Liu, Jing Shao
First submitted to arxiv on: 22 Oct 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The research introduces a novel approach to enhance fairness and privacy awareness in Large Language Models (LLMs) by leveraging information theory. The study reveals a counter-intuitive trade-off between privacy and fairness, where Supervised Fine-Tuning (SFT) methods decrease LLMs’ fairness awareness despite improving their privacy awareness. To address this issue, the authors propose a training-free method called DEAactivate the fairness and privacy coupled Neurons (DEAN), which theoretically and empirically reduces the mutual information between fairness and privacy awareness. Experimental results demonstrate that DEAN eliminates the trade-off phenomenon and improves LLMs’ fairness and privacy awareness simultaneously, such as improving Qwen-2-7B-Instruct’s fairness awareness by 12.2% and privacy awareness by 14.0%. The study highlights the importance of concurrently addressing fairness and privacy concerns in LLMs and provides insights for developing more ethical AI systems. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The research explores how to make Large Language Models (LLMs) fairer and more private. Currently, making an LLM more private actually makes it less fair. This is a problem because we want both fairness and privacy in our AI models. To solve this issue, the authors create a new way to train LLMs that keeps them fair and private at the same time. They call this method DEAN. The study shows that DEAN works well and can improve an LLM’s fairness and privacy by 12.2% and 14.0%, respectively. This is important because it means we can create more ethical AI systems. |
Keywords
» Artificial intelligence » Fine tuning » Supervised
