Summary of Clear: Towards Robust and Generalized Parameter-efficient Fine-tuning For Noisy Label Learning, by Yeachan Kim et al.
CleaR: Towards Robust and Generalized Parameter-Efficient Fine-Tuning for Noisy Label Learning
by Yeachan Kim, Junho Kim, SangKeun Lee
First submitted to arxiv on: 31 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores parameter-efficient fine-tuning (PEFT) methods for language models in real-world settings with noisy labels. PEFT’s adaptability to noisy environments is investigated, revealing both strengths and limitations. While PEFT has difficulty memorizing noisy labels due to its limited capacity, making it more robust, this same limitation makes it vulnerable to interference from noisy labels, hindering the learning of clean samples. To address this issue, the authors propose Clean Routing (CleaR), a routing-based PEFT approach that adapts PEFT modules to prioritize exposure to clean data and minimize noisy influence. Experimental results demonstrate CleaR’s effectiveness in improving performance in noisy environments. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how language models can be trained quickly and efficiently, even when the training data is not perfect. They find that while this approach can work well, it also has some limitations. The main problem is that the model can get stuck learning from bad examples, rather than good ones. To solve this issue, the authors developed a new way to fine-tune the language models, called Clean Routing (CleaR). This method helps the model focus on the good examples and ignore the bad ones. The results show that CleaR works much better than previous methods in noisy environments. |
Keywords
* Artificial intelligence * Fine tuning * Parameter efficient
