Summary of Editing Factual Knowledge and Explanatory Ability Of Medical Large Language Models, by Derong Xu et al.
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models
by Derong Xu, Ziheng Zhang, Zhihong Zhu, Zhenxi Lin, Qidong Liu, Xian Wu, Tong Xu, Wanyu Wang, Yuyang Ye, Xiangyu Zhao, Enhong Chen, Yefeng Zheng
First submitted to arxiv on: 28 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Model editing is a technique that enables precise adjustments to large language models (LLMs) while preserving unrelated knowledge. This approach has shown promise in addressing issues like hallucination and outdated information in LLMs. However, its potential application in the medical field remains largely unexplored, despite the need for resolving hallucinations being pressing. To tackle this challenge, we propose MedLaSA, a novel Layer-wise Scalable Adapter strategy for medical model editing. MedLaSA combines the strengths of adding extra parameters and locate-then-edit methods to edit medical models. We utilize causal tracing to identify associations between knowledge in neurons across different layers and generate a corresponding scale set based on these associations. Our approach incorporates scalable adapters into dense LLM layers, assigning scaling values based on specific knowledge. This ensures precise editing of semantically identical knowledge while avoiding impact on unrelated knowledge. To evaluate the editing’s impact on medical LLMs, we propose two studies: editing factual knowledge for medical specialization and editing explanatory ability for complex knowledge. We built novel medical benchmarking datasets and introduced comprehensive metrics. Extensive experiments demonstrate MedLaSA’s editing efficiency without affecting unrelated knowledge. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Model editing helps make large language models better at certain things while keeping other knowledge the same. This technique can help fix problems like hallucinations in language models. In medicine, this could be especially important. But no one has looked into using model editing in medical settings yet. We want to change that! We’re proposing a new way to edit medical language models called MedLaSA. It combines two existing methods to make it work well for medical data. We use special tracing to figure out how different pieces of knowledge are related and then adjust the model’s behavior accordingly. This lets us make sure the model gets better at things that are similar, without messing up other areas. We want to test this idea by studying how well it works for two specific tasks: making language models better at medical facts and helping them explain complex information more clearly. We’re building new datasets and metrics to help us measure how well our approach works. |
Keywords
» Artificial intelligence » Hallucination
