Summary of Lived Experience Not Found: Llms Struggle to Align with Experts on Addressing Adverse Drug Reactions From Psychiatric Medication Use, by Mohit Chandra et al.
Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use
by Mohit Chandra, Siddharth Sriraman, Gaurav Verma, Harneet Singh Khanuja, Jose Suarez Campayo, Zihang Li, Michael L. Birnbaum, Munmun De Choudhury
First submitted to arxiv on: 24 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the potential of Large Language Models (LLMs) in detecting Adverse Drug Reactions (ADRs) related to psychiatric medications and providing effective harm reduction strategies. The authors introduce a new benchmark, Psych-ADR, and an evaluation framework, ADRA, to assess LLM performance in this task. Results show that LLMs struggle with nuanced ADR understanding and differentiation between types of ADRs. While they align with expert opinions on expressed emotions and tone, their responses are more complex and less actionable, with only 70.86% alignment with expert strategies. The study provides a comprehensive benchmark and evaluation framework for assessing LLMs in strategy-driven tasks within high-risk domains. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how computers can help us identify problems caused by medicines used to treat mental health issues. It’s important because these problems are very common and can be serious. The researchers create a new way to test computer programs’ ability to understand and solve this problem. They found that while the computers can understand some things, they have trouble with more complex issues. The computers also don’t give as many helpful suggestions as experts would. This study helps us understand how we can use computers to better handle these problems. |
Keywords
» Artificial intelligence » Alignment
