Summary of Burextract-llama: An Llm For Clinical Concept Extraction in Breast Ultrasound Reports, by Yuxuan Chen et al.
BURExtract-Llama: An LLM for Clinical Concept Extraction in Breast Ultrasound Reports
by Yuxuan Chen, Haoyan Yang, Hengkai Pan, Fardeen Siddiqui, Antonio Verdone, Qingyang Zhang, Sumit Chopra, Chen Zhao, Yiqiu Shen
First submitted to arxiv on: 21 Aug 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This study proposes a pipeline for developing an in-house large language model (LLM) to extract clinical information from radiology reports. The unstructured nature of these reports, with varying linguistic styles and formatting, makes it challenging to extract key findings like lesion characteristics and malignancy assessments. While proprietary LLMs like GPT-4 are effective, they can be costly and raise privacy concerns when handling protected health information. To address this, the authors use GPT-4 to create a small labeled dataset, which is then fine-tuned on a Llama3-8B model. The resulting model achieves an average F1 score of 84.6%, comparable to GPT-4’s performance. This study demonstrates the feasibility of developing an in-house LLM that not only matches GPT-4’s performance but also offers cost reductions and enhanced data privacy. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research makes it possible for doctors to automatically extract important information from radiology reports, like what kinds of abnormalities are present and whether they might be cancerous. The reports are currently hard to read because they’re written in different styles and don’t follow a standard format. This means that doctors have to spend a lot of time reading through the reports to find the most important information. The authors developed a special computer program, called a large language model (LLM), to help with this problem. They used a well-known LLM, GPT-4, to create a small dataset of labeled reports, and then fine-tuned another LLM, Llama3-8B, on this dataset. The results show that their in-house LLM is just as good at finding important information in radiology reports as the proprietary LLM, but it’s more affordable and keeps patients’ health information private. |
Keywords
» Artificial intelligence » F1 score » Gpt » Large language model
