Summary of Core-behrt: a Carefully Optimized and Rigorously Evaluated Behrt, by Mikkel Odgaard et al.
CORE-BEHRT: A Carefully Optimized and Rigorously Evaluated BEHRT
by Mikkel Odgaard, Kiril Vadimovic Klein, Sanne Møller Thysen, Espen Jimenez-Solem, Martin Sillesen, Mads Nielsen
First submitted to arxiv on: 23 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents research on optimizing BERT-based models for Electronic Health Records (EHR) analysis. Building on popular models like BEHRT and Med-BERT, the study isolates the impact of design choices such as data representation, individual components, and training procedures. The authors evaluate their optimizations across a range of generic tasks, including death prediction, pain treatment, and infection detection, finding significant improvements in average downstream performance when incorporating medication and timestamp information. They also demonstrate consistent results across 25 diverse clinical prediction tasks, highlighting the generalizability of their findings. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper explores ways to improve BERT-based models for analyzing Electronic Health Records (EHR). By fine-tuning these models, researchers can better understand healthcare data. The study looks at what works best for different parts of EHR analysis and finds that adding more information about medications and timestamps helps make predictions more accurate. |
Keywords
» Artificial intelligence » Bert » Fine tuning
