Summary of When Less Is Not More: Large Language Models Normalize Less-frequent Terms with Lower Accuracy, by Daniel B. Hier and Thanh Son Do and Tayo Obafemi-ajayi
When Less Is Not More: Large Language Models Normalize Less-Frequent Terms with Lower Accuracy
by Daniel B. Hier, Thanh Son Do, Tayo Obafemi-Ajayi
First submitted to arxiv on: 11 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A machine learning-based approach to normalize terms from free text to standardized concepts in the Human Phenotype Ontology (HPO) is crucial for precision medicine initiatives. Large language models like GPT-4o can perform this task but may retrieve incorrect HPO IDs, leading to inaccurate results. A study using a comprehensive dataset of 268,776 phenotype annotations found that GPT-4o achieved an accuracy of 13.1% in normalizing terms, with higher-frequency and shorter terms being normalized more accurately than lower-frequency and longer terms. Feature importance analysis identified low-term frequency as the most significant predictor of normalization errors. These findings suggest that training and evaluation datasets should balance low- and high-frequency terms to improve model performance, particularly for infrequent terms critical to precision medicine. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Term normalization is important for precision medicine because it helps match patients with similar diseases. A big language model called GPT-4o can do this job but sometimes gets it wrong. Researchers used a huge dataset of disease descriptions and found that GPT-4o got it right 13% of the time, but was better at matching common words than rare ones. This means we need to use more diverse training data to help the model make better predictions. |
Keywords
» Artificial intelligence » Gpt » Language model » Machine learning » Precision
