Summary of Small Models Are Llm Knowledge Triggers on Medical Tabular Prediction, by Jiahuan Yan et al.
Small Models are LLM Knowledge Triggers on Medical Tabular Prediction
by Jiahuan Yan, Jintai Chen, Chaowen Hu, Bo Zheng, Yaojun Hu, Jimeng Sun, Jian Wu
First submitted to arxiv on: 3 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Recent advancements in large language models (LLMs) have showcased impressive domain expertise on unstructured textual or multi-modal tasks. However, the application of LLMs on structured tabular data prediction still lags behind, primarily due to numerical insensitivity and modality discrepancy between LLM reasoning and statistical tabular learning. The proposal of SERSAL, a general self-prompting method by synergy learning with small models, aims to enhance LLM tabular prediction in an unsupervised manner. SERSAL utilizes the LLM’s prior outcomes as soft noisy annotations, which are dynamically leveraged to teach a better small student model. The process can be repeatedly applied for continuous progress. Comprehensive experiments on medical domain tabular datasets demonstrate that applying SERSAL to OpenAI GPT reasoning processes attains substantial improvement compared to linguistic prompting methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about using large language models (LLMs) to predict structured data, like numbers and tables. Right now, LLMs are great at understanding text and images, but they struggle with numerical data. The problem is that LLMs don’t understand how to work with numbers well. To fix this, the researchers propose a new method called SERSAL, which helps LLMs learn from small student models. This process can be repeated to make the LLM better and better. When tested on medical data, this method showed great results! |
Keywords
* Artificial intelligence * Gpt * Multi modal * Prompting * Student model * Unsupervised
