Summary of On the Robustness Of Language Models For Tabular Question Answering, by Kushal Raj Bhandari et al.
On the Robustness of Language Models for Tabular Question Answering
by Kushal Raj Bhandari, Sixue Xing, Soham Dan, Jianxi Gao
First submitted to arxiv on: 18 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Large Language Models (LLMs) have demonstrated impressive capabilities in various text comprehension tasks. Our study investigates the impact of in-context learning, model scale, instruction tuning, and domain biases on Tabular Question Answering (TQA). We evaluate the robustness of LLMs on Wikipedia-based WTQ and financial report-based TAT-QA datasets. The results show that instructions significantly improve performance, with recent models like Llama3 exhibiting greater robustness over earlier versions. However, data contamination and practical reliability issues persist, especially with WTQ. Our findings highlight the need for improved methodologies to develop more reliable LLMs for table comprehension. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at how well Large Language Models (LLMs) can understand tables. We tested these models on two different types of datasets: one based on Wikipedia and another based on financial reports. The results show that giving the models specific instructions helps them do better, especially if they’re newer models like Llama3. However, we also found some problems with how the data is used and handled. This means we need to come up with new ways to improve these models so they can be more reliable. |
Keywords
» Artificial intelligence » Instruction tuning » Question answering
