Summary of Fintextqa: a Dataset For Long-form Financial Question Answering, by Jian Chen et al.
FinTextQA: A Dataset for Long-form Financial Question Answering
by Jian Chen, Peilin Zhou, Yining Hua, Yingxin Loh, Kehui Chen, Ziyuan Li, Bing Zhu, Junwei Liang
First submitted to arxiv on: 16 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel financial question answering dataset called FinTextQA is introduced, consisting of 1,262 high-quality QA pairs extracted from finance textbooks and government agency websites. The dataset aims to address the lack of scope diversity and question complexity in current financial QA datasets. A Retrieval-Augmented Generation-based system is developed for long-form question answering (LFQA) in finance, comprising an embedder, retriever, reranker, and generator. The system’s performance is benchmarked using a multi-faceted evaluation approach under noisy conditions. Results show that the Baichuan2-7B generator competes closely with GPT-3.5-turbo in accuracy score, while the most effective system configuration involves setting specific components. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary FinTextQA is a new dataset for asking and answering questions about finance. Right now, it’s hard to test how well computer systems can answer financial questions because there aren’t enough examples of different kinds of questions and answers. This paper fixes that by creating FinTextQA, which has 1,262 high-quality questions and answers taken from finance textbooks and government agency websites. The authors also built a special system for answering these questions, using a combination of algorithms to get the best answer. They tested how well this system worked under different conditions, and found that it does really well, especially with a certain type of generator. |
Keywords
» Artificial intelligence » Gpt » Question answering » Retrieval augmented generation
