Summary of Practiq: a Practical Conversational Text-to-sql Dataset with Ambiguous and Unanswerable Queries, by Mingwen Dong et al.
PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries
by Mingwen Dong, Nischal Ashok Kumar, Yiqun Hu, Anuj Chauhan, Chung-Wei Hang, Shuaichen Chang, Lin Pan, Wuwei Lan, Henghui Zhu, Jiarong Jiang, Patrick Ng, Zhiguo Wang
First submitted to arxiv on: 14 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces PRACTIQ, a new dataset and system for text-to-SQL queries that handles ambiguous and unanswerable questions inspired by real-world user interactions. The authors identify four categories of ambiguous and unanswerable questions, generating conversations with multiple turns to clarify the user’s intent. For ambiguous queries, they also develop SQL responses that consider multiple aspects of ambiguity. The authors benchmark their approach using large language model (LLM)-based baselines and report that state-of-the-art systems struggle to effectively handle these challenging question types. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way for computers to understand and respond to user questions that are tricky or unclear. Right now, most computer systems can only answer simple questions with clear answers. But in real life, people often ask confusing or ambiguous questions that need more clarification. The authors of this paper wanted to create a dataset and system that could handle these kinds of questions. They identified four types of ambiguous questions and developed conversations between users and computers to clarify what the user means. They also created SQL responses that take into account multiple aspects of ambiguity. The results show that current computer systems are not very good at handling ambiguous or unclear questions. |
Keywords
» Artificial intelligence » Large language model
