Summary of Subjective-qa: Measuring Subjectivity in Earnings Call Transcripts’ Qa Through Six-dimensional Feature Analysis, by Huzaifa Pardawala et al.
SubjECTive-QA: Measuring Subjectivity in Earnings Call Transcripts’ QA Through Six-Dimensional Feature Analysis
by Huzaifa Pardawala, Siddhant Sukhani, Agam Shah, Veer Kejriwal, Abhishek Pillai, Rohan Bhasin, Andrew DiBiasio, Tarun Mandapati, Dhruv Adha, Sudheer Chava
First submitted to arxiv on: 28 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces SubjECTive-QA, a manually annotated dataset to tackle subjective features in Question-Answer (QA) settings across multiple domains. The dataset comprises 49,446 annotations for long-form QA pairs across six features: Assertive, Cautious, Optimistic, Specific, Clear, and Relevant. These features reflect the tone of answers provided during QA sessions. Models like RoBERTa-base and Llama-3-70b-Chat were evaluated on this dataset, with better performance on features with lower subjectivity (e.g., Relevant and Clear) but struggling with higher-subjectivity features (e.g., Specific and Assertive). The study also demonstrates the broader applicability of SubjECTive-QA across domains, including White House Press Briefings and Gaggles. This research contributes to the development of more effective QA systems that can better handle subjective responses. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a special dataset to help computers understand when answers are not just correct but also clear, relevant, and written in a way that makes sense. The dataset includes many examples of questions and answers from company representatives during financial press conferences. Researchers tested different computer models on this dataset and found that some models were better at understanding certain types of answers than others. This study can help improve the accuracy of computer-based question-answering systems. |
Keywords
» Artificial intelligence » Llama » Question answering
