Summary of Transformer-based Joint Modelling For Automatic Essay Scoring and Off-topic Detection, by Sourya Dipta Das et al.
Transformer-based Joint Modelling for Automatic Essay Scoring and Off-Topic Detection
by Sourya Dipta Das, Yash Vadi, Kuldeep Yadav
First submitted to arxiv on: 24 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Automated Open Essay Scoring (AOES) model uses a novel topic regularization module (TRM) attached to a transformer model, trained using a hybrid loss function. The model jointly scores essays and detects off-topic responses, outperforming baseline methods on two essay-scoring datasets in both tasks. Experimental evaluations show the method’s robustness against human-level perturbations. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Automated Essay Scoring systems are popular for grading, but they often struggle to detect irrelevant responses. This paper proposes a new way to score essays and detect when answers don’t match the question. The system uses a special module that helps the model focus on relevant topics. It also uses a unique loss function during training. The method was tested on two sets of essay-scoring data and outperformed earlier methods in both scoring and detecting off-topic responses. |
Keywords
* Artificial intelligence * Loss function * Regularization * Transformer
