Summary of Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era Of Large Language Models, by Shubham Kumar Nigam et al.
Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language Models
by Shubham Kumar Nigam, Aniket Deroy, Subhankar Maity, Arnab Bhattacharya
First submitted to arxiv on: 14 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This study leverages transformer-based models like InLegalBERT, BERT, XLNet, Llama-2, and GPT-3.5 Turbo, alongside large language models (LLMs) such as Llama-2 and GPT-3.5 Turbo to predict judgments in a realistic Indian scenario. The research simulates the real-world scenario where judges make decisions without hindsight, using only information available at the time, including case facts, statutes, precedents, and arguments. For transformer models, hierarchical transformers and fact summarization are experimented with to optimize input. LLMs are found to excel in realistic scenarios, particularly GPT-3.5 Turbo, demonstrating robust performance in judgment prediction. Incorporating legal information like statutes and precedents significantly improves the outcome. The LLMs also provide explanations for their predictions. Two human evaluation metrics, Clarity and Linking, are introduced to assess the quality of these predictions and explanations. Automatic and human evaluations indicate that while LLMs have made progress, they still fall short of expert-level performance in judgment prediction and explanation tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study is about using special AI models called transformers to predict how a judge will make a decision based on the facts of a case. The researchers wanted to see if these models could make good decisions without having all the information – just like judges do in real life. They tested different types of transformer models and found that some did better than others. One model, GPT-3.5 Turbo, was especially good at making predictions. The researchers also found that including more legal information, like laws and past court cases, made their predictions even better. Another important part of this study is that the AI models can explain why they make certain decisions. |
Keywords
» Artificial intelligence » Bert » Gpt » Llama » Summarization » Transformer
