Summary of A Comparison Of Llm Finetuning Methods & Evaluation Metrics with Travel Chatbot Use Case, by Sonia Meyer et al.
A Comparison of LLM Finetuning Methods & Evaluation Metrics with Travel Chatbot Use Case
by Sonia Meyer, Shreya Singh, Bertha Tam, Christopher Ton, Angel Ren
First submitted to arxiv on: 7 Aug 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper compares various fine-tuning methods for large language models (LLMs), including Quantized Low Rank Adapter (QLoRA), Retrieval Augmented fine-tuning (RAFT), and Reinforcement Learning from Human Feedback (RLHF). The study also evaluates different LLM evaluation metrics, such as End-to-End benchmark method “Golden Answers”, traditional natural language processing (NLP) metrics, RAG Assessment (Ragas), OpenAI GPT-4 evaluation metrics, and human evaluation. Two pretrained LLMs, LLaMa 2 7B and Mistral 7B, were used for fine-tuning and evaluation. The results show that the best model according to human evaluation was Mistral RAFT, which underwent RLHF training pipeline. The study highlights the importance of keeping humans in the loop for evaluation and suggests that traditional NLP metrics are insufficient. Key findings include: LLaMa generally outperformed Mistral, RAFT outperforms QLoRA, but still needs postprocessing, and RLHF improves model performance significantly. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper compares different ways to fine-tune big language models (LLMs). It also looks at how to evaluate these models. The study uses two types of LLMs: LLaMa 2 7B and Mistral 7B. It finds that one method, called RAFT on Mistral 7B, is the best according to human evaluation. The research shows that it’s important to involve humans in evaluating AI models because traditional methods don’t work well. |
Keywords
» Artificial intelligence » Fine tuning » Gpt » Llama » Natural language processing » Nlp » Rag » Reinforcement learning from human feedback » Rlhf
