Summary of User-specific Dialogue Generation with User Profile-aware Pre-training Model and Parameter-efficient Fine-tuning, by Atsushi Otsuka and Kazuya Matsuo and Ryo Ishii and Narichika Nomoto and Hiroaki Sugiyama
User-Specific Dialogue Generation with User Profile-Aware Pre-Training Model and Parameter-Efficient Fine-Tuning
by Atsushi Otsuka, Kazuya Matsuo, Ryo Ishii, Narichika Nomoto, Hiroaki Sugiyama
First submitted to arxiv on: 2 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel approach to generating user-specific dialogs by combining parameter-efficient fine-tuning with pre-trained dialogue models. The method leverages user profiles and simple prompts to generate speech that is tailored to individual users. In contrast to previous work on personalized dialogues, this approach focuses on reproducing real-user dialogue beyond persona-based dialogue. Fine-tuning using the target user’s dialogue history is an efficient learning method for a user-specific model, but it can be prone to overfitting and model destruction due to limited data. The proposed method addresses these limitations by adding a small number of parameters to the entire model, making it robust to model destruction. The pre-trained model learns by automatically inferring user profiles from simple prompts, which can generate speech with enhanced knowledge of the user’s profile even with little training data during fine-tuning. Experimental results show that the proposed method outperforms large-language-model utterance generation using prompts containing users’ personal information in terms of reproducibility. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making computers talk like real people. It’s different from other research on this topic because it tries to make the computer generate speech that sounds just like a specific person, rather than just a general type of person (like a “young adult” or “businessperson”). The researchers developed a new way to train the computer model using information about the target user and simple prompts. This approach helps the computer learn quickly and accurately even with limited training data. The results show that this method is better at generating speech that sounds like real people than other methods. |
Keywords
» Artificial intelligence » Fine tuning » Large language model » Overfitting » Parameter efficient
