Summary of Cprm: a Llm-based Continual Pre-training Framework For Relevance Modeling in Commercial Search, by Kaixin Wu et al.
CPRM: A LLM-based Continual Pre-training Framework for Relevance Modeling in Commercial Search
by Kaixin Wu, Yixin Ji, Zeyuan Chen, Qiang Wang, Cunxiang Wang, Hong Liu, Baijun Ji, Jia Xu, Zhongyi Liu, Jinjie Gu, Yuan Zhou, Linjian Mo
First submitted to arxiv on: 2 Dec 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a framework called CPRM (Continual Pre-training for Relevance Modeling) to continually pre-train large language models (LLMs) for relevance modeling in commercial search engines. LLMs have shown remarkable achievements in natural language processing tasks, but lack domain-specific knowledge and underutilize structured item text. The proposed framework includes three modules: employing queries and multi-field items to jointly pre-train for enhancing domain knowledge, applying in-context pre-training, and conducting reading comprehension on items to produce associated domain knowledge. The results demonstrate convincing performance compared to strong baselines. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making search engines better by training special language models called LLMs. These models are good at understanding natural language, but they don’t know much about specific topics like cars or sports. The authors propose a new way to train these models that uses both the questions people ask and the information available on websites. This approach helps the models learn more about different topics and understand how they relate to each other. |
Keywords
* Artificial intelligence * Natural language processing
