Direct Nash Optimization: Teaching Language Models to Self-Improve with General Preferencesby Corby Rosset, Ching-An Cheng,…
Fine-Tuning Language Models with Reward Learning on Policyby Hao Lang, Fei Huang, Yongbin LiFirst submitted…
The N+ Implementation Details of RLHF with PPO: A Case Study on TL;DR Summarizationby Shengyi…
RewardBench: Evaluating Reward Models for Language Modelingby Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda,…
Parameter Efficient Reinforcement Learning from Human Feedbackby Hakim Sidahmed, Samrat Phatale, Alex Hutcheson, Zhuonan Lin,…
HRLAIF: Improvements in Helpfulness and Harmlessness in Open-domain Reinforcement Learning From AI Feedbackby Ang Li,…
ALaRM: Align Language Models via Hierarchical Rewards Modelingby Yuhang Lai, Siyuan Wang, Shujun Liu, Xuanjing…
Overcoming Reward Overoptimization via Adversarial Policy Optimization with Lightweight Uncertainty Estimationby Xiaoying Zhang, Jean-Francois Ton,…
Provable Multi-Party Reinforcement Learning with Diverse Human Feedbackby Huiying Zhong, Zhun Deng, Weijie J. Su,…
Proxy-RLHF: Decoupling Generation and Alignment in Large Language Model with Proxyby Yu Zhu, Chuxiong Sun,…