Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inferenceby Qining Zhang, Lei…
Reward-Robust RLHF in LLMsby Yuzi Yan, Xingzhou Lou, Jialian Li, Yiping Zhang, Jian Xie, Chao…
RLHFuse: Efficient RLHF Training for Large Language Models with Inter- and Intra-Stage Fusionby Yinmin Zhong,…
From Lists to Emojis: How Format Bias Affects Model Alignmentby Xuanchang Zhang, Wei Xiong, Lichang…
Quantile Regression for Distributional Reward Models in RLHFby Nicolai DorkaFirst submitted to arxiv on: 16…
Towards Data-Centric RLHF: Simple Metrics for Preference Dataset Comparisonby Judy Hanwen Shen, Archit Sharma, Jun…
Scores as Actions: a framework of fine-tuning diffusion models by continuous-time reinforcement learningby Hanyang Zhao,…
Policy Filtration in RLHF to Fine-Tune LLM for Code Generationby Wei Shen, Chuheng ZhangFirst submitted…
Length Desensitization in Direct Preference Optimizationby Wei Liu, Yang Bai, Chengcheng Han, Rongxiang Weng, Jun…
On the Limited Generalization Capability of the Implicit Reward Model Induced by Direct Preference Optimizationby…