Summary of Scaling Laws For Reward Model Overoptimization in Direct Alignment Algorithms, by Rafael Rafailov et al.
Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithmsby Rafael Rafailov, Yaswanth Chittepu, Ryan…
Summary of Adaptive Preference Scaling For Reinforcement Learning with Human Feedback, by Ilgee Hong et al.
Adaptive Preference Scaling for Reinforcement Learning with Human Feedbackby Ilgee Hong, Zichong Li, Alexander Bukharin,…
Summary of Self-improving Robust Preference Optimization, by Eugene Choi et al.
Self-Improving Robust Preference Optimizationby Eugene Choi, Arash Ahmadian, Matthieu Geist, Oilvier Pietquin, Mohammad Gheshlaghi AzarFirst…
Summary of Exploratory Preference Optimization: Harnessing Implicit Q*-approximation For Sample-efficient Rlhf, by Tengyang Xie et al.
Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHFby Tengyang Xie, Dylan J. Foster, Akshay…
Summary of Would I Lie to You? Inference Time Alignment Of Language Models Using Direct Preference Heads, by Avelina Asada Hadji-kyriacou and Ognjen Arandjelovic
Would I Lie To You? Inference Time Alignment of Language Models using Direct Preference Headsby…
Summary of One-shot Safety Alignment For Large Language Models Via Optimal Dualization, by Xinmeng Huang et al.
One-Shot Safety Alignment for Large Language Models via Optimal Dualizationby Xinmeng Huang, Shuo Li, Edgar…
Summary of Value-incentivized Preference Optimization: a Unified Approach to Online and Offline Rlhf, by Shicong Cen et al.
Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHFby Shicong Cen, Jincheng Mei,…
Summary of Symmetric Reinforcement Learning Loss For Robust Learning on Diverse Tasks and Model Scales, by Ju-seung Byun et al.
Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scalesby Ju-Seung Byun,…
Summary of On the Algorithmic Bias Of Aligning Large Language Models with Rlhf: Preference Collapse and Matching Regularization, by Jiancong Xiao et al.
On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching…
Summary of Provably Mitigating Overoptimization in Rlhf: Your Sft Loss Is Implicitly An Adversarial Regularizer, by Zhihan Liu et al.
Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizerby Zhihan Liu,…