Would I Lie To You? Inference Time Alignment of Language Models using Direct Preference Headsby…
One-Shot Safety Alignment for Large Language Models via Optimal Dualizationby Xinmeng Huang, Shuo Li, Edgar…
Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Groundingby Shenghuan Sun, Alexander Schubert, Gregory M. Goldgof,…
Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHFby Shicong Cen, Jincheng Mei,…
Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignmentby Keming Lu, Bowen Yu,…
Symmetric Reinforcement Learning Loss for Robust Learning on Diverse Tasks and Model Scalesby Ju-Seung Byun,…
RLSF: Reinforcement Learning via Symbolic Feedbackby Piyush Jha, Prithwish Jana, Pranavkrishna Suresh, Arnav Arora, Vijay…
Provably Mitigating Overoptimization in RLHF: Your SFT Loss is Implicitly an Adversarial Regularizerby Zhihan Liu,…
On the Algorithmic Bias of Aligning Large Language Models with RLHF: Preference Collapse and Matching…
Direct Preference Optimization With Unobserved Preference Heterogeneityby Keertana Chidambaram, Karthik Vinay Seetharaman, Vasilis SyrgkanisFirst submitted…