Probabilities of Chat LLMs Are Miscalibrated but Still Predict Correctness on Multiple-Choice Q&Aby Benjamin Plaut,…
In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitznessby Liam Collins, Advait Parulekar, Aryan…
Measuring and Controlling Instruction (In)Stability in Language Model Dialogsby Kenneth Li, Tianle Liu, Naomi Bashkansky,…
Rethinking Self-Distillation: Label Averaging and Enhanced Soft Label Refinement with Partial Labelsby Hyeonsu Jeong, Hye…
Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Banditby Stefana Anita,…
RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationby Zhikai Li, Xuewen…
The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicryby Michael Zhang, Kush Bhatia,…
On Least Square Estimation in Softmax Gating Mixture of Expertsby Huy Nguyen, Nhat Ho, Alessandro…
What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinementby Xisen Jin, Xiang…
Ultra Fast Transformers on FPGAs for Particle Physics Experimentsby Zhixing Jiang, Dennis Yin, Elham E…