BATON: Enhancing Batch-wise Inference Efficiency for Large Language Models via Dynamic Re-batchingby Peizhuang Cong, Qizhi…
Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Databy Anup Shirgaonkar,…
3D Shape Completion with Test-Time Trainingby Michael Schopf-Kuester, Zorah Lähner, Michael MoellerFirst submitted to arxiv…
Scaling up Masked Diffusion Models on Textby Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang,…
KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharingby Yifei Yang, Zouying Cao, Qiguang Chen,…
Fast Inference for Augmented Large Language Modelsby Rana Shahout, Cong Liang, Shiji Xin, Qianru Lao,…
LEGO: Language Model Building Blocksby Shrenik Bhansali, Alwin Jin, Tyler Lizzo, Larry HeckFirst submitted to…
CoreInfer: Accelerating Large Language Model Inference with Semantics-Inspired Adaptive Sparse Activationby Qinsi Wang, Saeed Vahidian,…
AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower…
R2Gen-Mamba: A Selective State Space Model for Radiology Report Generationby Yongheng Sun, Yueh Z. Lee,…