Mnemosyne: Parallelization Strategies for Efficiently Serving Multi-Million Context Length LLM Inference Requests Without Approximationsby Amey…
INT-FlashAttention: Enabling Flash Attention for INT8 Quantizationby Shimao Chen, Zirui Liu, Zhiying Wu, Ce Zheng,…
CNN Mixture-of-Depthsby Rinor Cakaj, Jens Mehnert, Bin YangFirst submitted to arxiv on: 25 Sep 2024CategoriesMain:…
Accelerating TinyML Inference on Microcontrollers through Approximate Kernelsby Giorgos Armeniakos, Georgios Mentzos, Dimitrios SoudrisFirst submitted…
Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observabilityby Carlos E. Luis,…
AlignedKV: Reducing Memory Access of KV-Cache with Precision-Aligned Quantizationby Yifan Tan, Haoze Wang, Chao Yan,…
Functional Stochastic Gradient MCMC for Bayesian Neural Networksby Mengjing Wu, Junyu Xuan, Jie LuFirst submitted…
A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmsby Ruihao Gong, Yifu Ding,…
Towards Representation Learning for Weighting Problems in Design-Based Causal Inferenceby Oscar Clivio, Avi Feller, Chris…
A QoE-Aware Split Inference Accelerating Algorithm for NOMA-based Edge Intelligenceby Xin Yuan, Ning Li, Quan…