Summary of Unimem: Towards a Unified View Of Long-context Large Language Models, by Junjie Fang et al.
UniMem: Towards a Unified View of Long-Context Large Language Modelsby Junjie Fang, Likai Tang, Hongzhe…
Summary of Fractal Patterns May Illuminate the Success Of Next-token Prediction, by Ibrahim Alabdulmohsin et al.
Fractal Patterns May Illuminate the Success of Next-Token Predictionby Ibrahim Alabdulmohsin, Vinh Q. Tran, Mostafa…
Summary of Infini-gram: Scaling Unbounded N-gram Language Models to a Trillion Tokens, by Jiacheng Liu et al.
Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokensby Jiacheng Liu, Sewon Min, Luke…
Summary of Pushing the Envelope Of Low-bit Llm Via Dynamic Error Compensation, by Yeonhong Park et al.
Pushing the Envelope of Low-Bit LLM via Dynamic Error Compensationby Yeonhong Park, Jake Hyun, Hojoon…
Summary of Deliberation in Latent Space Via Differentiable Cache Augmentation, by Luyang Liu et al.
Deliberation in Latent Space via Differentiable Cache Augmentationby Luyang Liu, Jonas Pfeiffer, Jiaxing Wu, Jun…
Summary of Mixllm: Llm Quantization with Global Mixed-precision Between Output-features and Highly-efficient System Design, by Zhen Zheng et al.
MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designby Zhen Zheng, Xiaonan…
Summary of Resq: Mixed-precision Quantization Of Large Language Models with Low-rank Residuals, by Utkarsh Saxena et al.
ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualsby Utkarsh Saxena, Sayeh Sharify, Kaushik…
Summary of Swan: Sgd with Normalization and Whitening Enables Stateless Llm Training, by Chao Ma et al.
SWAN: SGD with Normalization and Whitening Enables Stateless LLM Trainingby Chao Ma, Wenbo Gong, Meyer…
Summary of Model-diff: a Tool For Comparative Study Of Language Models in the Input Space, by Weitang Liu et al.
Model-diff: A Tool for Comparative Study of Language Models in the Input Spaceby Weitang Liu,…
Summary of Wonderful Matrices: Combining For a More Efficient and Effective Foundation Model Architecture, by Jingze Shi and Bingheng Wu
Wonderful Matrices: Combining for a More Efficient and Effective Foundation Model Architectureby Jingze Shi, Bingheng…