Summary of Skim: Any-bit Quantization Pushing the Limits Of Post-training Quantization, by Runsheng Bai et al.
SKIM: Any-bit Quantization Pushing The Limits of Post-Training Quantizationby Runsheng Bai, Bo Liu, Qiang LiuFirst…
Summary of Dfrot: Achieving Outlier-free and Massive Activation-free For Rotated Llms with Refined Rotation, by Jingyang Xiang et al.
DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotationby Jingyang Xiang, Sai…
Summary of Coap: Memory-efficient Training with Correlation-aware Gradient Projection, by Jinqi Xiao et al.
COAP: Memory-Efficient Training with Correlation-Aware Gradient Projectionby Jinqi Xiao, Shen Sang, Tiancheng Zhi, Jing Liu,…
Summary of Attamba: Attending to Multi-token States, by Yash Akhauri et al.
Attamba: Attending To Multi-Token Statesby Yash Akhauri, Safeen Huda, Mohamed S. AbdelfattahFirst submitted to arxiv…
Summary of Pushing the Limits Of Large Language Model Quantization Via the Linearity Theorem, by Vladimir Malinovskii et al.
Pushing the Limits of Large Language Model Quantization via the Linearity Theoremby Vladimir Malinovskii, Andrei…
Summary of Clover: Cross-layer Orthogonal Vectors Pruning and Fine-tuning, by Fanxu Meng et al.
CLOVER: Cross-Layer Orthogonal Vectors Pruning and Fine-Tuningby Fanxu Meng, Pingzhi Tang, Fan jiang, Muhan ZhangFirst…
Summary of Is Training Data Quality or Quantity More Impactful to Small Language Model Performance?, by Aryan Sajith et al.
Is Training Data Quality or Quantity More Impactful to Small Language Model Performance?by Aryan Sajith,…
Summary of Fusegpt: Learnable Layers Fusion Of Generative Pre-trained Transformers, by Zehua Pei et al.
FuseGPT: Learnable Layers Fusion of Generative Pre-trained Transformersby Zehua Pei, Hui-Ling Zhen, Xianzhi Yu, Sinno…
Summary of Deriving Activation Functions Using Integration, by Allen Hao Huang et al.
Deriving Activation Functions Using Integrationby Allen Hao Huang, Imanol SchlagFirst submitted to arxiv on: 20…
Summary of Efficient Adaptive Optimization Via Subset-norm and Subspace-momentum: Fast, Memory-reduced Training with Convergence Guarantees, by Thien Hang Nguyen et al.
Efficient Adaptive Optimization via Subset-Norm and Subspace-Momentum: Fast, Memory-Reduced Training with Convergence Guaranteesby Thien Hang…