Summary of Mechanistic Design and Scaling Of Hybrid Architectures, by Michael Poli et al.
Mechanistic Design and Scaling of Hybrid Architecturesby Michael Poli, Armin W Thomas, Eric Nguyen, Pragaash…
Summary of Language Models Scale Reliably with Over-training and on Downstream Tasks, by Samir Yitzhak Gadre and Georgios Smyrnis and Vaishaal Shankar and Suchin Gururangan and Mitchell Wortsman and Rulin Shao and Jean Mercat and Alex Fang and Jeffrey Li and Sedrick Keh and Rui Xin and Marianna Nezhurina and Igor Vasiljevic and Jenia Jitsev and Luca Soldaini and Alexandros G. Dimakis and Gabriel Ilharco and Pang Wei Koh and Shuran Song and Thomas Kollar and Yair Carmon and Achal Dave and Reinhard Heckel and Niklas Muennighoff and Ludwig Schmidt
Language models scale reliably with over-training and on downstream tasksby Samir Yitzhak Gadre, Georgios Smyrnis,…
Summary of Mixtures Of Experts Unlock Parameter Scaling For Deep Rl, by Johan Obando-ceron et al.
Mixtures of Experts Unlock Parameter Scaling for Deep RLby Johan Obando-Ceron, Ghada Sokar, Timon Willi,…
Summary of Scaling Laws For Fine-grained Mixture Of Experts, by Jakub Krajewski et al.
Scaling Laws for Fine-Grained Mixture of Expertsby Jakub Krajewski, Jan Ludziejewski, Kamil Adamczewski, Maciej Pióro,…
Summary of Model Collapse Demystified: the Case Of Regression, by Elvis Dohmatob et al.
Model Collapse Demystified: The Case of Regressionby Elvis Dohmatob, Yunzhen Feng, Julia KempeFirst submitted to…
Summary of A Tale Of Tails: Model Collapse As a Change Of Scaling Laws, by Elvis Dohmatob et al.
A Tale of Tails: Model Collapse as a Change of Scaling Lawsby Elvis Dohmatob, Yunzhen…
Summary of A Resource Model For Neural Scaling Law, by Jinyeop Song et al.
A Resource Model For Neural Scaling Lawby Jinyeop Song, Ziming Liu, Max Tegmark, Jeff GoreFirst…
Summary of Scaling Laws For Downstream Task Performance in Machine Translation, by Berivan Isik et al.
Scaling Laws for Downstream Task Performance in Machine Translationby Berivan Isik, Natalia Ponomareva, Hussein Hazimeh,…
Summary of Position: Graph Foundation Models Are Already Here, by Haitao Mao et al.
Position: Graph Foundation Models are Already Hereby Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao,…
Summary of Towards Neural Scaling Laws on Graphs, by Jingzhe Liu et al.
Towards Neural Scaling Laws on Graphsby Jingzhe Liu, Haitao Mao, Zhikai Chen, Tong Zhao, Neil…