Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensemblingby Cristian Rodriguez-Opazo, Ehsan Abbasnejad,…
On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capabilityby Chenyu Zheng, Wei Huang, Rongzhen Wang,…
Unsupervised Generative Feature Transformation via Graph Contrastive Pre-training and Multi-objective Fine-tuningby Wangyang Ying, Dongjie Wang,…
Zamba: A Compact 7B SSM Hybrid Modelby Paolo Glorioso, Quentin Anthony, Yury Tokpanov, James Whittington,…
LoQT: Low-Rank Adapters for Quantized Pretrainingby Sebastian Loeschcke, Mads Toftrup, Michael J. Kastoryano, Serge Belongie,…
SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMsby Mohammad Mozaffari, Amir Yazdanbakhsh, Zhao…
Polyp Segmentation Generalisability of Pretrained Backbonesby Edward Sanderson, Bogdan J. MatuszewskiFirst submitted to arxiv on:…
BiMix: A Bivariate Data Mixing Law for Language Model Pretrainingby Ce Ge, Zhijian Ma, Daoyuan…
Analysis of Atom-level pretraining with Quantum Mechanics (QM) data for Graph Neural Networks Molecular property…
Mixture of Experts Meets Prompt-Based Continual Learningby Minh Le, An Nguyen, Huy Nguyen, Trang Nguyen,…