Summary of Relu-kan: New Kolmogorov-arnold Networks That Only Need Matrix Addition, Dot Multiplication, and Relu, by Qi Qiu et al.
ReLU-KAN: New Kolmogorov-Arnold Networks that Only Need Matrix Addition, Dot Multiplication, and ReLUby Qi Qiu,…
Summary of Activation-descent Regularization For Input Optimization Of Relu Networks, by Hongzhan Yu et al.
Activation-Descent Regularization for Input Optimization of ReLU Networksby Hongzhan Yu, Sicun GaoFirst submitted to arxiv…
Summary of Optimizing Cnn-bigru Performance: Mish Activation and Comparative Analysis with Relu, by Asmaa Benchama and Khalid Zebbara
Optimizing cnn-Bigru performance: Mish activation and comparative analysis with Reluby Asmaa Benchama, Khalid ZebbaraFirst submitted…
Summary of Large Deviations Of Gaussian Neural Networks with Relu Activation, by Quirin Vogel
Large Deviations of Gaussian Neural Networks with ReLU activationby Quirin VogelFirst submitted to arxiv on:…
Summary of Novel Kernel Models and Exact Representor Theory For Neural Networks Beyond the Over-parameterized Regime, by Alistair Shilton et al.
Novel Kernel Models and Exact Representor Theory for Neural Networks Beyond the Over-Parameterized Regimeby Alistair…
Summary of Adversarial Training Of Two-layer Polynomial and Relu Activation Networks Via Convex Optimization, by Daniel Kuelbs et al.
Adversarial Training of Two-Layer Polynomial and ReLU Activation Networks via Convex Optimizationby Daniel Kuelbs, Sanjay…
Summary of Sigmoid Gating Is More Sample Efficient Than Softmax Gating in Mixture Of Experts, by Huy Nguyen et al.
Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Expertsby Huy Nguyen,…
Summary of Keep the Momentum: Conservation Laws Beyond Euclidean Gradient Flows, by Sibylle Marcotte et al.
Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flowsby Sibylle Marcotte, Rémi Gribonval, Gabriel PeyréFirst…
Summary of A Method on Searching Better Activation Functions, by Haoyuan Sun et al.
A Method on Searching Better Activation Functionsby Haoyuan Sun, Zihao Wu, Bo Xia, Pu Chang,…
Summary of Using Degeneracy in the Loss Landscape For Mechanistic Interpretability, by Lucius Bushnaq et al.
Using Degeneracy in the Loss Landscape for Mechanistic Interpretabilityby Lucius Bushnaq, Jake Mendel, Stefan Heimersheim,…