Summary of Advancing Training Efficiency Of Deep Spiking Neural Networks Through Rate-based Backpropagation, by Chengting Yu et al.
Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation
by Chengting Yu, Lei Liu, Gaoang Wang, Erping Li, Aili Wang
First submitted to arxiv on: 15 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Neural and Evolutionary Computing (cs.NE)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel training strategy for deep Spiking Neural Networks (SNNs) called rate-based backpropagation. The authors build upon recent insights that rate-coding is a primary form of information representation in BPTT, and develop a method to exploit this representation to reduce the complexity of SNNs training. The proposed approach minimizes reliance on detailed temporal derivatives by focusing on averaged dynamics, streamlining the computational graph to reduce memory and computational demands. The authors demonstrate the rationality of the gradient approximation through both theoretical analysis and empirical observations. Experimental results on CIFAR-10, CIFAR-100, ImageNet, and CIFAR10-DVS validate that rate-based backpropagation achieves comparable performance to BPTT counterparts, and outperforms state-of-the-art efficient training techniques. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps make deep learning networks run more efficiently by using a new way of training called “rate-based backpropagation”. The authors want to make it easier to train these networks without needing powerful computers. They found that this method works just as well as the old way, and even beats some other efficient methods. This is important because it could help us use these networks in places where we don’t have access to super-powerful computers. |
Keywords
» Artificial intelligence » Backpropagation » Deep learning
