Summary of Finding Lottery Tickets in Vision Models Via Data-driven Spectral Foresight Pruning, by Leonardo Iurada et al.

Main: Computer Vision and Pattern Recognition (cs.CV)
Secondary: Artificial Intelligence (cs.AI)

Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning

by Leonardo Iurada, Marco Ciccone, Tatiana Tommasi

First submitted to arxiv on: 3 Jun 2024

GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty	Written by	Summary
High	Paper authors	High Difficulty Summary Read the original abstract here
Medium	GrooveSquid.com (original content)	Medium Difficulty Summary The paper proposes a new pruning algorithm that leverages the Neural Tangent Kernel (NTK) theory to align the training dynamics of sparse networks with those of dense ones. The algorithm, called Path eXclusion (PX), is designed to preserve parameters that mostly influence the NTK’s trace and can find lottery tickets even at high sparsity levels. PX reduces the need for additional training and extracts subnetworks directly usable for several downstream tasks. When applied to pre-trained models, it achieves performance comparable to dense counterparts with significant cost and computational savings.
Low	GrooveSquid.com (original content)	Low Difficulty Summary The paper uses a new way of pruning neural networks before they’re trained. It helps reduce the amount of computation needed and memory used. The authors propose an algorithm called Path eXclusion (PX) that works by preserving important connections in the network. This allows for better performance on different tasks, like image recognition or language translation.