Summary of Forml: a Riemannian Hessian-free Method For Meta-learning on Stiefel Manifolds, by Hadi Tabealhojeh et al.
FORML: A Riemannian Hessian-free Method for Meta-learning on Stiefel Manifolds
by Hadi Tabealhojeh, Soumava Kumar Roy, Peyman Adibi, Hossein Karshenas
First submitted to arxiv on: 28 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a Hessian-free approach for meta-learning in Riemannian space, which significantly reduces computational load and memory footprint. The authors introduce a Stiefel fully-connected layer that enforces orthogonality constraint on the parameters of the last classification layer, strengthening representation reuse in gradient-based meta-learning methods. Experimental results across various few-shot learning datasets demonstrate the superiority of the proposed method compared to state-of-the-art methods, including MAML and its Euclidean counterpart. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Meta-learning is a way for AI models to learn from other AI models. This paper makes it faster and more efficient by using a new approach that doesn’t need to compute complicated second-order derivatives. The authors also introduce a special type of neural network layer that helps the model learn better representations. They tested their method on several datasets and found it outperformed other state-of-the-art methods. |
Keywords
* Artificial intelligence * Classification * Few shot * Meta learning * Neural network
