Summary of Fome: a Foundation Model For Eeg Using Adaptive Temporal-lateral Attention Scaling, by Enze Shi et al.
FoME: A Foundation Model for EEG using Adaptive Temporal-Lateral Attention Scaling
by Enze Shi, Kui Zhao, Qilong Yuan, Jiaqi Wang, Huawen Hu, Sigang Yu, Shu Zhang
First submitted to arxiv on: 19 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Signal Processing (eess.SP)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed FoME (Foundation Model for EEG) is a novel approach that addresses the challenges of signal heterogeneity, low signal-to-noise ratios, and limited labeled datasets in electroencephalography (EEG). It uses adaptive temporal-lateral attention scaling to capture complex temporal and spectral EEG dynamics. The model is pre-trained on a diverse 1.7TB dataset of scalp and intracranial EEG recordings, comprising 745M parameters trained for 1,096k steps. FoME introduces two key innovations: time-frequency fusion embedding technique and adaptive time-lateral attention scaling (ATLAS) mechanism. These components synergistically enable the model to adapt to varying patterns across diverse data streams and facilitate robust multi-channel modeling. Evaluations across four downstream tasks demonstrate FoME’s superior performance in classification and forecasting applications, consistently achieving state-of-the-art results. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary FoME is a new way to analyze brain activity measured by EEG. It helps solve some big problems with this type of measurement. The model learns from a huge dataset of different types of EEG recordings and uses special techniques to understand complex patterns in the data. This makes it really good at predicting what will happen next, which is important for things like diagnosing diseases or controlling prosthetic limbs. FoME can even help people communicate with computers just by thinking. |
Keywords
» Artificial intelligence » Attention » Classification » Embedding
