Summary of Seke: Specialised Experts For Keyword Extraction, by Matej Martinc et al.
SEKE: Specialised Experts for Keyword Extraction
by Matej Martinc, Hanh Thi Hong Tran, Senja Pollak, Boshko Koloski
First submitted to arxiv on: 18 Dec 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel supervised keyword extraction approach called SEKE, which utilizes the mixture of experts (MoE) technique. The MoE framework is based on a learnable routing sub-network that directs information to specialized experts, allowing them to specialize in distinct regions of the input space. SEKE uses DeBERTa as its backbone model and integrates it with a recurrent neural network (RNN) to enable successful extraction even on smaller corpora where specialization is harder due to lack of training data. The MoE framework also provides insights into the inner workings of individual experts, enhancing explainability. The proposed approach achieves state-of-the-art performance compared to strong supervised and unsupervised baselines on multiple English datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper develops a new way to find important words in documents called SEKE (Specialised Experts for Keyword Extraction). It uses a special type of neural network that can focus on different parts of the document. This helps it work well even when there isn’t much training data available. The approach is better than other methods at finding keywords and provides insight into how it makes decisions. The results are very good, beating strong competitors on several English-language datasets. |
Keywords
» Artificial intelligence » Mixture of experts » Neural network » Rnn » Supervised » Unsupervised
