Summary of Re-visiting Skip-gram Negative Sampling: Dimension Regularization For More Efficient Dissimilarity Preservation in Graph Embeddings, by David Liu et al.
Re-visiting Skip-Gram Negative Sampling: Dimension Regularization for More Efficient Dissimilarity Preservation in Graph Embeddings
by David Liu, Arjun Seshadri, Tina Eliassi-Rad, Johan Ugander
First submitted to arxiv on: 30 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Social and Information Networks (cs.SI); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel connection is established between Skip-Gram Negative Sampling (SGNS) and dimension regularization, showing that node-wise repulsion can be viewed as an approximate re-centering of node embedding dimensions. This finding extends insights from self-supervised learning to the skip-gram model. The authors propose an algorithm augmentation framework that leverages this observation to accelerate any existing algorithm using SGNS, prioritizing node attraction and replacing SGNS with dimension regularization. Instantiations for LINE and node2vec are demonstrated, preserving downstream performance while significantly improving efficiency. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at how a popular way of dealing with big graphs works. It’s called Skip-Gram Negative Sampling (SGNS) and it helps by pushing node embeddings away from each other when they’re different. The researchers found that this process is actually like re-centering the space where these nodes live, which makes things more efficient. They came up with a way to speed up existing algorithms using SGNS, which works just as well but is faster. |
Keywords
» Artificial intelligence » Embedding » Regularization » Self supervised
