Summary of Mlem: Generative and Contrastive Learning As Distinct Modalities For Event Sequences, by Viktor Moskvoretskii et al.
MLEM: Generative and Contrastive Learning as Distinct Modalities for Event Sequences
by Viktor Moskvoretskii, Dmitry Osin, Egor Shvetsov, Igor Udovichenko, Maxim Zhelnin, Andrey Dukhovny, Anna Zhimerikina, Evgeny Burnaev
First submitted to arxiv on: 29 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores the application of self-supervised learning techniques for event sequences in various domains like banking, e-commerce, and healthcare. It compares previously identified best-performing methods to determine the most suitable approach. The study finds that neither contrastive nor generative method is superior and highlights the potential benefits of combining both methods. To develop a hybrid model, the authors adapt a baseline model from another domain but observe its underperformance. They then develop a novel method called Multimodal-Learning Event Model (MLEM), which treats contrastive learning and generative modeling as distinct yet complementary modalities, aligning their embeddings. The results demonstrate that combining contrastive and generative approaches into one procedure with MLEM achieves superior performance across multiple metrics. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at using artificial intelligence to help machines learn from events like transactions or doctor visits. Right now, there’s not much research on this topic, so the authors compare some existing methods to see which one works best. They find that none of them are perfect, but combining two different approaches can actually make it better. To try and make a hybrid model work, they take an idea from another field and modify it, but it doesn’t quite work as planned. So, they come up with a new approach called MLEM, which combines two types of learning in a way that makes sense for event sequences. The results show that this combination is actually pretty effective. |
