Summary of Distributionally Robust Self-supervised Learning For Tabular Data, by Shantanu Ghosh et al.
Distributionally robust self-supervised learning for tabular data
by Shantanu Ghosh, Tiankang Xie, Mikhail Kuznetsov
First submitted to arxiv on: 11 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper addresses the problem of learning robust representation in tabular data during self-supervised pre-training, particularly in the presence of error slices. Empirical Risk Minimization (ERM) models often exhibit systematic errors on specific subpopulations of tabular data, which can be detrimental to overall generalization performance. The authors develop a framework that utilizes an encoder-decoder model trained with Masked Language Modeling (MLM) loss to learn robust latent representations. They employ the Just Train Twice (JTT) and Deep Feature Reweighting (DFR) methods during pre-training to fine-tune ERM-pretrained models, creating balanced datasets for specific categorical features and up-weighting error-prone samples. This approach results in specialized models for each feature, which are then combined using an ensemble method to enhance downstream classification performance. The authors demonstrate the efficacy of their approach through extensive experiments across various datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine you’re trying to train a computer model to make predictions about different groups of people based on data. But what if this model does really badly for certain groups? This is called an “error slice” and it’s a big problem in machine learning. Researchers have developed a new way to teach models how to be fair and accurate across all groups, even when they’re dealing with tricky data. They use a special kind of training that helps the model learn from its mistakes and make better predictions. This approach is really important because it can help us create more reliable and trustworthy AI systems. |
Keywords
» Artificial intelligence » Classification » Encoder decoder » Generalization » Machine learning » Self supervised
