Summary of Uncertainty-aware Self-training with Expectation Maximization Basis Transformation, by Zijia Wang et al.
Uncertainty-aware self-training with expectation maximization basis transformation
by Zijia Wang, Wenbin Yang, Zhisong Liu, Zhen Jia
First submitted to arxiv on: 2 May 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Self-training is a powerful approach to deep learning, where a pseudo-label for modeling is found. However, previous self-training algorithms suffer from over-confidence due to hard labels, even confidence-related regularizers cannot fully address uncertainty. We propose a new self-training framework that combines uncertainty information from both the model and dataset. Specifically, we use Expectation-Maximization (EM) to smooth labels and estimate uncertainty. A basis extraction network is designed to obtain initial bases from datasets, which are filtered based on uncertainty and transformed into real hard labels for iterative updating of models and bases. Our methods demonstrate advantages among confidence-aware self-training algorithms with 1-3 percentage improvement on different datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Self-training in deep learning finds a pseudo-label for modeling. But earlier approaches had problems with over-confidence from using hard labels, even extra tools didn’t fully fix uncertainty issues. We created a new way to combine model and dataset uncertainties. First, we smoothed the labels and estimated uncertainty using Expectation-Maximization (EM). Next, we designed a network to get initial bases from datasets, which were filtered based on uncertainty and turned into real hard labels for iterative updates of models and bases. Our methods work better than others with 1-3 percentage improvements. |
Keywords
» Artificial intelligence » Deep learning » Self training
