Summary of Epistemic Uncertainty Quantification For Pre-trained Neural Network, by Hanjing Wang and Qiang Ji
Epistemic Uncertainty Quantification For Pre-trained Neural Network
by Hanjing Wang, Qiang Ji
First submitted to arxiv on: 15 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper addresses epistemic uncertainty quantification (UQ) for pre-trained non-Bayesian models, which can be applied to any network architecture or training technique without requiring the original training data or model modifications. The authors introduce a gradient-based approach to quantify epistemic UQ by analyzing the gradients of outputs relative to model parameters, indicating necessary model adjustments to accurately represent inputs. This method is theoretically guaranteed and improves upon existing methods through class-specific weights for integrating gradients and emphasizing distinct contributions from neural network layers. The authors also enhance UQ accuracy by combining gradient and perturbation methods to refine gradients. The proposed approach outperforms current state-of-the-art UQ methods in out-of-distribution detection, uncertainty calibration, and active learning tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary For any pre-trained model, the new method can identify where it lacks knowledge without requiring the original training data or model modifications. This is achieved by analyzing the gradients of outputs relative to model parameters, which indicates necessary adjustments for accurate representation. The approach is theoretically guaranteed and improves upon existing methods through class-specific weights and emphasizing neural network layers. The proposed method is evaluated on out-of-distribution detection, uncertainty calibration, and active learning tasks, showing its superiority over current state-of-the-art UQ methods. |
Keywords
» Artificial intelligence » Active learning » Neural network
