Summary of Lca-on-the-line: Benchmarking Out-of-distribution Generalization with Class Taxonomies, by Jia Shi et al.
LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies
by Jia Shi, Gautam Gare, Jinjin Tian, Siqi Chai, Zhiqiu Lin, Arun Vasudevan, Di Feng, Francesco Ferroni, Shu Kong
First submitted to arxiv on: 22 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Lowest Common Ancestor (LCA)-on-the-Line framework aims to predict models’ Out-of-Distribution (OOD) performance using in-distribution (ID) measurements without requiring OOD data. This approach revisits the established concept of LCA distance, which measures the hierarchical distance between labels and predictions within a predefined class hierarchy. The method is evaluated on 75 models using ImageNet as the ID dataset and five significantly shifted OOD variants, showing a strong linear correlation between ID LCA distance and OOD top-1 accuracy. The framework provides an alternative for understanding why Visual-Language Models (VLMs) tend to generalize better than Vision Models (VMs), despite having similar or lower ID performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Predicting how well models will perform when shown new, unfamiliar data is a big challenge in machine learning. Researchers have developed ways to test model performance using familiar data, but these methods aren’t always accurate. A new approach called the Lowest Common Ancestor (LCA)-on-the-Line framework aims to improve this by looking at how similar or different new data is from what the model has seen before. This method works well for certain types of models and could help us understand why some models are better than others at handling new situations. |
Keywords
* Artificial intelligence * Machine learning
