Summary of Large Language Models Are Interpretable Learners, by Ruochen Wang and Si Si and Felix Yu and Dorothea Wiesmann and Cho-jui Hsieh and Inderjit Dhillon
Large Language Models are Interpretable Learners
by Ruochen Wang, Si Si, Felix Yu, Dorothea Wiesmann, Cho-Jui Hsieh, Inderjit Dhillon
First submitted to arxiv on: 25 Jun 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Symbolic Computation (cs.SC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Large Language Model-based Symbolic Programs (LSPs) bridge the gap between expressiveness and interpretability in predictive models for classification and decision-making. By combining pre-trained Large Language Models (LLMs) with symbolic programs, LSPs provide a massive set of interpretable modules that transform raw input into natural language concepts. These modules are then integrated into an interpretable decision rule using symbolic programs. To train LSPs, a divide-and-conquer approach is developed to incrementally build the program from scratch, guided by LLMs. The effectiveness of LSPs in extracting interpretable and accurate knowledge from data is evaluated using IL-Bench, a collection of diverse tasks across different modalities. Empirical results show that LSPs outperform traditional neurosymbolic programs and vanilla automatic prompt tuning methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary LSPs are a new way to make predictive models more understandable. They combine two types of programming: Large Language Models (LLMs) that can understand natural language, and symbolic programs that can reason about rules. This combination allows LSPs to take raw input and turn it into natural language concepts. The model is then trained using a special approach that builds the program step by step, guided by the LLM. To test how well LSPs work, researchers created a set of tasks called IL-Bench, which includes both made-up scenarios and real-world examples from different areas like text, images, or audio. Results show that LSPs perform better than other similar models. |
Keywords
» Artificial intelligence » Classification » Large language model » Prompt
