Summary of Ceebert: Cross-domain Inference in Early Exit Bert, by Divya Jyoti Bajpai and Manjesh Kumar Hanawal
CEEBERT: Cross-Domain Inference in Early Exit BERT
by Divya Jyoti Bajpai, Manjesh Kumar Hanawal
First submitted to arxiv on: 23 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Pre-trained Language Models (PLMs) like BERT have shown remarkable performance and generalization across various tasks, but their large size leads to inference latency issues. To address this, side branches are attached at intermediate layers, enabling early inference of samples without requiring them to pass through all layers. The challenge lies in deciding which layer to infer and exit each sample while balancing accuracy and latency. Moreover, the distribution of samples to be inferred may differ from that used for training, necessitating cross-domain adaptation. This paper proposes an online learning algorithm, CeeBERT, that dynamically determines early exits of samples based on confidence levels at each exit point. CeeBERT learns optimal thresholds from domain-specific confidence observed at intermediate layers on the fly, eliminating the need for labeled data. Experimental results on five distinct datasets with BERT and ALBERT models demonstrate CeeBERT’s ability to improve latency by reducing unnecessary computations with minimal drop in performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making language models like BERT work faster without sacrificing accuracy. These models are very good at understanding text, but they take a long time to make predictions because they’re very large and complex. The researchers propose a new way to speed up these models by stopping them early when they’re confident in their answers. This approach, called CeeBERT, can reduce the time it takes for BERT to make predictions by 2-3.5 times without losing much accuracy. |
Keywords
» Artificial intelligence » Bert » Domain adaptation » Generalization » Inference » Online learning
