Summary of Jetmoe: Reaching Llama2 Performance with 0.1m Dollars, by Yikang Shen et al.
JetMoE: Reaching Llama2 Performance with 0.1M Dollars
by Yikang Shen, Zhen Guo, Tianle Cai, Zengyi Qin
First submitted to arxiv on: 11 Apr 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A new Large Language Model (LLM) called JetMoE-8B has been trained with remarkable results, despite being significantly cheaper to develop than previous models. This LLM was created using a combination of publicly available datasets and training code, with the goal of making it more accessible and efficient for future development. The model is based on an efficient architecture that reduces inference computation by about 70% compared to other similar models. JetMoE-8B outperforms several other LLMs, including the Llama2-7B and Llama2-13B-Chat models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new kind of computer program called a Large Language Model (LLM) has been created. This program is very good at understanding and generating human-like text. The new LLM was made to be more affordable and efficient, so that others can use it to make even better programs. It works by using less computer power than other similar programs, which makes it faster and cheaper to use. |
Keywords
» Artificial intelligence » Inference » Large language model
