Summary of Sc-phi2: a Fine-tuned Small Language Model For Starcraft Ii Macromanagement Tasks, by Muhammad Junaid Khan and Gita Sukthankar
SC-Phi2: A Fine-tuned Small Language Model for StarCraft II Macromanagement Tasks
by Muhammad Junaid Khan, Gita Sukthankar
First submitted to arxiv on: 17 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces SC-Phi2, a fine-tuned StarCraft II small language model for macromanagement tasks, which is a streamlined version of large language models requiring less power and memory. The authors create a new text dataset for teaching Microsoft’s Phi2 model about StarCraft races, roles, and actions using self-supervised learning. They pair this language model with a Vision Transformer (ViT) from the pre-trained BLIP-2 model, fine-tuning it on the MSC replay dataset to construct dynamic prompts including visual game state information. Unlike large models used in StarCraft LLMs like GPT-3.5, Phi2 is trained primarily on textbook data and contains little inherent knowledge of StarCraft II beyond what’s provided by training. The authors demonstrate that their model performs well at micromanagement tasks with a small number of parameters using LoRA (Low-rank Adaptation) and quantization, which enables training on a single GPU. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper makes a cool computer program that can play StarCraft II really well! They took a special kind of AI called a “small language model” and trained it to understand the game. The model learns from lots of text about the game, like what each character can do. Then they added some extra parts to make the model even better. It’s really good at making decisions during the game, like what to build or when to attack. This is special because most AI models that are good at games need a lot of power and memory. This one doesn’t! The authors did all this to show how their model can play StarCraft II well with just a small amount of information. |
Keywords
» Artificial intelligence » Fine tuning » Gpt » Language model » Lora » Low rank adaptation » Quantization » Self supervised » Vision transformer » Vit
