Summary of Personal Intelligence System Unilm: Hybrid On-device Small Language Model and Server-based Large Language Model For Malay Nusantara, by Azree Nazri et al.
Personal Intelligence System UniLM: Hybrid On-Device Small Language Model and Server-Based Large Language Model for Malay Nusantara
by Azree Nazri, Olalekan Agbolade, Faisal Aziz
First submitted to arxiv on: 9 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel Personal Intelligence System designed to efficiently integrate on-device and server-based language models in resource-constrained contexts. The system combines SLiM-34M, an optimized model for low memory and power usage, with MANYAK-1.3B, a high-performance model for server-based tasks. The models demonstrate significant results across machine translation, question-answering, and translate IndoMMLU tasks. Notably, SLiM-34M achieves high accuracy improvements using 2 times fewer pre-training tokens than other large language models (LLMs). This work challenges the assumption that large-scale computational resources are necessary to build effective language models, contributing to the development of resource-efficient models for Malay languages. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper talks about how to make language processing better on devices with limited power and memory. The researchers created a system that uses two different types of models: one that runs on the device itself (SLiM-34M) and another that uses server-based computing (MANYAK-1.3B). This system is designed to work well in situations where computers are not powerful enough, which is important for languages like Malay. The results show that these models can be very accurate and efficient, even when using less data than other language processing systems. |
Keywords
» Artificial intelligence » Question answering » Translation
