What is a small language model? Examples you can run on your phone
· By Modelbase Team
A small language model (SLM) is an AI language model small enough to run on an everyday device such as a phone or a laptop, typically with somewhere between a few hundred million and a few billion parameters. Large language models like the ones behind ChatGPT run in data centers; small ones run on your own hardware, which makes them private, available offline and free to use as much as you like. Today’s SLMs handle writing, summarizing, answering questions and even looking at photos, on an iPhone.
This explainer covers what counts as “small”, what numbers like 1.2B and 4-bit mean, real examples with their sizes, and what SLMs are and aren’t good at.
How small is “small”?
There’s no official cut-off. In practice, a model is called small when it fits the memory of consumer devices:
- Under 1 billion parameters: tiny models for quick, simple tasks; they run on any phone.
- 1 to 4 billion parameters: the sweet spot for phones, good at everyday chat and writing.
- Up to roughly 10 billion: still “small” by today’s standards and comfortable on many laptops; on a phone only with aggressive compression.
Every one of the 53 models in Modelbase’s catalog is a small language model by this measure, from Google’s Gemma 3 with 270 million parameters to PrismML’s Ternary Bonsai with 8.2 billion.
What “parameters” means
A model’s parameters (also called weights) are the numbers it learned during training. They encode everything the model knows about language and the world. The “B” in a model’s name counts them in billions:
- LFM 2.5 (1.2B): 1.2 billion parameters.
- Phi 4 Mini (3.8B): 3.8 billion.
- LFM 2.5 VL (450M): 450 million (“M” for million).
More parameters generally means more knowledge and better reasoning, but also a bigger file, more memory and more work for the chip on every word. Small models make up for some of that with better training data and newer designs, so a recent 2B model can do much more than one from a few years ago. (Models also differ in how much text they can consider at once, their context window, measured in tokens; see what a token is.)
One special case: Google’s Gemma 4 (E2B) and Gemma 3n (E2B) are described in effective parameters, about 2 billion, the number the model works with at a time. They hold more parameters in total, which is why even the text-only Gemma 4 Text (E2B) is a 2.67 GB download, bigger than a typical 2B model.
What quantization does
Each parameter is stored as a number with a certain precision. Models are usually trained with 16 bits per parameter. Quantization stores them with fewer bits, most often 4, which makes the model about a quarter of the size, with a small loss in quality.
A rough rule: download size ≈ parameters × bits ÷ 8, plus a little extra for the parts kept at higher precision. The models in Modelbase show how well it holds:
| Model | Parameters | Bits per weight | Rule of thumb | Actual download |
|---|---|---|---|---|
| LFM 2.5 (1.2B) | 1.2B | 4 | 0.6 GB | 0.66 GB |
| Phi 4 Mini (3.8B) | 3.8B | 4 | 1.9 GB | 2.18 GB |
| SmolVLM 2 (500M) | 0.5B | 16 (unquantized) | 1.0 GB | 1.02 GB |
| Ternary Bonsai (8B) | 8.2B | 2 | 2.05 GB | 2.32 GB |
Two things stand out. SmolVLM 2 (500M) has less than half the parameters of LFM 2.5 (1.2B) but a bigger download, because it isn’t quantized. And Ternary Bonsai fits 8.2 billion parameters into 2.32 GB because each weight is only -1, 0 or +1. That’s how a model with twice the parameters of the 4B models fits on a recent iPhone.
There’s also quantization-aware training (QAT): training a model so it keeps its quality after compression. Gemma 3 QAT (1B and 4B) in the catalog are trained this way.
Memory matters as much as storage: a model needs its download size and a bit more in free memory while it answers. That’s why bigger models are recommended for newer iPhones with more memory.
Small language models you can run today
These all run on an iPhone, iPad or Mac in Modelbase. Sizes are the downloads in the app.
| Model | Maker | Parameters | Download | Known for |
|---|---|---|---|---|
| Gemma 3 (270M) | 0.27B | 0.19 GB | Simple instructions, summaries | |
| Granite 4.0 H (350M) | IBM | 0.35B | 0.20 GB | Quick answers on any device |
| LFM 2.5 (350M) | Liquid AI | 0.35B | 0.23 GB | Everyday chat on older iPhones |
| Qwen 3 (0.6B) | Qwen | 0.6B | 0.35 GB | A light first model |
| Qwen 3.5 (0.8B) | Qwen | 0.8B | 0.65 GB | Fast answers, sees images |
| LFM 2.5 (1.2B) | Liquid AI | 1.2B | 0.66 GB | Comes with Modelbase |
| Llama 3.2 (1B) | Meta | 1.2B | 0.71 GB | Multilingual dialogue |
| DeepSeek R1 (1.5B) | DeepSeek | 1.5B | 1.01 GB | Step-by-step math and logic |
| MiniCPM 5 (2B) | OpenBMB | 2B | 1.43 GB | Coding, math, English and Chinese |
| Qwen 3.5 (2B) | Qwen | 2B | 1.75 GB | Balanced all-rounder with vision |
| SmolLM 3 (3B) | Hugging Face | 3B | 1.75 GB | Long conversations in six languages |
| Granite 4.0 H Micro | IBM | 3B | 1.81 GB | Following instructions |
| Llama 3.2 (3B) | Meta | 3.2B | 1.82 GB | Dialogue and summaries |
| Phi 4 Mini (3.8B) | Microsoft | 3.8B | 2.18 GB | Math, logic, instructions |
| Ternary Bonsai (8B) | PrismML | 8.2B | 2.32 GB | A big model in a small download |
| Ministral 3 (3B) | Mistral AI | 3.4B | 2.78 GB | Many languages, sees images |
| Qwen 3.5 (4B) | Qwen | 4B | 3.06 GB | The most capable small Qwen |
| Gemma 4 (E2B) | about 2B effective | 3.58 GB | Conversation, images, reasoning |
For help choosing between them, see Which AI model should you run on your iPhone?
What small models are good at, and where they struggle
Good at:
- Writing and rewriting: emails, messages, summaries, a friendlier or shorter version of your text.
- Working with what you give them: a pasted text, a PDF, a photo, your notes. When the facts are in front of them, small models are reliable.
- Everyday questions, explanations and brainstorming.
- Translation and multilingual chat (Qwen 3.5, Ministral 3, Llama 3.2 and others are trained on many languages).
- Structured tasks: extracting dates and names, making lists and tables.
- Step-by-step math and logic, with reasoning models such as LFM 2.5 Thinking or DeepSeek R1.
Where they struggle:
- Obscure facts. A small model simply holds less knowledge, so it’s more likely to make something up about a niche topic. Check facts that matter, or turn on web search when you’re online.
- Long, complex reasoning and big pieces of code. These still favor large models.
- Recent events. A model knows what it learned in training; anything newer needs web search.
Small vs large language models
| Small language model | Large language model | |
|---|---|---|
| Size | Hundreds of millions to a few billion parameters | Far larger |
| Where it runs | On your phone, tablet or laptop | In a company’s data center |
| Needs internet | No, once downloaded | Yes, always |
| Privacy | Your text stays on your device | Your text goes to the provider |
| Cost | Free once downloaded | Often a subscription or pay per use |
| Knowledge and hard reasoning | Good, with limits | Best available |
You don’t have to choose one forever. In Modelbase, you can chat with small models on your device and, when you need more, connect a large cloud model through OpenRouter with your own key; both appear in the same model picker. More in On-device AI vs cloud AI, and for the main small-model families compared, Gemma vs Llama vs Qwen.
How to try one
Modelbase is free for iPhone, iPad (iOS 18 or later) and Macs with Apple silicon. It comes with LFM 2.5 (1.2B) installed, so you can try a small language model the moment the app opens, even offline. To add more, tap the model’s name at the top of a chat, then Manage models. The full walkthrough is in How to run an LLM locally on iPhone, iPad and Mac.
FAQ
What is a small language model, in one sentence?
A small language model is an AI model that understands and writes text, small enough (usually a few billion parameters or fewer) to run on a phone or laptop instead of in a data center.
Is ChatGPT a small language model?
No. ChatGPT runs large models on OpenAI’s servers. Small language models are the ones you can download and run yourself, such as Qwen 3.5, Gemma, Llama 3.2, Phi 4 Mini or LFM 2.5.
What’s the smallest useful language model?
Models under half a billion parameters, such as Gemma 3 (270M) or LFM 2.5 (350M), already handle simple instructions, short answers and summaries. For general chat, 1 to 4 billion parameters is where small models become genuinely helpful.
Can small language models see images?
Some can. Vision models such as LFM 2.5 VL, Qwen 3.5 and Gemma 4 (E2B) answer questions about photos on the device. See how to ask AI about a photo.
Do small language models work offline?
Yes. Once downloaded, a small model runs entirely on your device, with no connection needed.
What does 4-bit mean?
Each of the model’s parameters is stored in 4 bits instead of the usual 16, which makes it about a quarter of the size. Most models in Modelbase are 4-bit, in the format of Apple’s MLX.
Try it on your iPhone, iPad or Mac
Modelbase is free on the App Store.