There are good reasons to run an AI model on your own machine: privacy, no monthly bill, no rate limits, and nothing leaving your laptop. There is also a bad reason, which is believing a 4GB laptop will run something that competes with a data centre. It will not. Here is what genuinely runs at home in 2026, matched to the hardware you actually own.
First, pick a runner
Two tools own this space. Ollama is a developer-first command-line server, ideal when you want to script things or wire a model into your own app. LM Studio is a polished desktop app with a proper interface that anyone can use on day one. Both are free, both run the popular open-weight families. If you want the shortest path, install LM Studio. If you are automating, reach for Ollama.
What to run, by VRAM
| Your hardware | Best pick | Realistic use |
|---|---|---|
| 8GB (or an M1 Air) | Phi-4-mini (3.8B) | Basic assistant, ~15-20 tokens/sec |
| 16GB | Gemma 4 12B | A genuinely good daily assistant |
| 24GB | Qwen3 14B / 30B | Strong all-rounder, light coding |
| 32GB+ | Qwen3.6-35B class | The best most people can run at home |
The model that became the default
Ask experienced local users what to run and most now say the same thing: Qwen3. The 8B, 14B and 30B versions cover the practical range for a home setup, and they punch above their size. For coding specifically, Qwen3.6 27B is the pick; for the tightest 8GB budget, Phi-4-mini is the only thing that fits comfortably while staying usable.
A tip that matters more than the model
Whatever runner you land on, use Unsloth’s Dynamic GGUF quantisations. The right quantisation squeezes a larger model onto smaller hardware with a smaller quality hit than the naive versions, and it is the single change that most often turns “too slow to bother” into “actually fine”.
Be honest about the ceiling
A 30B model on your desk will not match Opus 4.8 or GPT-5.6. What it will do is handle drafting, summarising, classification, private document work and a lot of coding help, with zero data leaving the house and no meter running. For plenty of jobs that trade is worth making. For the hardest reasoning, you will still reach for a hosted model, and that is fine.
Related on Top Tool Stack: Best AI Coding Assistants in 2026 · How to Cut Your AI Bills in 2026
Did you know: a well-quantised 12B model on a 16GB laptop can hold a coherent working conversation entirely offline, on a train, in a tunnel, with the wifi off. The novelty wears off; the privacy does not.