Own Your AI is a nine-part Top Tool Stack series on running your own local AI instead of renting it from the giants. This is Part 3 of 9. See the whole series →
So far this series has argued the why and the honest trade-offs. Now the fun part: actually running an AI on your own machine. If you can install an app and copy-paste one line, you can do this. No coding, no GPU required to start, and about twenty minutes.
The one tool to start with: Ollama
There are many ways to run a local model. For a first-timer, the gentlest is Ollama. It is a free app for Mac, Windows and Linux that handles the fiddly parts (downloading models, running them, serving them to other apps) behind a dead-simple interface. Install it from ollama.com like any normal program.
Step two: download a model and talk to it
Open a terminal (on Windows, “Command Prompt” or “PowerShell”; on Mac, “Terminal”) and type one line:
ollama run gemma3That downloads Google’s small open model, Gemma, and drops you into a chat prompt. Ask it anything. That is a real AI model, running entirely on your computer, with nothing leaving the machine. If you would rather try Meta’s model, swap in llama3.1:8b; for Alibaba’s, qwen3. They are all a single line.
Step three (optional): a proper chat window
Talking to AI in a terminal gets old fast. To get a clean, ChatGPT-style window, most people add a free front-end. The most popular is Open WebUI, which runs in your browser and connects to Ollama automatically. If you would prefer a single desktop app that bundles the model runner and the chat window together, LM Studio is the friendliest option and needs no terminal at all. We will compare these properly in Part 5.
What to expect from your hardware
Being honest about this saves a lot of frustration. A rough guide:
- Any modern laptop (8GB RAM, no dedicated graphics): small models (around 3B to 8B) will run, slowly, on the processor. Fine for trying it out and light tasks.
- 16GB RAM plus a graphics card with 8GB of memory: a comfortable home for 7B to 8B models. This is the sweet spot for most people.
- A recent Apple Silicon Mac (M-series): punches well above its weight because the memory is shared with the graphics chip. An M-series Mac with 16GB or more is a genuinely good local-AI machine.
One word you’ll see everywhere: quantisation
You will notice model names with tags like Q4. Quantisation just means the model has been compressed to run on normal hardware, a bit like zipping a file. A “4-bit” version is smaller and faster with a small, usually unnoticeable, drop in quality. For local use, quantised models are almost always the right choice, and Ollama picks a sensible one for you by default.
What it’s good at, and what it isn’t
Set expectations correctly and you will be delighted; set them at “free GPT-5” and you will be disappointed. A small local model on a normal machine is genuinely good at summarising text, drafting emails and posts, answering questions about documents you give it, brainstorming, and routine coding help. It is not going to match a frontier cloud model on the hardest reasoning, the longest context, or the most obscure knowledge. That is fine. The goal here is a capable, private, always-available assistant for the bulk of everyday work, not a miracle.
That is genuinely it. You now have a local AI. Next week we point it at something far more ambitious: PewDiePie’s Odysseus, the all-in-one workspace, and we find out who it is actually for.
The AI tool stack actually worth paying for
One email a week. The tools, models and moves that matter, minus the hype and the horseshit filter set to maximum. Free.