What is local AI?
Local AI means running the model on your own computer instead of somebody else's. Here is what you gain, what you give up, and whether your machine can handle it.
By Nick
On this page
Real screenshot or sketch goes here
NOTHING LEAVES THE ROOM.
When you use most AI tools, your words travel to a company’s computers, get processed there, and the answer comes back. Local AI skips that trip. The model sits on your machine and runs on your machine.
What you get
Privacy that is structural, not promised. Nothing is sent anywhere, so there is no policy to trust and no breach to worry about. For anything sensitive (client work, personal notes, half-formed ideas), that is a genuine difference.
No bill. No subscription, no per-use charge, no anxiety about how much that long document cost to summarise.
It works offline. On a plane, on bad hotel wifi, or when a service is having an outage.
What you give up
Quality. A model small enough to run on a laptop is meaningfully less capable than the large ones. For rewriting, summarising and drafting, the gap is small. For careful reasoning and long, complicated documents, the gap is obvious.
Speed, sometimes. On a recent Mac, responses feel immediate. On an older Windows laptop, they can be slow enough to break your concentration.
Convenience. Someone else’s model is always up to date. Yours is whatever you downloaded.
Can your computer do it?
The number that matters is memory, not processor speed.
- Under 8GB: technically possible, practically frustrating. Not recommended.
- 8–16GB: small models run fine. This covers most everyday text work.
- 16GB and up: comfortable, and you can run larger models that are noticeably better.
Apple Silicon Macs are unusually good at this because the memory is shared with the graphics chip. On Windows and Linux, a dedicated graphics card makes the biggest difference.
You will also need disk space. Each model is a few gigabytes, and it is easy to end up with several.
The sensible position
Most people end up using both. A local model for anything private, routine or offline, and a hosted model for the hard problems. That is not a compromise, it is just picking the right tool, and it means no single outage or price change can stop you working.
The setup guide gets you to a working model in about ten minutes.