How to run local AI models (without coding)

Running AI locally sounds like something that requires a powerful machine and a computer science degree. It requires neither. It requires a download and some patience while it finishes.

1. Check your computer can handle it

The number that matters is memory.

• 8GB: small models work fine.
• 16GB or more: comfortable, with better models available to you.
• Under 8GB: it will run, but slowly enough that you probably will not stick with it.

Apple Silicon Macs are particularly good at this. On Windows, a dedicated graphics card helps a lot but is not required.

2. Install Ollama

Go to ollama.com, download the version for your system, and install it the way you would install any other app. There is nothing unusual about it.

3. Run your first model

Open a terminal (Terminal on a Mac, PowerShell on Windows) and type:

ollama run llama3

The first time, this downloads a few gigabytes. It is a one-off. Get a coffee.

When it finishes, you get a prompt. Type a question, press enter, and you are talking to a model running on your own machine.

To leave, type /bye.

4. Try it on something real

The temptation is to test it with trivia, which is the thing small models are worst at. Try it on the work you would actually use it for instead:

• Rewrite this paragraph more clearly.
• Summarise these notes into five bullet points.
• Draft a reply to this message.

That tells you whether it is good enough for you, which is the only question that matters.

5. Pick a better model if you need one

ollama list shows what you have. New models appear constantly, and the best choice this month may not be the best choice next month, which is exactly why this guide does not give you a definitive ranking.

Bigger models are better and slower. If responses feel sluggish, drop to a smaller one.

6. Use it without the terminal

If the terminal is not for you, several free desktop apps connect to Ollama and give you a normal chat window. You still install Ollama first. The apps are just a nicer front door to the same thing.

What to expect, honestly

It will not match the best hosted models. On everyday text work the difference is small. On hard reasoning it is obvious. Most people end up using both, and that is a perfectly sensible place to land.