How to Install & Run Ollama: Your First Local AI Model in 10 Minutes

Every prompt you send to ChatGPT or Claude leaves your machine. It hits a server you don't control, gets logged in ways you can't fully audit, and costs money every time you use it at scale. Ollama fixes all three problems at once, and the install takes less time than brewing coffee.

This guide gets you from zero to a running local AI model on Mac, Windows, or Linux. You'll pull a real open-weight model (Llama 3.1 or Gemma 2), run your first chat in the terminal, and know what to do when something goes sideways. No prior coding experience required.

Start a timer. Ten minutes.

What Ollama Actually Does (and Why It Matters)

Ollama is a runtime that packages large language models into downloadable units called model tags and handles all the dependency management underneath. A useful analogy: it's like Docker, but for AI models. You don't install CUDA libraries manually, configure Python environments, or fight with version conflicts. You run one command and the model works.

Three reasons to care about running models locally:

• Privacy. Your prompts never leave your machine. Nothing is logged on a third-party server.
• Cost. After the initial download, inference is free. No API fees, no subscription tier limits.
• Offline access. Once a model is pulled, you don't need internet to use it.

The tradeoff is hardware. Cloud models run on data center GPUs. Your local model runs on whatever you have at home. That matters for speed, so check your specs before picking a model size.

Not sure whether local AI is right for your specific use case? The breakdown in Local AI vs Cloud AI: What Each Wins, and Where to Run What is worth a read before you commit.

System Requirements and Hardware Check

Ollama itself is lightweight. The models are not. Check these before downloading anything.

Minimum specs

• RAM: More is always better with local models. Tighter memory means slower inference and less headroom for larger models. The exact amount you need depends on which model you pick and your GPU situation, so use the table below as a guide rather than a hard floor.
• Storage: SSD strongly recommended. Model files for the sizes covered here are each several gigabytes, and an HDD will load them noticeably slower.
• CPU: Any modern processor works. Apple Silicon (M1/M2/M3/M4) handles local inference particularly well because of its unified memory architecture.
• GPU: Not required, but speeds things up significantly if you have an NVIDIA card with CUDA support. Ollama detects and uses it automatically.

Model size vs. hardware: what to expect

If your system is on the lighter side, start with Llama 3.1 8B. It's a capable model, not a consolation prize. Gemma 2 9B is a competitive alternative, but it pushes closer to the edge of what constrained systems can handle comfortably.

Running an Apple Silicon Mac? The Apple M-series benchmarking guide covers how to measure and optimize local AI performance on those chips.

Step-by-Step Installation on Mac, Windows, and Linux

Pick your OS. Follow those steps. Ignore the others.

Mac

The cleanest install uses Homebrew. If you don't have Homebrew, install it first at brew.sh, then come back.

brew install ollama

Once it finishes, verify the install:

ollama version

You should see a version number printed to the terminal. That's it for Mac. You're done with installation.

Alternatively, Ollama offers a native Mac app download at ollama.com/download if you prefer a GUI-based install.

Windows

Ollama runs natively on Windows 10 and 11. No WSL required.

1. Download the installer from ollama.com/download.
2. Run the .exe file. Accept the license agreement, choose an install location, and click Install. Standard Windows installer, nothing unusual.
3. Open a new Command Prompt or PowerShell window (important: a new one, so the PATH updates take effect).
4. Verify:

ollama version

If you see a version number, you're good to go.

Linux (Ubuntu / Debian)

One command handles everything:

curl -fsSL https://ollama.com/install.sh | sh

This script detects your hardware, installs the right dependencies, and sets up Ollama as a system service. After it finishes:

ollama version

For other Linux distributions, or to confirm the latest install instructions, check the official docs at ollama.com/download/linux.

Quick reference: all three OSes

Mac and Windows installs are quick on a decent connection. Linux takes a bit longer because the script also sets up a system service on top of the binary install.

Pulling Your First Open-Weight Model (Llama 3.1 or Gemma 2)

With Ollama installed, the next step is downloading a model. This is separate from the Ollama install because you choose which model you want.

If your system is on the lighter side, Llama 3.1 8B is the safer starting point.

Option A: Llama 3.1 8B (Meta)

Meta's Llama 3.1 8B handles writing, coding help, Q&A, and summarization well. It's the most accessible of the models covered here in terms of hardware demands, which makes it the right place to start if you're not sure how your system will perform.

ollama pull llama3.1:8b

Option B: Gemma 2 9B (Google)

Google's Gemma 2 9B is a competitive alternative. Some users find it gives more precise, structured answers on technical tasks. It's a step up in memory demands from Llama 3.1 8B, and on tighter systems it can push things to the limit. Get Llama 3.1 8B working first, then experiment with Gemma 2 9B once you know your setup has the headroom.

ollama pull gemma2:9b

Both models are substantial downloads. Leave the pull command running in the background and come back when the progress bar completes. Once the terminal prompt returns, the model is ready.

Not sure which model fits your actual use case? Phi-4 vs Gemma vs Llama: Which AI Model is Right for You? compares them head to head.

Running Your First Chat in the Terminal

One command starts the conversation:

ollama run llama3.1:8b

Or if you pulled Gemma 2:

ollama run gemma2:9b

Ollama loads the model into memory, then drops you into an interactive prompt. Type anything and press Enter.

Try this first prompt:

Explain the difference between RAM and storage in two sentences, as if I'm 12 years old.

The model responds directly in the terminal. No browser, no account, no internet required after the initial download.

Switching models mid-session

To exit the current model, type:

/bye

Then start a different model:

ollama run gemma2:9b

Each model loads fresh. There's no shared context between sessions unless you build it in.

Saving your chat history

Ollama's terminal interface doesn't save conversation history automatically. Two practical options:

• Quick and dirty: Select all terminal output, copy it, paste it into a text file. Ugly but it works.
• Cleaner: Use a terminal multiplexer like tmux or screen to log the entire session. Run tmux new -s ollama-chat before starting Ollama, and the session stays alive even if you close the terminal window.

For most beginners, copy-paste into a text file is fine. Reach for tmux when you're running longer research sessions you want to revisit.

Listing your downloaded models

At any point, see what models you have locally:

ollama list

Common Errors and Quick Fixes

"ollama: command not found"

The install succeeded but the terminal can't find the binary. This usually means Ollama's install directory isn't in your PATH.

• Mac/Linux: Close the terminal completely, open a new window, and try again. If that doesn't work, add the install directory to your shell config (~/.zshrc or ~/.bashrc).
• Windows: Open a new Command Prompt or PowerShell window (not the one open during install). Windows updates the PATH for new windows only.

"not enough memory" or model fails to load

The model needs more memory than your system currently has available. Two options:

• Close everything else running on your machine (browsers, other apps) to free up memory, then try again.
• Pull a smaller model. Llama 3.2 3B (ollama pull llama3.2:3b) runs on modest hardware and is a reasonable fallback if the 8B model is too heavy for your setup.

Response times are very slow

On CPU-only hardware, some slowness is expected. Throughput varies widely depending on your processor, model size, and how much memory is free, and it will feel slower than a cloud API. If things seem unusually sluggish:

• Close background applications, especially browsers and anything syncing to the cloud.
• Try a smaller model.
• If you have an NVIDIA GPU, make sure the drivers are current. Ollama detects GPU automatically, but outdated drivers can prevent it.

Download stalls or errors mid-pull

Run the same ollama pull command again. In most cases this gets the download moving again. If a stall persists, check your connection, then retry.

FAQ

Does Ollama work on Windows 10 and 11 without WSL?

Yes. Ollama has native Windows support and does not require WSL (Windows Subsystem for Linux). Download the installer from ollama.com/download and run it like any other Windows application.

Which model should I download first if my system is on the lighter side?

Start with Llama 3.1 8B (ollama pull llama3.1:8b). It's the most accessible of the models covered here and handles most everyday tasks well. Gemma 2 9B is a capable alternative but is a step up in memory demands. Get Llama 3.1 8B working first, then experiment from there.

How do I update Ollama and my models later?

To update a model, run ollama pull llama3.1:8b (or whichever model) again. Ollama checks for a newer version and downloads only what changed. To update Ollama itself, re-run the original install command for your OS. On Mac with Homebrew: brew upgrade ollama. On Linux, re-run the install script. On Windows, download and run the latest installer.

Can I use Ollama with a GUI instead of the terminal?

Yes. Several GUI frontends work with Ollama. Open WebUI is the most popular, and it gives you a chat interface similar to ChatGPT running entirely on your own machine. The Ollama library page at ollama.com lists compatible integrations. The terminal is the fastest way to start, but you don't have to stay there.

Is it safe to run open-weight models locally?

From a privacy standpoint, local models are safer than cloud APIs because your prompts never leave your machine. The models themselves (Llama 3.1, Gemma 2) are from Meta and Google respectively, with published research papers and open weights. The risk profile is fundamentally different from running unknown software.

You're Running Local AI. Now What?

You've got Ollama installed, a real open-weight model downloaded, and a working terminal chat session. That took less than 10 minutes. Your data stays on your machine, it costs nothing to run, and the model works whether or not your internet does.

The terminal gets you started. But the quality of what you get back depends almost entirely on how you prompt. If you want to skip the trial-and-error phase, Ultra Prompt's structured prompt templates are built for exactly this, including templates designed specifically for local model workflows.