Choosing the Right Local Model: Llama vs Gemma vs Phi for Everyday Tasks

Your data doesn't have to leave your machine. Three open-weight model families, Llama 3.2, Gemma 2, and Phi-3, now run fast enough on a normal laptop to be genuinely useful for daily writing, coding help, and Q&A. No cloud subscription. No API key. No conversation logs shipping off somewhere you can't see.

The problem isn't access anymore. It's the decision. Each family has multiple sizes, the quantization options multiply the combinations further, and most comparison articles either focus on 70B+ monsters that need a server rack, or they test one model in isolation and call it a day.

This article gives you the thing none of those do: a single decision framework for exactly the three model families people actually ask about, on exactly the hardware most people actually own, mapped to the three tasks that cover roughly 80% of everyday AI use.

If you want the full background on why running AI locally matters for privacy, What Cloud AI Sees About You, and Why Local Models Fix It is worth reading first. And if you haven't installed Ollama yet, start with How to Install & Run Ollama: Your First Local AI Model in 10 Minutes before coming back here.

Pick in 30 Seconds: The Decision Table

The table below maps each model to its approximate size class, RAM footprint at Q4_K_M quantization, and relative performance on writing, coding, and Q&A. The 2B–4B models in this class are practical on most laptops for everyday tasks without a discrete GPU.

Bottom line: Phi-3 Mini delivers strong quality at a speed that feels real-time on Apple Silicon. Gemma 2 2B is a fast, lightweight option when raw throughput matters most, and its ~1.7 GB footprint keeps it well within reach on tight hardware. Gemma 2 9B tends toward richer output than its 2B sibling and fits in ~5.7 GB at Q4_K_M, which makes it a realistic choice on machines with adequate free RAM. Llama 3.2 3B is the most versatile generalist and has the widest community support.

Quick picks:
Best all-rounder: Phi-3 Mini 3.8B
Best pure writing: Gemma 2 9B (fits in ~5.7 GB at Q4_K_M)
Fastest raw throughput: Gemma 2 2B
Best community + tool support: Llama 3.2 3B

Hardware Reality Check: What Your Laptop Can Actually Run

RAM is the ceiling. Not your CPU clock speed, not your GPU. If a model's RAM requirement exceeds your available memory, it either won't load or it'll page to disk and slow to a crawl.

8 GB RAM

Phi-3 Mini at Q4_K_M is your best option here. Its compact footprint loads with plenty of room to spare. Gemma 2 2B at ~1.7 GB also fits comfortably. Llama 3.2 3B's moderate footprint puts it within range as well. Gemma 2 9B is a different story: at ~5.7 GB, that footprint is substantial relative to 8 GB total system RAM, and once background processes take their share, headroom can get tight. If you're on 8 GB, test Gemma 2 9B and watch how much free RAM you actually have before committing to it. When in doubt, Phi-3 Mini or Gemma 2 2B are the safer picks.

8–16 GB RAM

All four models are practical here. Phi-3 Mini and Gemma 2 2B run with plenty to spare. Gemma 2 9B at ~5.7 GB and Llama 3.2 3B both fit comfortably alongside normal desktop use on most machines in this range, making them realistic choices even before you hit 16 GB.

16 GB RAM or more

All four options in the table work without any trade-offs. Gemma 2 9B at Q4_K_M leaves substantial headroom for your browser, text editor, and other apps simultaneously, which is the right environment for longer writing sessions where you're moving between the model and your work.

Apple Silicon vs Intel CPU

Apple Silicon Macs have a real structural advantage: unified memory means the model's weights stay in the same memory pool your GPU-equivalent cores read from. There's no data transfer bottleneck between system RAM and a discrete GPU. The M-series Neural Engine and GPU cores accelerate inference directly, which is why Apple Silicon machines often feel noticeably faster than Intel machines at the same RAM tier on these small models.

Intel CPUs are fully capable, but expect noticeably lower throughput than M-series hardware. An i7 with 16 GB gets the job done. It just feels slower.

The 2B–4B size class exists specifically to be practical on everyday hardware. That's the point.

Task-by-Task: Which Model to Actually Use

Writing

For writing, output quality tends to correlate with parameter count in this size class. Larger models in a family generally produce richer, more varied prose than their smaller siblings. Phi-3 Mini is excellent for structured writing, outlines, and shorter pieces, but occasionally repeats itself in longer generations. Llama 3.2 3B is a solid workhorse for general drafting.

The prompt matters as much as the model. Compare these two requests on any model:

Write me a short story about a robot.

Better: "Write a compelling sci-fi short story, approximately 500 words, focusing on the internal conflict of a maintenance robot that discovers sentience while repairing a damaged spacecraft in deep space. Write in third-person limited POV. Prioritize atmosphere over action."

The second prompt gives even a 3B model enough constraints to produce something usable. A well-prompted Gemma 2 9B typically generates substantially more detailed and engaging output than any smaller model given a vague one-liner.

Pick for writing: Gemma 2 9B for quality. Phi-3 Mini for speed or tighter RAM constraints.

Coding

Phi-3 Mini is the surprise here. Microsoft trained it heavily on synthetic, reasoning-focused data, and that shows in code tasks. It performs above what you'd typically expect from a model in this size class, producing cleaner structure and better reasoning through multi-step problems than its parameter count would suggest.

Here's the difference a structured prompt makes for coding:

Write a Python function to calculate the factorial of a number.

Better: "Write a well-documented Python function that accepts a non-negative integer and returns its factorial using recursion. Include a docstring, type hints, and error handling that raises a ValueError for negative inputs with a descriptive message."

Phi-3 Mini responds to the second prompt with functional, readable code that includes all the requested elements. Llama 3.2 3B produces working code but often skips the error handling and type hints unless explicitly instructed. Gemma 2 (both sizes) tends to struggle with anything beyond simple snippets.

For multi-step coding tasks, the 3B–4B models have real limits. They can help you write individual functions, refactor short blocks, or explain what a piece of code does. They'll lose coherence on anything that requires holding a large codebase in context simultaneously. Use them as a pair programmer for focused tasks, not an autonomous agent for full features.

Pick for coding: Phi-3 Mini, by a clear margin in this size class.

Q&A and Reasoning

Simple factual questions (capital cities, definitions, date math) are generally handled well by models in this class. The gaps show up on nuanced or multi-step reasoning.

Take a request like this:

Explain quantum entanglement in simple terms for a 10-year-old.

Phi-3 Mini produces an analogy-driven explanation that tracks well for a non-technical reader. Llama 3.2 3B gives a reasonable answer but defaults to more technical vocabulary. Gemma 2 2B often produces a shorter, less complete response on reasoning-heavy prompts.

The pattern holds across most Q&A tasks that require inference or explanation: Phi-3 Mini's reasoning-focused training pays off. Gemma 2 9B is competitive here and worth considering when writing quality is also a priority.

Pick for Q&A: Phi-3 Mini. Gemma 2 9B as a close second.

Quantization: The Setting You Shouldn't Ignore

Quantization compresses a model's weights to lower numerical precision, which shrinks file size and RAM usage at the cost of some output quality. For everyday tasks, the quality difference between Q4 and Q5 is small enough that most users never notice it in practice.

Two formats worth knowing:

• Q4_K_M: A well-supported starting point in Ollama. Good balance of speed, size, and quality. Use this first. Ollama's default pull may use a generic 4-bit format rather than Q4_K_M specifically; to get this exact quantization, use the :q4_K_M tag where available or pull the GGUF directly from Hugging Face.
• Q5_K_S: Higher quality, higher RAM requirement, and reduced speed. Worth trying if you have 16+ GB and the Q4 output feels flat.

In terms of relative throughput at Q4_K_M, Gemma 2 2B is lightweight and generally fast given its small footprint. Phi-3 Mini is close and produces noticeably better output. Llama 3.2 3B sits in the middle. Gemma 2 9B is the slowest of the four, which is the expected trade-off for its larger parameter count. On Apple Silicon, all models in this size class feel quick for conversational use. On Intel hardware, throughput drops meaningfully, though the models remain usable.

Ollama Commands to Get Started

Pull and run any of these models with Ollama in two lines:

ollama pull phi3
ollama run phi3

ollama pull llama3.2:3b
ollama run llama3.2:3b

ollama pull gemma2
ollama run gemma2

Note: use llama3.2:3b to pull the 3B variant specifically. For Phi-3 Mini, confirm which variant the bare phi3 tag resolves to in the current Ollama library before assuming it's the Mini; if you want a specific size, append the appropriate tag (for example, phi3:mini) or check the Ollama model page to verify. The explicit tag is worth the extra characters to make sure you get the right model and size.

For a specific quantization level, pull the GGUF file directly from Hugging Face and load it via the Ollama Modelfile, or use the :q4_K_M tag variants where available in the Ollama model library.

If you want to see how these same workflows apply to other local models, the Phi-4 vs Gemma vs Llama: Which AI Model is Right for You? post covers the larger-size variants of these same families, which is useful context once your use case outgrows the 3B–4B class.

The Quantization Trade-off You'll Actually Feel

Going from Q5 to Q4 costs you some nuance in longer outputs. You might notice slightly flatter phrasing in a 600-word essay, or a function that's mostly correct but misses an edge case. For most everyday tasks, short Q&A, bullet-point summaries, and code snippets under 50 lines, you won't notice the difference.

Going from Q4 to Q3 is where quality risks increase. With less numerical precision and less parameter capacity to absorb that loss, smaller models can produce noticeably degraded output on reasoning-heavy tasks. Stay at Q4_K_M unless you're working with very constrained hardware and you've tested Q4 and found the quality acceptable for your specific tasks.

Frequently Asked Questions

Which of Llama, Gemma, or Phi runs fastest on a MacBook Air M2 with 16 GB RAM?

Gemma 2 2B is lightweight and tends toward high raw throughput at Q4_K_M. Phi-3 Mini is close and produces noticeably better output quality. If speed is your only criterion, Gemma 2 2B. If you want the best speed-to-quality ratio, Phi-3 Mini.

Can these models handle multi-step coding tasks without cloud APIs?

Phi-3 Mini handles moderately complex coding tasks well at the function and module level. It can write, refactor, and debug focused code blocks without issue. All three families struggle when a task requires reasoning across a large codebase simultaneously. Break the problem into smaller pieces and these models become genuinely useful coding partners. Treat them as a capable junior developer who needs clear, scoped instructions.

How much quality do I lose running 3B–9B models instead of 70B?

More than you'd want for complex creative or research tasks, less than you'd expect for structured everyday tasks. A well-prompted Phi-3 Mini handles email drafting, summarization, Q&A on provided text, and basic code generation at a quality level that's practically useful. It won't write a nuanced long-form essay or solve novel math problems the way a 70B model might. For the tasks that represent most people's daily AI use, the gap is workable.

What quantization level should I use for daily writing and note-taking?

Q4_K_M is the right default. It fits comfortably in RAM on most laptops, runs fast enough to feel responsive, and the quality difference from Q5 is imperceptible for short writing tasks. If you're on 8 GB RAM or doing something especially RAM-constrained, Q4_K_M is still the right call. Drop to Q3 only as a last resort.

Is Phi better than Gemma for reasoning on a normal Windows laptop?

For reasoning tasks specifically, yes. Phi-3 Mini's training data skews heavily toward reasoning and instruction-following, which shows up consistently in Q&A and logic tasks. Gemma 2 2B is faster but shallower on multi-step reasoning. Gemma 2 9B is competitive with Phi-3 Mini on reasoning but runs slower. On a Windows machine where RAM headroom can be tight, Phi-3 Mini's compact footprint relative to its quality output makes it the practical choice.

The Prompt Still Does Half the Work

One thing no benchmark captures: a well-structured prompt on a 3B model often beats a lazy prompt on a 9B model. These small models are sensitive to instruction quality in a way that larger cloud models are more forgiving of. You need to tell them what you want, in what format, for what audience, with what constraints.

That's not a weakness. It's actually useful, because it forces you to think clearly about what you're asking. The model becomes a mirror for the quality of your thinking, which is the right relationship between a person and an AI tool. You stay in the driver's seat. The model handles the generation.

For structured prompt templates designed specifically for local model use, Ultra Prompt's writing, coding, and Q&A prompt packs are built around exactly this principle: give the model enough scaffolding to do its best work, so your time goes to the part only you can do.

Final Verdict

For most people on most laptops, Phi-3 Mini at Q4_K_M is the right starting point. Its compact RAM footprint loads fast, leaves room for the rest of your machine, and punches well above its weight on writing, coding, and reasoning. If writing quality is your primary concern and you have enough free RAM to spare, Gemma 2 9B is worth adding to your roster for longer-form work.

The decision is simpler than the internet makes it look. Pull Phi-3 Mini, run one of the structured prompts above on a real task you actually need done, and see what you get.

If you want pre-built templates for exactly these tasks, Ultra Prompt's prompt library has structured packs for writing, coding, and Q&A that work with any local model.