How much VRAM does a local model need?

Rule of thumb at Q4 quantization: about 0.6–0.7 GB per billion parameters, plus 1–4+ GB for the context window. A 8B model needs ~6 GB; a 32B model ~20 GB; a 70B model ~40+ GB.

What quantization should I use?

Q4_K_M is the field default — the best quality-per-gigabyte trade for most models. Go Q8 only if you have memory to spare; avoid Q2–Q3 for anything requiring precision, especially agent tool calls.

What is the best local model for coding?

In the sizes most people can run: coding-specialized models in the 7B–32B range (e.g., Qwen's coder line, and newer agent-trained coding models) outperform general models of the same size. On 24 GB-class hardware, a 32B coder is the workhorse choice.

FM-05 · EQUIPMENT SELECTIONLAST VERIFIED 2026-07 · FREE ISSUE

The Model Picker: Which Local Model for Which Job

Purpose. End model-selection paralysis. Three rules, one table, one drill — and you'll pick correctly for any task on any hardware, including models released after this manual was printed.

Section 1 — The three rules

1-1.Rule 1: Fit beats size. A model that fits fully in VRAM/unified memory at a moderate quant will outperform a "better" model that spills to system RAM or disk — because the second one is unusably slow. The fit rule at Q4: ~0.6–0.7 GB per billion parameters, plus 1–4+ GB for context. An 8B model wants ~6 GB; 32B wants ~20 GB; 70B wants ~40+ GB.

1-2.Rule 2: Specialists beat generalists at equal size. A 7B coding-tuned model writes better code than a 7B generalist; a reasoning-distilled model solves harder problems than its base. When a task category dominates your usage, issue the specialist.

1-3.Rule 3: Newest-within-class usually wins. Open models improve fast; a current 12B routinely matches last year's 30B. When a new release lands, don't ask "is it good?" — ask "what class is it, does it fit, does it beat my incumbent on my drill (§4)?" That procedure outlives any table, including this one.

Section 2 — The picker table

Model names below are the reliable picks as of this revision — the classes and sizes are the durable knowledge. Ollama tags shown.

Job	~8 GB (entry)	~16 GB (standard)	~24–32 GB (heavy)	~64 GB+ (max)
General assistant chat, drafting, summaries	`llama3.2:3b` · `gemma3:4b`	`qwen3:8b` · `gemma3:12b`	`gemma3:27b` · `qwen3:32b`	`llama3.3:70b`
Coding completion, refactors, review	`qwen2.5-coder:3b`	`qwen2.5-coder:7b`	`qwen2.5-coder:32b` · agent-trained coders (e.g. `devstral`)	32B coder at Q8, or 70B general
Agents / tool use loops, routing, function calls	`qwen3:4b`	`qwen3:8b` · `llama3.1:8b`	`qwen3:32b` · `mistral-small`	`llama3.3:70b`
Hard reasoning math, multi-step logic	— (be realistic)	`deepseek-r1:8b`	`deepseek-r1:32b` · `qwen3:32b` (thinking)	`deepseek-r1:70b`
Fast pipeline steps classify, extract, route	`qwen3:0.6b`–`1.7b`	`llama3.2:3b`	same — small is correct here	same
Vision images, screenshots, docs	`gemma3:4b`	`gemma3:12b` · `llama3.2-vision`	`gemma3:27b`	`gemma3:27b` at Q8

2-1.Reading the table. Columns are memory available to the model — on PCs that's GPU VRAM; on Apple Silicon it's unified memory minus ~6–8 GB for the OS (details: FM-02 §3). The "fast pipeline steps" row is the most under-used: multi-model setups that route easy calls to a 1–3B model and hard calls upward are how professionals make local feel fast (and how they cut cloud bills — FM-04 §3-2).

Section 3 — Quantization doctrine

3-1. Quantization compresses model weights to fewer bits so bigger models fit in less memory, trading a little quality for a lot of capacity. The field settings:

Quant	Doctrine
Q4_K_M	The default. Best quality-per-GB for nearly all models and jobs. Ollama's standard tags are usually this. When in doubt, this.
Q8 / FP16	Spend memory on it only when you have surplus after fitting the model class you want. A bigger model at Q4 usually beats a smaller one at Q8.
Q2–Q3	Emergency rations. Noticeable degradation — and the first casualty is precision: exact formats, numbers, and agent tool-call arguments. Never for agents.

Section 4 — The 10-minute evaluation drill

4-1. Leaderboards don't run your workload. Build a personal drill once, reuse it forever:

Collect 5 real tasks from your actual work — a function you needed written, a doc you needed summarized, an email to draft, a tool-call scenario, one task that previously failed.
Freeze them in a file with what "good" looks like for each. This is your golden set.
Run every candidate model against all 5 (ollama run <model> < task.txt). Score pass/fail, note speed.
Issue the winner. Re-run the drill whenever a new release tempts you. Ten minutes, evidence, done.

Field note: the drill kills the two chronic diseases of local AI — chasing every release (new model must beat the drill to earn a slot) and loyalty to a stale pick (the drill has no feelings).

Section 5 — FAQ

How much VRAM does a model need?: At Q4: ~0.6–0.7 GB per billion parameters plus 1–4+ GB for context. 8B → ~6 GB; 32B → ~20 GB; 70B → ~40+ GB.
What quantization should I use?: Q4_K_M unless you have a measured reason otherwise. Avoid Q2–Q3 for agents and anything needing precision.
Best local coding model?: The coding specialist that fits your memory class — at 24 GB-class hardware, a 32B coder is the workhorse; at 16 GB, a 7B coder.
The table will go stale. Then what?: The rules won't: fit beats size, specialist beats generalist, newest-within-class wins — verified by your own drill. That's the manual's real payload. Updated picks ship in the weekly dispatch.

Cross-references

Set up the machine: FM-02. Put the model in an agent loop: FM-01. Decide what runs local vs cloud: FM-04.

Get the Field Kit (includes the picker as a printable sheet)