Ollama on a Mac: The Complete Field Manual
Purpose. Configure a Mac as a serious local-AI machine: what Apple Silicon can actually run, the exact settings that prevent the two most common failures, and how to serve models to your whole network.
Section 1 — Why the Mac is a privileged platform
1-1.Unified memory is the whole story. On a PC, a model must fit in the GPU's VRAM — and consumer cards top out around 24 GB. On Apple Silicon, CPU and GPU share one pool of unified memory, and the GPU can address most of it. A 64 GB MacBook is, for local AI purposes, a ~48 GB "GPU" — a capacity that costs multiples more to assemble from discrete cards. This is why Mac Studios are quietly everywhere in the local-AI community.
1-2.The trade. Macs have huge memory but lower raw compute than big NVIDIA cards. Practical effect: Macs excel at running large models at conversational speed; NVIDIA excels at serving many parallel requests fast. For one operator or a small team, the Mac trade is usually the right one.
Section 2 — Installation
2-1.Install and verify. Metal GPU acceleration is automatic — there is nothing to configure.
brew install ollama # or the app from ollama.com (adds menu bar control)
ollama --version
ollama run llama3.2:3b "Reply with the single word READY."
READY2-2.Where things live. Models are stored under ~/.ollama/models. They are large — a 70B model at Q4 is ~40 GB on disk. Keep 100+ GB free if you intend to collect.
Section 3 — The model-by-memory table
3-1.The fit rule. At the standard Q4_K_M quantization, budget roughly 0.6–0.7 GB of memory per billion parameters, then leave headroom for the context window and macOS itself (assume the OS wants 6–8 GB). A model that doesn't fit will still "run" — by swapping to SSD at unusable speed. Fit beats size, always.
| Unified memory | Comfortable class | Issue these first | Realistic use |
|---|---|---|---|
| 8 GB | 3–4B | llama3.2:3b, qwen3:4b, gemma3:4b | Drafting, summarization, light agents; keep other apps closed |
| 16 GB | 7–14B | qwen3:8b, llama3.1:8b, gemma3:12b, phi4:14b | Daily assistant, capable agents, coding help |
| 32 GB | 24–32B | qwen3:32b, gemma3:27b, qwen2.5-coder:32b, mistral-small | Serious coding agents, strong reasoning, RAG |
| 64 GB | 70B-class | llama3.3:70b, deepseek-r1:70b | Near-frontier quality; the sweet spot for professionals |
| 128 GB+ | 70B high-quant / 100B+ MoE | 70B at Q8; large mixture-of-experts releases | Team server, maximum-quality local inference |
Model names move fast; the memory classes don't. When a new model drops, ask one question: how many GB is the quant, and does it fit with headroom? Current picks per task: FM-05.
Section 4 — The two settings that fix most Mac problems
4-1.Context length (the silent killer). Ollama defaults to a small context window. Long chats, agents, and RAG silently truncate — the model "forgets" your earlier instructions and loops or contradicts itself, and no error is shown. Raise it:
# per-server (recommended). Persist via launchctl or the app's settings:
OLLAMA_CONTEXT_LENGTH=16384 ollama serve
# or per-request in the API: options: {"num_ctx": 16384}Cost: context eats memory (rough order: gigabytes at long contexts, varies by model architecture). If a bigger context makes the model swap, use a smaller model with a bigger window — for agent work, that trade is nearly always correct.
4-2.Keep-alive (the cold-start fix). Ollama unloads idle models after ~5 minutes; the next request pays a multi-second reload. If the Mac is a dedicated AI machine:
OLLAMA_KEEP_ALIVE=24h ollama serve # or "-1" to pin the model in memorySection 5 — Serving your network (the Mac-as-AI-server pattern)
5-1.One Mac, every device. A single capable Mac can serve models to your laptop, phone apps, and editor plugins. Bind Ollama to all interfaces:
OLLAMA_HOST=0.0.0.0 ollama serve
# other devices now use: http://<mac-ip>:11434 (OpenAI-compatible endpoint: /v1)0.0.0.0 exposes the API to everyone on the network, unauthenticated. Do this on a trusted LAN only, or put a reverse proxy with auth in front. Never port-forward 11434 to the internet.
5-2.What plugs in. Anything that speaks the OpenAI API format: VS Code agents (Cline, Continue), Open WebUI for a ChatGPT-style interface, and the agent loop from FM-01 pointed at the Mac's IP.
Section 6 — When to use MLX instead
6-1. MLX is Apple's own ML framework; MLX-format models (via LM Studio or mlx-lm) are often meaningfully faster on Apple Silicon than the same model in Ollama, at the cost of a smaller catalog and less tooling. Doctrine: start on Ollama for the ecosystem; if a model you run constantly exists in MLX format, benchmark it — free speed is free speed. LM Studio runs both formats and is the easiest way to test.
Section 7 — Common failures
| Symptom | Cause | Fix |
|---|---|---|
| Tokens crawl; fans roar; Mac stutters | Model + context exceed memory → SSD swap | Smaller model or lower quant. Watch memory pressure in Activity Monitor while loaded. |
| Agent forgets instructions / loops | Default context window truncating | OLLAMA_CONTEXT_LENGTH=16384 (§4-1) |
| First request each session is slow | Model unloaded after idle | OLLAMA_KEEP_ALIVE=24h (§4-2) |
| Other devices can't reach the Mac | Ollama bound to localhost only | OLLAMA_HOST=0.0.0.0 (§5-1) + firewall allow |
| Disk mysteriously full | Model collection under ~/.ollama | ollama list then ollama rm <model> |
Section 8 — FAQ
- Why are Macs good for local AI?
- Unified memory: the GPU can address nearly all system RAM, so a 64 GB Mac behaves like a ~48 GB GPU — capacity that costs far more with discrete cards.
- What size model can my Mac run?
- At Q4: ~0.6–0.7 GB per billion parameters plus headroom. 16 GB → up to ~14B comfortably; 32 GB → 30B-class; 64 GB → 70B-class.
- Is 8 GB enough?
- For 3–4B models, yes — genuinely useful for drafting and light agents. It is a starter kit, not a workshop.
- Ollama or LM Studio?
- Ollama for scriptability, serving, and ecosystem; LM Studio for a GUI and easy MLX testing. Many operators run both.