Run Your First Local AI Agent in 20 Minutes

Section 1 — Orientation

1-1.What an agent is. A chatbot answers. An agent acts. The difference is not the model — it's the loop around the model. The loop hands the model a goal and a set of tools, lets it choose a tool, executes that tool, feeds the result back, and repeats until the model declares the job done. Every agent product you have seen — coding agents, research agents, computer-use agents — is a version of this loop with better tools and tighter guardrails.

GOAL → PLAN → ACT (call a tool) → OBSERVE (read result) → repeat → DONE

1-2.Why local. Three reasons operators run agents on their own hardware: cost (agents are token-hungry — a single multi-step run can burn hundreds of thousands of tokens, which is free locally), privacy (your files and prompts never leave the machine), and iteration speed (no rate limits, no network latency, works on a plane). See FM-04 for the full cost math.

1-3.Required equipment. Any of: a Mac with Apple Silicon (M1 or later, 8 GB+ unified memory — 16 GB recommended); a Windows/Linux PC with 16 GB RAM (CPU-only works, slowly) or an NVIDIA GPU with 8 GB+ VRAM; Python 3.10+. That's all.

Section 2 — Install and verify Ollama

2-1.Install. Ollama is the standard runtime for local models — think Docker, but for LLMs. One installer, one command to run a model, and it exposes a local API your code can call.

# macOS
brew install ollama        # or download the app from ollama.com

# Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows: download the installer from ollama.com

2-2.Pull a model and verify. Start with a small, tool-capable model. Qwen's small models and Llama 3.x are reliable tool-callers at this size; pick by your RAM (full table in FM-05).

ollama pull qwen3:8b          # 16 GB machines
# or:  ollama pull llama3.2:3b   — 8 GB machines

ollama run qwen3:8b "Say OPERATIONAL and nothing else."
OPERATIONAL

2-3.Verify the API. Ollama serves an HTTP API on port 11434 whenever it's running. Your agent will talk to this endpoint.

curl http://localhost:11434/api/tags
{"models":[{"name":"qwen3:8b", ...}]}

Section 3 — Build the agent loop

3-1.Install the client library.

pip install ollama

3-2.The agent, complete. Save as agent.py. It gives the model two tools — list files in a directory and read a file — and a loop with a hard iteration cap. It can answer questions about your project it has never seen, by deciding for itself which files to open.

import ollama

MODEL = "qwen3:8b"
MAX_STEPS = 8                      # hard cap: agents must not run forever
ALLOWED_DIR = "."                  # tool safety: agent can only touch this dir

# ---- tools: plain Python functions with docstrings ----
import os

def list_files(directory: str) -> str:
    """List files in a directory (non-recursive)."""
    directory = os.path.normpath(os.path.join(ALLOWED_DIR, directory))
    if not directory.startswith(os.path.abspath(ALLOWED_DIR)) and directory != ".":
        return "DENIED: outside allowed directory"
    try:
        return "\n".join(sorted(os.listdir(directory))[:100])
    except OSError as e:
        return f"ERROR: {e}"

def read_file(path: str) -> str:
    """Read a text file and return up to its first 4000 characters."""
    path = os.path.normpath(os.path.join(ALLOWED_DIR, path))
    try:
        with open(path, "r", errors="replace") as f:
            return f.read(4000)
    except OSError as e:
        return f"ERROR: {e}"

TOOLS = {"list_files": list_files, "read_file": read_file}

# ---- the loop: plan → act → observe → repeat ----
def run(goal: str):
    messages = [
        {"role": "system", "content":
         "You are a careful agent. Use tools to gather facts before answering. "
         "When you have enough information, answer directly without calling tools."},
        {"role": "user", "content": goal},
    ]
    for step in range(1, MAX_STEPS + 1):
        resp = ollama.chat(model=MODEL, messages=messages,
                           tools=[list_files, read_file])
        msg = resp["message"]
        messages.append(msg)

        calls = msg.get("tool_calls") or []
        if not calls:                       # no tool call = the agent is done
            print(f"\n[DONE after {step} step(s)]\n{msg['content']}")
            return
        for call in calls:                  # ACT, then OBSERVE
            name = call["function"]["name"]
            args = call["function"]["arguments"] or {}
            print(f"[step {step}] {name}({args})")
            result = TOOLS[name](**args) if name in TOOLS else "ERROR: unknown tool"
            messages.append({"role": "tool", "content": str(result)})
    print("\n[STOPPED] hit MAX_STEPS — raise the cap or narrow the goal")

if __name__ == "__main__":
    run("What does this project do? Inspect the files and summarize in 3 bullets.")

3-3.Run it from inside any project folder:

python agent.py
[step 1] list_files({'directory': '.'})
[step 2] read_file({'path': 'README.md'})
[DONE after 3 step(s)]
• A static site generator written in Python ...

3-4.Read the loop like an operator. Everything that matters in production agents is already visible in these 80 lines: the model chooses actions (tool_calls); execution happens in your code, not the model's; results return as role: "tool" messages; the run ends when the model stops calling tools; and MAX_STEPS is your circuit breaker. Frameworks like LangGraph and CrewAI add orchestration on top — the physics stays the same.

Section 4 — Field discipline (safety)

4-1.Standing rules. (a) Allowlist, don't blocklist — expose only the exact functions needed. (b) Read before write — first agents get read-only tools; earn write access later. (c) Cap everything — steps, output size, file size. (d) Log every call — the print in the loop is not decoration; keep it in production. (e) Confine the blast radius — note ALLOWED_DIR above; agents get a working directory, not a filesystem.

Section 5 — Common failures

Section 6 — FAQ

What is an AI agent, in one sentence?

A language model wrapped in a loop that lets it plan, call tools, observe results, and repeat until the job is done — a chatbot only answers.

Can I run an AI agent without paying for an API?

Yes. Ollama serves open-weight models on your hardware; after the download, inference is free and works offline.

What hardware do I need?

8 GB RAM runs small models well enough to learn on; 16 GB or a 12 GB+ GPU makes agents genuinely useful. Apple Silicon is excellent because unified memory acts as VRAM.

Do I need LangChain or CrewAI to build agents?

No. Frameworks help with orchestration at scale, but the core loop is ~80 lines of plain Python. Learn the loop first; adopt a framework when you need state machines or multi-agent handoffs.