Local LLM Agent-Capability: Which Open Models Reliably Drive Tools (2026)

A local LLM that writes a beautiful paragraph can still fail completely at the one thing an agent needs: reliably emitting a valid tool call, every time, in the exact shape your code expects. That’s a distinct, trained-in skill, and it’s what separates a chatbot from an agent brain. This database is the agent-capability layer on top of our local-LLM model catalog — it scores the major self-hostable families on how well they actually drive tools, measured against the benchmarks of record rather than vibes, with every number dated because this leaderboard genuinely moves month to month.

Native format beats grammar — but you want both

The single distinction that decides whether a model works as an agent is whether it has a native, trained-in tool template. Qwen (Hermes-style), Mistral ([TOOL_CALLS]), Llama (<|python_tag|>), Hermes (which actually invented the <tool_call> format everyone else now borrows), Granite, GLM and Phi-4-mini all have one — the model has been taught to choose the right call and format it correctly. Gemma 2 and 3, base Phi-4, and DeepSeek-R1 do not; with them, every tool call is prompt-engineered and parsed by hand. That’s where grammar-constrained decoding comes in: tools like llama.cpp’s GBNF, Outlines, and XGrammar (now the default backend in both vLLM and SGLang) restrict the model’s allowed tokens so the output physically cannot leave your JSON schema. The key nuance: a native format makes the model pick the right call; grammar only guarantees the call is shaped validly. You want both — and on small or weak models, grammar-constraining is what makes them usable at all.

Who’s actually winning, and the size cliff

As of mid-2026 the open-weight tool-calling crown belongs to Qwen, which holds roughly ten of the thirteen slots on the Berkeley Function-Calling Leaderboard; for the heavier agentic benchmarks (tau-bench, SWE-bench) GLM-4.5/4.6 and Kimi K2.6 lead, all under permissive MIT-class licenses. Llama and Mistral, despite strong 2024–25 numbers, have slipped off the live top table. And there’s a hard floor: reliability falls off a cliff below about four billion parameters — Qwen3.5 drops from 0.661 at 9B to 0.503 at 4B to 0.436 at 2B — and parameter count is a poor predictor under 8B, with genuinely non-monotonic quirks. The smallest safe local agent driver is the ~4B on-device trio (Phi-4-mini being the safest single pick: MIT-licensed, native tool tokens, fits an 8 GB GPU), always paired with grammar constraints; go smaller and you should switch to a fine-tuned single-purpose specialist rather than expect general tool-calling.

One caution the table makes explicit: Cohere’s entire Command lineup is capable but non-commercially licensed, a production dead end despite excellent RAG-plus-tools behavior. For the models themselves — VRAM, quantization, general specs — see the local LLM model database; size the hardware with the VRAM calculator; check what phones home with the telemetry audit; and for the speech layer of an agent, the voice-AI database. Start at the Local LLM Canada hub. And to see how these tool-calling scores sit alongside coding, reasoning, vision and our own first-party hardware-design results, zoom out to the LLM Benchmark Hub. And re-check every score here after August 2026 — in this corner of the stack, a dataset without dates is a dataset lying to you.