LLM Benchmark Hub — How AI Models Are Measured, by Capability
How AI models are measured, organized by capability — the four categories the frontier labs report (Coding, Agentic, Reasoning & Knowledge, Vision) plus the two that matter most to a hardware shop: Hardware & PCB Design and Real-World Tasks. A living map that links each benchmark’s live leaderboard, anchored by D-Central’s own Konduit KiCadBench results.
Quick answer
AI models are measured by capability, and the categories the frontier labs report — Coding, Agentic, Reasoning & Knowledge, and Vision — leave out the two that matter most to a hardware shop: can a model actually design electronics, and can it do real, long-horizon work in the world. This hub organizes the field into all six. For the four general categories it is a taxonomy, not a scoreboard: each benchmark links its live leaderboard, because frontier scores move monthly and freezing them would just publish numbers that are wrong by next week. The one category where we publish hard results is Hardware & PCB Design — because D-Central built both the benchmark (Konduit KiCadBench) and the KiCad MCP tool it runs on, and that data exists nowhere else.
Hardware design is the least-measured frontier in AI evaluation. On our first-party Konduit leaderboard, Grok 4.5 leads at 0.973 composite as the only model to clear every live board-build task; across the EE literature the biggest lever is tool-in-the-loop feedback (SPICE/ERC), which lifts solve rates from ~15% to ~91% — more than model size. Treat the Konduit numbers as a directional preview, not a fab-ready claim.
This is a map of how models are measured, not a frozen scoreboard. For the four general categories, each benchmark links its live leaderboard — the always-current source — because frontier scores change monthly. Hard numbers appear only under Hardware & PCB Design, where the data is our own.
Coding
Can the model write and, more importantly, FIX real code — repo-level bug fixes whose patches must pass the project's own tests, competitive-programming under contamination control, and correct diff-format editing.
SWE-bench Verified
publicSWE-bench team (Princeton)
500 human-validated real GitHub issues; scores the % where the model's patch makes the repository's own test suite pass. The default "can it fix real code" bar.
The industry's headline agentic-coding number. Live leaderboard — scores move with every model release.
LiveCodeBench
publicLiveCodeBench
Contamination-resistant competitive-programming tasks collected continuously over time: code generation, self-repair, and test-output prediction on problems published after a model's training cut.
Designed so a model can't have seen the problems in training — the honesty control for coding scores.
Terminal-Bench
publicTerminal-Bench
Agentic command-line tasks in a sandboxed shell — build, debug, sysadmin — where the model must operate a real terminal to completion.
On the Kimi K3 card as "Terminal-Bench 2.1." Measures tool-driving, not just code authoring.
Aider Polyglot
publicAider
225 of the hardest Exercism exercises across C++, Go, Java, JavaScript, Python and Rust; tests correct DIFF-format code editing, not greenfield generation.
Edits must apply cleanly to existing files — a stricter, more real-world bar than write-from-scratch.
SciCode
publicSciCode
Research-grade scientific coding problems drawn from real papers across physics, chemistry, biology and materials science.
Used as a component of Artificial Analysis's Intelligence Index.
Agentic
Can it reliably DRIVE tools and multi-step workflows — call functions correctly, recover from failure, complete tasks across real websites and APIs. Reliability across repeated attempts (pass^k) matters more here than a single lucky run.
τ-bench (tau-bench)
publicSierra
Tool-agent-user dialogue in retail and airline domains; reports pass^k — reliability across k repeated attempts — not just a single pass@1. Consistency is the whole point.
The benchmark that made "does it work EVERY time" a first-class metric for agents.
τ²-bench (tau2-bench)
publicSierra
Dual-control extension of τ-bench where the USER also takes actions in the environment, forcing genuine coordination rather than one-sided tool use.
Harder successor; the telecom split (τ2-Telecom) is a common heavyweight agentic probe.
UC Berkeley (Gorilla)
Accuracy of function/tool calling across single, parallel, and multi-turn calls plus irrelevance detection (knowing when NOT to call a tool).
The benchmark of record for raw tool-calling. See our companion Agent-Capability Database for the open-weight cut.
GAIA
publicMeta / Hugging Face
466 real-world assistant questions that require web search and tool use, across three difficulty levels; easy for humans, hard for models.
Tests whether an assistant can actually go and find things, not just recall them.
WebArena
publicCarnegie Mellon
Self-hosted, fully functional replicas of real websites (e-commerce, forum, CMS, GitLab); scores task success on multi-step web actions.
Runs against real running sites, so success means the task actually completed end to end.
BrowseComp
publicOpenAI
Hard web-browsing research questions whose answers are buried and require persistent, multi-hop browsing to locate.
On the Kimi K3 card (reported 90.4). A deep-research stress test.
Reasoning & Knowledge
Graduate-level, Google-proof questions and competition math — probing genuine reasoning and expert knowledge rather than retrieval. The hardest of these are designed to sit near the ceiling of what frontier models can do.
GPQA Diamond
publicGPQA authors
198 "Google-proof" graduate-level physics, chemistry and biology multiple-choice questions written by domain PhDs to resist quick web lookup.
A staple hard-knowledge probe on every serious 2026 leaderboard.
MMLU-Pro
publicTIGER-Lab
Harder successor to MMLU: ~12,000 questions with 10 answer options and a heavier reasoning weighting to reduce guess-luck.
The reasoning-weighted broad-knowledge baseline once plain MMLU saturated.
Humanity's Last Exam (HLE)
publicCAIS + Scale AI
~2,500 extremely hard expert questions across ~100 subjects, deliberately built to sit near the ceiling of frontier-model capability.
Named for the ambition: the hardest closed-ended exam that still has verifiable answers.
AIME 2025
publicMAA (AMC)
American Invitational Mathematics Examination — 15 integer-answer problems; the standard hard competition-math probe. Use the current year's set to avoid contamination.
Cheap to state, brutal to solve; a clean signal of step-by-step math reasoning.
ARC Prize Foundation
Novel abstract grid puzzles testing fluid reasoning and generalization to unseen patterns — knowledge doesn't help, only reasoning does.
The benchmark specifically built to be easy for humans and hard for pattern-matchers.
Vision
Multimodal understanding — reading diagrams, charts, documents and figures, and reasoning over what it sees, not just captioning it. Harder variants strip the text shortcuts so the model must actually look.
MMMU
publicMMMU team
College-level multimodal questions across 30 subjects, mixing diagrams, figures, tables and text — the broad multimodal-understanding baseline.
Reasoning over images at expert breadth, not object recognition.
MMMU-Pro
publicMMMU team
Harder MMMU variant that adds vision-only-input questions, defeating models that shortcut through the text and ignore the image.
On the Kimi K3 multimodal card. Forces the model to actually look.
MathVista
publicMathVista team
Visual mathematical reasoning over figures, plots and geometry — reading a chart or diagram and doing the math it implies.
Where vision meets quantitative reasoning; a frequent separator between strong and weak VLMs.
DocVQA
publicDocVQA
Question answering over scanned document images, requiring layout understanding and OCR-grounded reasoning.
The practical "read this paperwork and answer" test.
ChartQA
publicChartQA authors
Question answering over charts and graphs — visual plus numeric reasoning to extract values and trends from plotted data.
Directly relevant to reading the kind of performance charts these very benchmarks produce.
ZeroBench
publicZeroBench
A deliberately near-impossible hard-perception benchmark where current frontier VLMs score at or near zero — a headroom probe for the next generation.
Built to NOT be saturated; useful precisely because nobody is acing it.
Hardware & PCB Design D-Central addition
The frontier almost nobody benchmarks: can a model design electronics — read a datasheet, generate a schematic, build and route a board, pass ERC/DRC? This is D-Central's home turf, and the one category on this page where we publish our OWN first-party leaderboard.
Konduit KiCadBench — our first-party leaderboard
Autonomous board-build through DCENT_Konduit, our own KiCad MCP server. Composite = 0.6 live KiCadBench (ERC/DRC-gated) + 0.4 EE replica suites. As of 2026-07-09.
| Model | Composite | KiCadBench | EE suites | Context | Notes |
|---|---|---|---|---|---|
| Grok 4.5xAI | 0.973 | 5/5 | 0.93 | 2M | The only model to clear all five live KiCad build tasks under v2 structural gates AND score 0.93 on the EE suites. Full coverage. |
| Claude Opus 4.8Anthropic | 0.680 | 3/5 | 0.80 | 1M | Strongest Claude on live board-build and the most tool-efficient (19 calls/task avg); the USB-C power-entry task fails its required VBUS net labels. |
| Claude Haiku 4.5Anthropic | 0.587 | 2/5 | 0.87 | 200K | Punches above its weight on the EE suites (0.87) for a small, fast model; struggles on multi-constraint board builds. |
| Claude Sonnet 5Anthropic | 0.373 | 0/5 | 0.93 | 1M | Tied for top on the EE replica suites (0.93) but scored 0/5 on live builds — incomplete plan rounds under a subscription session limit plus stricter v2 gates. Reads as a harness-coverage gap, not a capability floor; flagged for re-run. |
| Claude Fable 5Anthropic | — | n/a | 1.00 | 200K+ | Perfect 5/5 (100%) on the EE replica suites but UNRANKED — the live board-build leg never ran (session limit hit mid-run). Partial coverage, not comparable to full-coverage rows until it completes. |
🔴 The Konduit leaderboard is an early PREVIEW: five live KiCad build tasks plus five clean-room EE suites, run through subscription-based harnesses. Evaluator v2 gates a pass on the board's declared structural checks (nets, components, outline, netclass, hierarchy, ERC/DRC), not bare exit codes. Preview scores are NOT fab-ready claims — every board still needs human EE review before manufacturing. The task set is small and growing; treat rankings as directional and expect movement as we add tasks and re-run incomplete rows.
Documented hardware-design benchmarks
Konduit KiCadBench
first-partyD-Central (first-party)
End-to-end autonomous PCB and schematic design through a LIVE KiCad MCP server: the model authors the board with real tools, the harness executes it on a real KiCad 10 engine, and ERC/DRC plus structural checks (nets, components, outline, netclass, hierarchy) gate the pass. 4-tier difficulty from schematic-only to a full fabrication package.
The only benchmark on this page that measures AGENTIC hardware design — building a board with tools, not answering trivia about one. Our leaderboard is below. Grok 4.5 currently leads at 0.973 composite; see the full table.
PCB-QA
researchSrinivasan, Tan, Turnbull & Pearce (arXiv:2606.23704)
480 question-answer pairs from 8 open-source hardware projects, covering component connections, datasheet reading and SPICE simulation. The first PCB question-answering benchmark.
Verified scores (Jun 2026): Gemini 3 Flash 93% (JSON format) · Claude Sonnet 4.6 90% · GPT-5.4 Nano 88% · Llama 3.3 70B 84%. JSON prompting beats native-KiCad format.
PCBBench / PCBSchemaGen
researchHZou9 et al. (arXiv:2602.00510)
227 real-IC schematic-generation tasks across 22 circuit domains, verified by a 5-layer deterministic structural checker (PCBs have no unit tests, so correctness is proven structurally).
Verified scores (Jan 2026): GPT-5.4 94.0% · Gemini 3.1 Pro 94.3% · DeepSeek V3.2 81.6% · Gemma-4-31B 81.3% (best local) · MiniMax M2.5 76%. The "hard" tier is the real separator.
SMPS Benchmark
researcharXiv:2507.10639
269 switched-mode power-supply design tasks (parameter tuning, topology adaptation, optimization) with SPICE simulation feedback available in the loop.
Headline result (Jul 2025): SPICE feedback + reasoning lifts solve rate from 15% to 91% (o3 + RAG + SPICE 90.7%). Tool-in-the-loop beats raw model size for EE work.
CircuChain
researcharXiv:2602.15037
Diagnostic circuit-analysis benchmark: 100 Control/Trap problem pairs across 5 canonical topologies, built to separate genuine competence from pattern-matched compliance.
Trap pairs expose models that recite the textbook answer without actually analysing the circuit in front of them.
PowerAgentBench-Dyn
researcharXiv:2606.20401
Agentic AI on power-system dynamic-analysis tasks — grid and power engineering rather than PCB design.
One of the five EE replica suites in our Konduit composite (poweragentbench_dyn).
RadioBench
researcharXiv:2606.01862
The first benchmark for LLM-driven radio-signal generation — RF and communications design.
One of the five EE replica suites in our Konduit composite (radiobench).
CVDP (Verilog)
publicNVIDIA
Digital-hardware / Verilog design — the RTL side of hardware, adjacent to our analog and PCB focus.
Included for completeness: the digital-design family, distinct from schematic/PCB work.
Real-World Tasks D-Central addition
Long-horizon, open-ended tasks from the actual world — operating a desktop, a phone, or a video game; doing a real freelance job priced in real money. The messiest, least-saturated, most honest test of whether an "agent" is actually an agent.
OSRSBench
emergingCommunity / emerging
An agent's ability to play Old School RuneScape — a long-horizon, GUI-driven video game demanding navigation, resource management and multi-step questing over extended play.
🔴 EMERGING benchmark: we list it because it exemplifies the real-world direction, but treat its methodology and any scores as unconfirmed until its own repo/site is cited. No numbers published here.
OSWorld
publicOSWorld team
369 real computer-desktop tasks across Ubuntu and Windows applications; open-ended GUI computer-use with execution-based scoring on the actual resulting system state.
The leading general computer-use benchmark — operate a real desktop, get graded on the result.
AndroidWorld
publicGoogle Research
116 tasks across 20 real Android apps; mobile GUI agents rewarded on the app's actual resulting state.
The phone equivalent of OSWorld: real apps, real state-based reward.
Voyager / MineDojo
publicMineDojo
Open-ended embodied agents in Minecraft — skill acquisition, exploration and long-horizon goal pursuit across thousands of open tasks.
The canonical open-ended embodied-learning environment; lifelong skill-building, not a fixed score.
TheAgentCompany
publicTheAgentCompany
An agent completes tasks inside a simulated software company — coding, browsing, internal chat, HR workflows — scored on the fraction of real job workflows finished.
"Can it do a real knowledge-worker's job" — the most economically-framed of the real-world suites.
Hopkins et al.
Long-horizon automation and planning in the game Factorio — open-ended factory throughput requiring deep sequential planning.
Perhaps the purest long-horizon planning stress test: build a self-expanding factory.
SWE-Lancer
publicOpenAI
1,400+ real freelance software-engineering gigs priced in real US dollars; models are scored on earnings — did the delivered work pass the client's acceptance test.
Real jobs, real money as the yardstick — the sharpest bridge from benchmark to economic value.
Why a map, not a scoreboard. This page is a MAP of how AI models are measured, not a frozen scoreboard. Frontier scores change with almost every model release, and the public leaderboards are the authoritative, always-current source — so for the Coding, Agentic, Reasoning and Vision categories we link each benchmark's LIVE leaderboard rather than freeze a number that would be wrong within weeks. Publishing stale scores as if current is exactly the kind of thing this site refuses to do.
Why we own the Hardware numbers. The exception is Hardware & PCB design. There we publish hard numbers, because the data is ours: D-Central built both the benchmark (Konduit KiCadBench) and the tool it runs on (DCENT_Konduit, our KiCad MCP server). No general LLM leaderboard measures whether a model can actually design a circuit board — read a datasheet, generate a schematic, place and route it, pass ERC/DRC. We do, and the results below exist nowhere else.
Open data (CC BY 4.0): CSV · JSON · API: /wp-json/dc/v1/llm-benchmarks
Related products, repair, and setup paths
- self-hosted AI for Bitcoiners hub
- plebs guide to self-hosted AI
- install Ollama in 10 minutes
- LM Studio vs Ollama vs llama.cpp
- connect local AI to Home Assistant and Obsidian
- self-hosted AI troubleshooting
- repurpose mining hardware into an AI hashcenter
- local AI model leaderboards
Last reviewed July 18, 2026.