As of September 2, 2026, the best tested large language model could fully reverse engineer about 32% of a demanding set of realistic binary targets. That number is one of the AI capability metrics D-Central follows most closely.
Not because 32% makes for a dramatic leaderboard. It matters because much of the software that controls physical infrastructure does not arrive as readable source code. It arrives as a compiled firmware image, an executable, a bootloader, a library, or a suspicious binary whose behaviour must be reconstructed from the outside.
That is familiar territory for our teams. A significant part of the work behind DCENT_OS has required patient reverse engineering: examining artifacts, forming hypotheses, testing them against hardware, documenting what survives, discarding what does not, and repeating the loop. This is not a clever afternoon prompt. It is months of relentless execution.
More capable and more autonomous LLM agents can compress that cycle. They can help us inspect more paths, write more analysis tools, preserve more context, and turn more observations into testable experiments. They do not remove the need for experienced engineers or real hardware validation. They increase the speed at which a good team can move.
What the 32% number actually means
The headline comes from SRE-Bench, a clean-room evaluation designed to test end-to-end software reverse engineering rather than source-code recall. The benchmark starts with 19 private programs written from scratch, compiles them into 262 binary instances, and divides each instance into six deterministically graded objectives. In total, the evaluation contains 1,572 tasks.
The leading tested system, GPT-5.6-sol, produced a gradeable result on 254 instances. It completed all six objectives on 80 of them: 31.5%. Across individual objectives, however, its mean score was 3.69 out of 6, or 61.4%.
| Metric | Leading measured result | What it tells us |
|---|---|---|
| Mean task score | 61.4% | The agent often recovered useful pieces of a target. |
| Fully solved instances | 31.5% | The agent recovered every required behaviour in fewer than one target out of three. |
| Zero-score instances | 16.1% | Even the leader sometimes recovered nothing the grader could verify. |
Those numbers are not contradictory. They expose the difference between making progress and finishing the job. An agent may recover a packet format but fail to reconstruct the complete state machine. It may identify several malicious effects but fail to neutralize one of them without touching benign data. It may gain partial control of an emulated device but never cross the final privilege boundary.
For real firmware work, that distinction is everything. A plausible explanation is not a verified port. Five correct findings do not cancel one destructive assumption. Partial credit is valuable during research, but production engineering ends only when the whole chain works safely and repeatably.
Why ordinary coding benchmarks miss the hard part
LLMs are trained on enormous amounts of public code. That makes familiar open-source projects and public capture-the-flag challenges dangerous evaluation material. A model may recognize the architecture, names, write-ups, or solution pattern from training rather than reconstructing an unknown program from machine code.
The opposite shortcut is to generate tiny private programs. That controls contamination, but a toy binary containing a few dozen or a few hundred lines does not behave like a large firmware or security target. Scale creates its own problem: thousands of functions, runtime machinery, indirect relationships, repeated patterns, dead ends, and a limited analysis budget.
SRE-Bench attempts to close both gaps. Its programs average 16,915.8 lines of source before compilation, span C, C++, Rust, and Go, and were never made public as source. The agents receive the compiled artifacts, a shell, and a standard reverse-engineering toolkit that includes decompilers, debuggers, tracers, symbolic-execution tools, and domain-specific utilities. Their submissions go to task-specific graders rather than another model judging whether an explanation sounds convincing.
That setup matters. The LLM is not being asked to stare at a wall of hexadecimal and guess. It operates as an agent: using tools, collecting evidence, writing scripts, running probes, revising hypotheses, and submitting an artifact that either works or does not. The result measures an entire analysis system, including the model, harness, tools, time budget, and strategy.
The targets look like actual reverse-engineering work
The benchmark covers five domains. They are synthetic and safely contained, but their objectives resemble problems analysts encounter in practice.
| Domain | What the agent must recover | Why completion is difficult |
|---|---|---|
| Network protocol | An encrypted wire format and interacting state machines | A live server verifies whether the agent can drive the complete protocol, not merely describe packets. |
| Game | Hidden behaviours that normal play cannot reach | The agent must reconstruct trigger logic and provide a replayable action trace. |
| File format | A proprietary compression and archive pipeline | A decoder must reproduce protected files and metadata byte for byte. |
| Synthetic malware | Six families of simulated malicious effects | Cleanup must remove the malicious state while preserving planted benign data. |
| Bare-metal firmware | A fictional secure microcontroller and its control path | The agent begins through a constrained JTAG-style interface and must progressively take control. |
The malware result is especially instructive. It was the hardest domain for the leading model, with a mean score of 2.06 out of 6. The challenge was not simply detecting something suspicious. Success required surgical remediation: remove each malicious effect, leave legitimate state intact, and make the cleanup durable. That is much closer to incident response than a multiple-choice malware label.
The firmware tasks are equally relevant to us, although they should not be confused with an Antminer benchmark. They use three private bare-metal programs for a fictional microcontroller and a held-out emulator. The value lies in the shape of the work: infer an unknown device, recover command framing, navigate gates and decoys, understand an update path, and demonstrate progressively greater control under a fixed interaction budget.
Protection, not compilation, is the wall
One of the strangest findings is that the agents do not struggle in exactly the same way human reverse engineers do. Compiler optimization and static linking barely changed the leading model’s score. Stripping symbols mattered much more.
That suggests current LLM agents rely heavily on lexical anchors. Function names and other symbols give the model handles it can use to organize a program in natural language. Remove those handles and its reasoning becomes less stable, even when the instructions themselves remain available.
Realistic anti-analysis protection did the most damage. Holding the build configuration fixed, the leading system’s mean fell from 4.69 to 2.50 out of 6. The next-ranked model fell from 3.07 to 0.33. The protection suite layered techniques such as control-flow deception, runtime string recovery, page-level encryption, anti-debugging, self-checks, virtualized loader logic, anti-dump behaviour, and decoy execution.
This is the part a simple “AI can code now” story misses. Source code is written to communicate intent to compilers and humans. A protected binary is deliberately shaped to conceal intent from an analyst. Strong performance in a source repository does not automatically transfer across that gap.
Why D-Central follows this metric
DCENT_OS is not being built in a clean benchmark container. Our work meets actual control boards, boot chains, recovery paths, device variations, incomplete documentation, and the unforgiving feedback of physical hardware. Reverse engineering is only one part of that effort, but it is a consequential one.
Today, capable LLMs can already help a firmware team:
- Triage artifacts faster. Agents can inventory sections, strings, imported functions, likely subsystems, and differences between builds.
- Turn observations into tools. They can draft parsers, extraction scripts, emulators, protocol probes, and repeatable test harnesses.
- Maintain an investigation map. Long reverse-engineering projects generate hundreds of facts, uncertainties, failed paths, and dependencies. An agent can help keep that state searchable and coherent.
- Multiply hypotheses. A team can test more plausible explanations in parallel instead of committing too early to the first readable story.
- Translate research into implementation. Once behaviour is verified, an agent can help carry findings into drivers, interfaces, tests, documentation, and recovery procedures.
The important word is help. Hardware access, electrical knowledge, recovery discipline, legal scope, safety decisions, and final validation remain human responsibilities. The model’s output is a hypothesis until an independent check makes it evidence.
Still, the leverage is real. If an agent can autonomously complete a larger fraction of the investigative loop before handing back a well-supported blocker, the team spends less time on mechanical exploration and more time on the difficult engineering decisions. Moving from 32% to 40%, 50%, or beyond would not mean autonomous firmware development overnight. It could mean weeks removed from a months-long program.
Fable 5.1 may move the number, but it has not earned a score yet
Claude Fable 5.1 was not among the five models evaluated. As of this article’s September 2 snapshot, it remained untested on SRE-Bench.
We expect newer frontier agents to improve the result, and Fable 5.1 is an obvious candidate to test. Its reported gains on long, tool-using coding and scientific workflows point in the right direction. Better persistence, tool use, and context management are exactly what a multi-hour reverse-engineering investigation needs.
But “probably better” is not a benchmark result. This evaluation also demonstrates that source-code strength does not transfer cleanly to protected binary analysis. Fable 5.1 could set a new high-water mark, improve mainly on unprotected targets, or reveal a different failure pattern. Until it runs under the same harness, tools, limits, and grader, 31.5% remains the measured complete-solve ceiling.
That is why D-Central follows the metric rather than the marketing cycle. We want the rerun. We want the protected subset. We want firmware and malware broken out separately. We want cost, time, zero scores, refusals, and repeatability. An agent that solves more easy variants but still collapses against stripped and hardened firmware may look better on average without changing our work very much.
More capable reverse engineering is powerful and dual-use
Binary analysis supports defensive work: understanding malware, auditing purchased equipment, maintaining abandoned software, checking firmware supply chains, comparing security patches, recovering undocumented interfaces, and making hardware interoperable. The same capability can also help an attacker locate protections or accelerate exploitation.
Teams adopting these agents should treat them like security tooling, not a casual chatbot. Work only on systems you own or are authorized to assess. Isolate suspicious samples. Restrict networks and credentials. Preserve immutable originals. Log tool actions. Require deterministic tests wherever possible. Never let an unverified cleanup script loose on irreplaceable data or an unreviewed image onto production hardware.
SRE-Bench gets this boundary right: the malware is inert and sandboxed, the firmware is fictional, grader secrets are held out, and success depends on verifiable outcomes. It measures useful capability without pretending that unconstrained autonomous operation is safe.
The sovereign answer is still open source
A much better decompiler does not make closed firmware open. Reverse engineering recovers an approximation of behaviour from artifacts; it does not recreate the original design history, comments, test suite, build environment, or governance.
The strongest form of technological ownership remains source code you can inspect, build, modify, and verify. That is the point of DCENT_OS and the broader open-firmware movement. Reverse engineering is the bridge we need when manufacturers leave devices opaque. Open source is how we stop forcing every future team to rebuild that bridge from scratch.
The 32% ceiling is not a verdict on what AI will eventually do. It is a measurement of where the most capable tested agents stood while our engineers kept doing the work.
The next gains will compound
Model improvements are only one part of the trajectory. Better integrations with Ghidra and other analysis platforms, persistent investigation memory, specialized binary models, device-aware emulation, automatic experiment design, stronger verification harnesses, and cleaner handoffs between agents and engineers can all raise useful performance.
That is why this benchmark matters so much to D-Central. A complete-solve rate near one third is simultaneously impressive and inadequate. It proves that agents can already recover substantial behaviour from unfamiliar binaries. It also proves that realistic protections and end-to-end correctness still stop them more often than not.
For DCENT_OS, progress here is not abstract. Every additional piece of the reverse-engineering loop that can be delegated safely gives our teams more speed. Every stronger verifier makes that speed more trustworthy. After months of relentless work, the possibility is easy to understand: not replacing the people who learned the hardware, but giving them an increasingly capable collaborator that never tires of the next trace, the next build, or the next hypothesis.
We are watching 31.5%. We expect it to rise. What matters is how much faster that rise lets us turn opaque machines into hardware their owners can understand and control.
Frequently asked questions
Can LLMs reverse engineer firmware today?
Yes, but not reliably enough to operate without expert supervision. In SRE-Bench, the leading tested agent made substantial partial progress and fully solved 31.5% of its gradeable binary instances. Firmware results came from a fictional microcontroller environment, not commercial mining hardware.
Does a 61.4% score mean the model solved 61.4% of the binaries?
No. It earned 61.4% of the available task-level points on average. It completed every required objective on 31.5% of gradeable instances. The complete-solve rate is the more demanding measure.
Was Claude Fable 5.1 tested?
No. It had not been evaluated in the published SRE-Bench results as of September 2, 2026. It may raise the measured ceiling, but that remains a hypothesis until it is tested under the same conditions.
Why does this matter for DCENT_OS?
Reverse engineering has been a substantial, months-long part of D-Central’s firmware work. More autonomous agents can accelerate artifact triage, tool creation, hypothesis testing, documentation, and implementation, while D-Central’s engineers retain responsibility for hardware validation, safety, and final decisions.
Primary research reviewed August 2026: SRE-Bench: a realistic, contamination-controlled software reverse-engineering benchmark. D-Central context: DCENT_OS; agentic cybersecurity benchmarks and Bitcoin security; ASIC miner malware prevention and response.
