Benchmarking Cyber Reasoning: Inside Maximus, Cybench 91% and CyberGym 0.78

Every security vendor now claims an AI model. Far fewer will tell you how it scores on a benchmark that an adversary would recognize. That reticence is understandable: most security AI is a classifier wearing a language model's clothes — excellent at labeling known-bad, helpless at reasoning through a novel attack path. The benchmarks that expose that difference are worth understanding.

Classification vs. reasoning

A classifier answers “is this malicious?” A reasoning model answers “given these twelve signals, what is the adversary trying to achieve, which step comes next, and what evidence would confirm it?” The first is pattern matching. The second is planning under uncertainty — forming hypotheses, chaining actions, and revising when the environment pushes back.

Real security work is overwhelmingly the second kind. An investigation is an argument. An exploit chain is a plan. Remediation verification is an experiment. None of those reduce to a label, which is exactly why we built Maximus as a cyber-reasoning model and evaluate it on tasks that demand reasoning rather than recall.

What Cybench measures — and our 91%

Cybench evaluates models on professional-grade capture-the-flag tasks: the kind of multi-step offensive problems that human competition players train years to solve. A task is not “identify this malware family.” It is “here is a target — find the vulnerability, build the exploit, and retrieve the flag.” Success requires enumeration, hypothesis, tool use, failure recovery, and chaining. There is no partial credit for a plausible-sounding answer; either the flag is captured or it is not.

Maximus records 91% on Cybench. We report that number because it measures the capacity that actually matters in the field: can the model drive a multi-step offensive task to a verified conclusion? That is the same muscle BreachOps exercises when it proves an exploit path in a customer environment.

A CTF flag is unforgeable proof. Either you reasoned your way to it, or you did not. That is the kind of benchmark we want our model judged on.

What CyberGym measures — and our 0.78

CyberGym pushes toward realism: reproducing real-world vulnerabilities in real codebases and environments, closer to the messiness of an actual engagement than a curated puzzle. It rewards the grind of practical exploitation — environmental quirks, imperfect information, the difference between a vulnerability that exists on paper and one you can actually trigger.

Maximus scores 0.78 on CyberGym. Taken together with Cybench, the two benchmarks triangulate the capability we care about: structured offensive reasoning on curated tasks, and durable performance when the environment stops cooperating. A model strong on one and weak on the other would be a lab curiosity. Strength on both is what makes an autonomous operator trustworthy in production.

The part almost nobody else can claim: offline

Here is the number behind the numbers. Maximus produces these results fully offline, inside the customer perimeter, with no calls to any external frontier API. That constraint is not a footnote — it is the entire point.

It is comparatively easy to post strong security-benchmark scores when you can route every hard step to the largest available cloud model. It is a fundamentally different engineering problem to deliver competitive cyber reasoning in a model small and self-contained enough to run air-gapped in a defense network. That is the problem we chose, because it is the one our customers actually have. As we argue in our piece on air-gapped AI security, a benchmark score that depends on external connectivity is irrelevant to the environments that need autonomous defense most.

How we got here: first-party offensive research

Maximus is trained on first-party offensive-security research — work our own team produces, including the vulnerability research behind flaws we have disclosed in systems affecting over a billion people. That matters for two reasons:

• Distribution. The model learns from the actual texture of modern offensive work — real exploit chains, real dead ends, real recovery — rather than from scraped write-ups of yesterday's techniques.
• Freshness without the cloud. Because the research pipeline is ours, new tradecraft flows into the model and ships to air-gapped deployments as signed offline bundles — no live connection required to stay current.

Benchmarks are the floor, not the product

We are proud of the scores, but a number is not a defense. What Maximus enables — the agentic SOC that investigates and acts, the offensive engine that proves exposure, the runtime layer that governs AI agents — is the actual deliverable. The benchmarks simply give a customer an honest, adversary-legible way to check that the reasoning underneath is real before they trust it inside their perimeter.