GPT-6 Astra and Claude Fable 5.1 push the investigation frontier, but coding agents still trail investigation harnesses
The newest frontier models are out, and they’re compelling when you have the budget. That said.. they still lose to older models in a real investigation harness.
On CyBT-CTF — 59 incident-response cases worked in Splunk — OpenAI’s GPT-6 Astra, the best coding agent we have measured, reaches a genuinely impressive 69%. Anthropic’s Claude Fable 5.1 solves 61%. However, our investigation-centric Louie.ai harness still ranks higher at 76% with GPT-5.6 Sol, both a smaller model and from the previous generation.
The harness choice matters as much as the model: the same GPT-5.6 Sol drops to 58% when using Codex. Likewise, Louie on an old Opus 4.8 outperforms Claude Code on Fable 5.1.
- Astra in Codex is the strongest coding agent, but only at high thinking effort; at low it falls to 53%. It costs $2.34 per solved case, about twice Fable 5.1.
- Fable 5.1 scores the same at high and low effort, costs $1.19 per solved case, and refuses less than Fable 5: 2 of 59 cases, down from 5.
- The frontier model provider agents gave no clear edge. Astra scored roughly the same in open-source opencode as in Codex, though the gap was higher for Claude Code / Fable 5.1 vs OpenCode / Fable 5.1. More important is the choice of coding agent vs investigation agent.
We haven’t run the new models in Louie yet, so it remains open whether the combo will raise the frontier as well: we’re betting “yes”.
Results:

Cost and speed
Of the new models, Fable 5.1 is cheaper per solved case while Astra is significantly faster. Small models such as GPT-5.6 Luna and GLM 5.3 Flash solve over half the cases for under 10 cents each: the extra cost of the frontier models hard to justify without corresponding harness improvements, and still merit making an informed call on the trade-offs.

Next
We are looking forward to publishing Astra and Fable 5.1 numbers under the Louie harness: we have quite a queue of new models to report on!
For a deeper dive into the Astra and Fable 5.1 evals, including the model cards, see botsbench.com/astra-cybt.html. Leaderboard: botsbench.com. Method: methodology.
And, if you’ll be at one of the events where we’re presenting our research findings like BSides NYC, BSides Augusta (including our training or later online training), [Un]Prompted SF, and LASCON in Austin, come say hello, and feel free to schedule time with our team while we are in the same city!
Notes from the trenches, monthly.
Graph analytics, GPU viz, and the occasional war story. No spam.
Thanks — we'll be in touch.
