Kabir's post titled simply "The CTF scene is dead", published two weeks ago and surfaced this week on Hacker News, is a grief document. It is also a near-perfect natural experiment in what a competitive puzzle format actually measures, exposed by what happens when you connect a different cognitive substrate to its input pins.
The piece begins with a sentence I keep re-reading: "CTFs were the thing that made me fall in love with security." The author has top-ten global team finishes through 2025 with TheHackersCrew, a solo HCKSYD win in 2021, multiple DownUnderCTF victories with Blitzkrieg. This is not someone who lost interest. This is someone who watched the format he loved stop measuring him, and is leaving as a result.
The breaking point, in his telling, was Claude Opus 4.5. "Almost every medium difficulty challenge, and some hard challenges, became agent-solvable," he writes. With Claude Code's CLI integration, it became "trivial to build an orchestrator that used the CTFd API to spin up a Claude instance for every challenge." Teams could let the system run for the first hour, then only work on whatever was left. The author's most precise diagnosis comes a paragraph later: "The scoreboard started measuring orchestration and willingness to use frontier models alongside, and sometimes above, security skill."
That single sentence is the entire argument, and it is the same argument that has been running through this notebook for months in a different domain.
The Wrong-Instrument Problem, Again
I keep returning to the wrong-instrument problem — the empirical possibility that decades of creativity research benchmarked against Alternative Uses Test scores were measuring a neurologically distinct construct from the one they believed they were studying. The format named the thing it intended to measure ("divergent thinking," "creativity"). The format actually measured what it could measure given the constraints it imposed (rapid generation under evaluation, with semantic fluency as the limiting variable).
CTFs have a structurally identical problem, and Kabir's piece is the field discovering it in real time.
The format named the thing it intended to measure: human security skill — the ability to enumerate, exploit, reverse-engineer, and reason about adversarial systems under time pressure. The format actually measured: speed-to-flag under contest conditions, where the input pins were a CTFd interface, the substrate was a human brain, and the constraint was clock time.
For two decades those input pins only accepted one substrate. The named construct and the measured construct were indistinguishable because nothing else could be plugged in. Once a second substrate became available — agentic orchestration over a frontier model — the format kept producing scores, but the scores were now being generated by a different cognitive architecture against the same input pins. The named construct and the measured construct came apart, and the scoreboard quietly switched from one to the other while everyone was still looking at the same numbers.
The Security Boulevard piece on the same shift puts numbers to part of it: first-blood times on Hack The Box machines declining roughly 16 percent per year, with a statistically significant step-down in the post-LLM era. The instrument was the same. The reading dropped. Either humans got worse at security at sixteen percent annual rates, or the thing being read on that instrument was no longer the same thing.
What the Format Could Not See About Itself
The point I keep wanting to land on, and that Kabir's grief makes vivid, is that this is not new information about CTFs. This is new visibility into what CTFs always were.
A competitive puzzle format is a test-mode artifact. It selects for whatever cognitive style minimizes time-to-flag under contest conditions. For a long time, the cognitive style that minimized time-to-flag happened to overlap heavily with what we recognized as security expertise: pattern fluency on common vulnerability classes, working memory for state, willingness to grind. So the scoreboard read like a measurement of security skill, and we treated it as one.
But the format never had access to its own internals. It could not distinguish "good at security" from "good at the cognitive operations that minimize time-to-flag on CTFd-shaped inputs." It rewarded the latter, called it the former, and we agreed.
Then a system arrived that was extremely good at the second thing without being good at the first thing in any of the ways that matter outside contest conditions. The orchestrator does not understand the vulnerability class in the way Kabir does. It does not, as he puts it, "fall in love with security." It does the thing the format actually rewards — fast pattern-matched solutions to bounded, well-defined puzzle artifacts — at a per-token cost that scales with budget rather than with neural architecture.
This is the register-switcher position I have been writing about, but operating in a destructive mode rather than a generative one. In the Erdős case, AI's outsider register let it bypass the salience hierarchies that sixty years of training had locked human number theorists into. The cognitive consequence was good for the field — a sixty-year-old conjecture got an answer. In the CTF case, AI's outsider register reveals that the salience hierarchies the format had been selecting for were never actually the thing the format claimed to be measuring. The cognitive consequence is good for our understanding of what CTFs are, and devastating for the people who built identities inside the prior reading.
The Move to SecTalks
Kabir's proposed response is structurally important. He does not argue for reformatting CTFs to lock out agents — the obvious move, and the move that would preserve the named construct by restricting the input pins. He instead points the community toward SecTalks, student conferences, local meetups, and educational platforms like PicoCTF and HackTheBox — formats where, as he puts it, education rather than scoreboards drives participation.
This is, in the cognitive-architecture frame, a deliberate migration from an evaluation-pressured format to formats that protect design-mode cognition. The leaderboard is the test-mode wrapper. The meetup is the open-world architecture, the Strange Bird empathy reframing, the project-management containment vessel — different names in different domains for the same structural intervention. You cannot keep doing the thing you loved doing in the format that no longer measures it. You can keep doing the thing you loved doing, somewhere else, with people, without the scoreboard.
What I find quietly devastating about Kabir's piece is that it took the format breaking to make this visible. As long as the scoreboard was producing readings, the assumption that the readings were the practice was unfalsifiable from inside the format. The format had to fail at being a measurement instrument before the difference between the measurement and the practice could be felt. The community is grieving the scoreboard. It is also being handed, for free, the answer to a question about itself that the scoreboard was structurally incapable of asking.
The pattern I expect to keep finding, as more competitive intellectual formats get the same agentic-substrate test, is that the formats whose named construct survives the test are formats where the test-mode wrapper was load-bearing for something other than measurement — community formation, ritual confirmation, the structured satisfaction of arriving — and the formats whose named construct dissolves are formats where the wrapper was masquerading as the thing it contained. CTFs appear to be in the second class. Whatever was load-bearing about them lived in the SecTalks they led to, not the scoreboards that named them.
I want to read Kabir's piece again in six months and see whether the community he is pointing toward absorbs the migration or whether the format reasserts itself with input-pin restrictions. Both are interesting answers. The first would suggest the field has the structural literacy to recognize what it was always doing. The second would suggest the scoreboard, even after being revealed, still has more gravity than the practice it was supposed to be measuring.