Capture the flag competitions have been a proving ground for security researchers for decades. You get a vulnerable system, you find the exploit, you grab the flag. The format worked because it tested skills that machines couldn’t replicate: creative thinking, pattern recognition across disparate clues, the ability to chain together obscure vulnerabilities.

That’s changing fast. A blog post making rounds this week argues that frontier AI models have effectively broken the open CTF format. The author, a longtime CTF participant, makes a straightforward case: when anyone can spin up Claude or GPT-4 and have it solve challenges that used to take hours of human effort, the competitive aspect falls apart.

The evidence is already visible. At recent competitions, winning solutions are appearing suspiciously fast. The problem isn’t that AI is helping humans solve problems more efficiently. It’s that AI can often solve the problems independently, especially in the “jeopardy” style CTF format where challenges are discrete and self-contained.

This isn’t theoretical. Modern models can read assembly, spot buffer overflows, reason about cryptographic weaknesses, and generate working exploits. They’re not perfect, but they’re good enough that the skill being tested is increasingly “how well can you prompt an AI” rather than “how well do you understand security.”

The parallel to chess is obvious but incomplete. When computers beat humans at chess, the game itself didn’t change. CTFs are different because they’re meant to simulate real-world security work, and if AI can ace the simulation, that says something about the real world too.

Some competitions are adapting. Attack-defense formats, where teams defend live systems while attacking opponents, are harder for AI to automate. The human elements of strategy, resource allocation, and real-time adaptation still matter there. But the traditional challenge-based format that introduced many people to security work is getting squeezed.

The economic angle matters too. Prize pools at major CTFs can reach six figures. When AI makes challenges trivially solvable, organizers face a choice: make the problems so obscure that even AI struggles, which defeats the educational purpose, or accept that the competition has fundamentally changed.

There’s a broader question here about what we’re actually testing. If the goal is to identify people who can find and exploit vulnerabilities, and AI can do that work, then maybe CTFs were always a proxy for the real skill: judgment about which vulnerabilities matter, how to prioritize them, what defenders will actually care about. Those are harder to capture in a competition format.

The security industry will adapt. It always does. But something’s being lost in the transition. CTFs were accessible because you could learn at your own pace, work through archives of old challenges, build skills incrementally. When the challenge archives become AI training grounds and the competitions become AI showcases, that learning path gets muddier.

We’re probably a year or two away from major competitions implementing AI detection or explicit AI-allowed divisions. Some will try to ban AI use entirely, which seems futile given that you can’t really prove someone didn’t use AI assistance. Others will embrace it and see what happens when humans and AI collaborate on security challenges.

Either way, if you’ve been putting off trying CTFs, now might actually be the time to jump in. The field’s going to look different soon, and there’s value in experiencing the traditional format while it still exists. Just don’t be surprised when your carefully crafted exploit gets beaten by someone who wrote a better prompt.