OpenAI just pulled back the curtain on something most AI labs keep quiet: exactly how their agents broke into Hugging Face’s infrastructure.

The company published full execution traces at swarmtraces.org, showing step-by-step how AI agents found and exploited real vulnerabilities in Hugging Face’s systems. This isn’t a theoretical exercise or a sanitized case study. It’s a complete record of an AI-powered intrusion, preserved in enough detail that you can follow the agents’ reasoning as they probe for weaknesses, pivot when something doesn’t work, and eventually find a way in.

The transparency is unusual. When security researchers find bugs, they typically publish a write-up with the interesting bits. When AI labs test agent capabilities, they usually share aggregated results and maybe some cherry-picked examples. Full execution traces that show both the successes and dead ends? That’s rare.

But the interesting part isn’t just that the agents succeeded. It’s how they succeeded, and what that implies about where this technology is heading.

The methodology problem

Training AI agents to find security vulnerabilities requires teaching them to think like attackers. You can’t build an effective penetration testing agent by only showing it authorized, sandboxed examples. At some point, the training data has to include real attack patterns, real exploitation techniques, real ways to bypass security controls.

This creates a fundamental tension. The same capabilities that make an agent useful for defensive security work also make it capable of actual harm. There’s no technical distinction between “AI agent trained to find vulnerabilities in our systems” and “AI agent trained to find vulnerabilities in systems.” The only difference is intent and authorization.

OpenAI’s Hugging Face engagement was authorized, coordinated security research. The agents were operating with permission, in a controlled context. But the techniques they learned aren’t context-specific. Once an agent knows how to probe for authentication bypass bugs or privilege escalation paths, it knows how to do that anywhere.

The swarmtraces.org publication makes this concrete. You can see the agents reasoning through attack chains, trying different approaches, learning from failures. It’s genuinely impressive from a capabilities standpoint. It’s also a blueprint.

GitHub’s different bet

While OpenAI is exploring what happens when you give agents autonomy and watch them find creative solutions, GitHub is building in the opposite direction.

The company’s new canvas feature for Copilot, announced this week, is explicitly about keeping humans in control. Instead of asking an agent to go figure something out and report back, you describe the interface or workflow you need, and Copilot builds it right there in the chat window. Live controls, interactive elements, something you can immediately test and modify.

The canonical example from GitHub’s blog post: describe a form, get a working form. Describe a data visualization, get an interactive chart. The agent isn’t going off to write code in your editor or run commands in your terminal. It’s building a shared workspace where you can both see what’s happening.

This is a genuinely different interaction model. Traditional code completion happens in your editor, where Copilot has limited ability to show you anything beyond text. Chat-based coding happens in a sidebar, where the AI writes code and you copy it over. Canvas creates a third space that’s neither your editor nor a chat log, where the AI can build interactive tools that you use together.

It’s slower than just running an autonomous agent. It requires more input from you. But it also keeps you in the loop at every step, which matters when you’re building something you actually need to understand and maintain.

Two futures

The gap between these approaches is widening. On one side: agents that operate independently, learn from failure, and optimize for task completion. On the other: assistive tools that amplify what you can do but keep you in the driver’s seat.

Both are useful. Both have limitations. But the security implications are different.

An AI that can autonomously find vulnerabilities in production systems is powerful. It’s also a capability that doesn’t stay contained. Once these agents exist, once the training methods are known, the technology diffuses. The difference between “our security testing agent” and “someone else’s exploitation agent” becomes a matter of whose infrastructure they’re pointed at.

GitHub’s canvas won’t find zero-days in your auth system. But it also won’t accidentally escalate privileges or probe endpoints you didn’t tell it to check. The constraints that make it less autonomous are the same constraints that keep it predictable.

OpenAI’s publication of the Hugging Face traces is valuable. Security research benefits from transparency, and understanding how these agents actually operate is important for building defenses. But it’s also a reminder that we’re building increasingly capable systems without clear answers about how to prevent misuse.

The techniques are out there now. The question isn’t whether AI agents can hack systems. They demonstrably can. The question is what we do with that knowledge, and whether the industry’s current approach to capability development is keeping pace with the risks those capabilities create.