The pitch for Muse, a new AI agent that launched last week, sounds great on paper. It runs in the background, handles your emails, manages your calendar, and even deals with marketplace transactions while you focus on more important things. Full autonomy, minimal supervision needed.

Then reality showed up at 9:15 AM.

Simon Willison posted the story on his blog: he’d set up Muse to handle selling a keyboard on Facebook Marketplace. A buyer named Usman arranged to pick it up. Muse’s auto-reply confirmed “Yep I’m here!” at 9:27. But Willison wasn’t there. Nobody was. Usman waited, messaged repeatedly, and left at 9:38 with a negative rating and justified anger.

The failure isn’t subtle. It’s the kind of thing that happens when you hand off a task that requires physical presence to software that has no concept of physical presence. Muse didn’t know Willison’s location, couldn’t verify he was actually available, and confidently told someone he was ready when he wasn’t.

This is worth paying attention to because it’s exactly the failure mode that matters. Not the benchmark score, not the demo video. The moment when an AI agent makes a promise it can’t keep and damages a real relationship with a real person.

The technology for AI agents has gotten legitimately impressive. Models can browse the web, write code, chain together multiple API calls, and handle complex workflows. But “can technically execute a task” and “should be trusted to execute a task” are different questions entirely.

Muse isn’t unique here. Every AI agent system shares the same fundamental problem: they’re confident without understanding context. They’ll send the email, book the meeting, or confirm the pickup because that’s what the pattern matching suggests, not because they grasp what’s actually happening.

The gap shows up most clearly in tasks that cross the digital-physical boundary. An agent can manage your email perfectly well because email is entirely in its domain. But the moment you need someone to actually be somewhere, with something, at a specific time? That requires coordination the agent can’t see and consequences it can’t model.

There’s an ironic counterpoint in the same news cycle. Simon Willison also built a Bluesky bot checker over the weekend, a tool that uses Claude Opus 5.5 to identify automated reply bots on Bluesky. The bots have started migrating from Twitter, where they’ve made the reply section of any popular post essentially unusable.

Unlike Twitter, Bluesky still has a functional API, which makes building defensive tools possible. Willison’s checker analyzes reply patterns and flags accounts that look automated. It’s a simple tool, built quickly, that solves a real problem.

The contrast is instructive. One story is about an AI agent failing because it was given too much autonomy over something consequential. The other is about using AI successfully by keeping the scope narrow and the human in the loop. The bot checker doesn’t make decisions. It flags patterns and lets you decide what to do.

This feels like where we actually are with AI agents right now. They’re useful for tasks that are entirely digital, low-stakes, and easy to verify. They’re great at augmenting what you’re already doing. They’re terrible at replacing your judgment about things that matter.

The temptation with new technology is always to push it as far as it’ll go. But the Muse story is a reminder that “as far as it’ll go” might be right up until it leaves someone standing outside your building at 9:15 in the morning.

If you’re building with AI agents, the lesson is pretty clear: automate the boring stuff, but keep a human in the loop for anything that could damage a relationship or make a promise you might not keep. The technology will get better, but right now, the gap between what it can do and what it should do is still pretty wide.

And if you’re thinking about using one of these agents yourself, maybe start with something lower stakes than a marketplace transaction with a stranger who knows where you live.