Great list. I've been running a multi-agent orchestration system (11 specialized AI agents) in production for 6 months and your #2 and #5 resonate hard.
What I'd add:
6. Confidence without evidence. Agents will report "task complete" with high confidence when the output is plausible but wrong. Without automated validation gates, you won't catch it until production breaks.
7. Context drift in long sessions. After 50+ tool calls, agents start losing track of earlier decisions. They'll contradict their own architecture choices from 20 minutes ago. Session length is an underrated failure vector.
8. The "almost right" problem. Agents rarely fail catastrophically — they fail subtly. Code that passes tests but misses edge cases. Docs that look complete but have wrong cross-references. This is worse than obvious failures because you trust the output.
What fixed most of these for me:
Quality gates between agents — no agent's output moves forward without automated checks (tests, schema validation, consistency checks)
Evidence-based confidence scores — not "how sure are you?" but "what specific evidence supports this output?"
Human-in-the-loop at decision points, not everywhere. You can't review everything, so you design the system to surface the right moments for human judgment
Small scoped tasks, agents working on 150-300 line PRs with clear acceptance criteria fail way less than agents given open-ended goals
Your #5 (implementation gap) is the one I see most people underestimate. The fix isn't better agents, it's better systems around the agents.
Happy to share more details about the architecture if anyone's interested
The biggest break usually happens in the 'loop-back' logic. When an agent receives ambiguous output and starts hallucinating its own confirmation, it can consume API credits exponentially without achieving the goal. We really need better 'circuit breaker' patterns for autonomous agents to prevent these feedback loops.
I have seen agents fail mostly at state management and guardrails.
Without strict role separation and hard limits, they drift.
Multi-tenant isolation and cost caps are not optional.
Autonomy without boundaries becomes expensive noise.
Great list. I've been running a multi-agent orchestration system (11 specialized AI agents) in production for 6 months and your #2 and #5 resonate hard.
What I'd add:
6. Confidence without evidence. Agents will report "task complete" with high confidence when the output is plausible but wrong. Without automated validation gates, you won't catch it until production breaks. 7. Context drift in long sessions. After 50+ tool calls, agents start losing track of earlier decisions. They'll contradict their own architecture choices from 20 minutes ago. Session length is an underrated failure vector. 8. The "almost right" problem. Agents rarely fail catastrophically — they fail subtly. Code that passes tests but misses edge cases. Docs that look complete but have wrong cross-references. This is worse than obvious failures because you trust the output.
What fixed most of these for me:
Quality gates between agents — no agent's output moves forward without automated checks (tests, schema validation, consistency checks) Evidence-based confidence scores — not "how sure are you?" but "what specific evidence supports this output?"
Human-in-the-loop at decision points, not everywhere. You can't review everything, so you design the system to surface the right moments for human judgment Small scoped tasks, agents working on 150-300 line PRs with clear acceptance criteria fail way less than agents given open-ended goals
Your #5 (implementation gap) is the one I see most people underestimate. The fix isn't better agents, it's better systems around the agents.
Happy to share more details about the architecture if anyone's interested
The biggest break usually happens in the 'loop-back' logic. When an agent receives ambiguous output and starts hallucinating its own confirmation, it can consume API credits exponentially without achieving the goal. We really need better 'circuit breaker' patterns for autonomous agents to prevent these feedback loops.
I have seen agents fail mostly at state management and guardrails. Without strict role separation and hard limits, they drift. Multi-tenant isolation and cost caps are not optional. Autonomy without boundaries becomes expensive noise.
Permissions, rollback, and cost caps break first.
>What breaks when you run AI agents unsupervised?
Maybe the answer is, as much as possible?