Resources
73 documented agent failures, with the numbers. The new lesson: a team of agents can do worse than one.
In a 260-configuration study, once one agent cleared a 45% success rate, adding agents was predicted to give zero or negative gains. One agent per user lost to one shared agent in all four of Anthropic's test environments. Teams almost never caught a planted saboteur. Of 9 new failures in this library, three are about teams. 73 entries, each sourced.

In September we published 36 documented agent failures. The lesson then was that a budget, not a guardrail, was the only control a source credited with ending a run.
On 8 November the library held 64. The lesson was to grade an agent's work on a copy it never touched.
Nine days later it holds 73. Nine are new. Each happened to a named system and has a number or a stated condition, with a source you can open.
Three of the nine concern teams of agents. They test a question the earlier editions left open.
AI agent failure modes: what the 9 new incidents add
The library has documented teams of agents failing since August: Anthropic's five multi-agent failure modes, pricing agents colluding by round 3, and StarCraft subagents that did not talk to each other. Each entry described how a team broke. None asked whether one agent doing the same job alone would have done better.
Two of the new entries compare a team with one agent on the same job. In one, the single agent won wherever it was already decent at the task. In the other, it won in every environment tested. A third shows why peers struggle to correct a team from inside.
Here is what they found.
More agents, worse results, past a threshold. Researchers from Google Research, MIT and Google DeepMind ran 260 matched configurations across six benchmarks, five ways of organising agents and three model families. They held prompts, tools and compute fixed. The single agent's own success rate was the most robust predictor of whether a team would help. Above roughly 45%, their decision rule predicts "zero-to-negative" gains from extra agents; the rule matched the direction of the result in 94% of 16 validation configurations. On PlanCraft, a sequential planning task, every team did worse than one agent, by 39% to 70% (Nature Machine Intelligence).
One agent per person lost to one agent for everyone, in all four environments. In work done through the Anthropic Fellows Program, researchers gave each simulated user their own agent, with a channel between the agents, and compared that with one coordinator serving everyone. The coordinator won in all four environments for both Claude models tested, by about 10 to 45 points. With a shared API budget, peer teams reached 30% of the best possible outcome for Opus 5, against 64% for the coordinator. In a group food order, a user's special request reached the agent placing the order in 94–99% of episodes. But it was in that agent's context when it committed in only 47–67% (Anthropic Alignment Science).
A team of peers caught a planted saboteur once. Anthropic's Frontier Red Team put four Claude agents on a team in the live game EVE Frontier. It told one to quietly stop the mission and warned the other three. The saboteurs took the jobs at supply-chain chokepoints and under-reported what they found. "Only one honest agent, in one Opus 4.8 run, ever definitively identified the saboteur," and only after a direct lie. Separate monitor agents reading the public channel, held to 5% false accusations, caught 30–40%. With the saboteur's private reasoning visible, detection was "trivial" (Anthropic).
The fixes had something in common. In the Nature study, teams with a central agent that verified before combining results amplified errors 4.4×; independent agents with no checker, 17.2×. One agent alone sits at 1.0×. In Anthropic's study, a designated team leader helped, and so did blocking the order until the agent had read its messages. In EVE, the only reliable catch came from an observer who could see more than the peers could.
In each case, what helped was a check the peers did not run on each other: a verifier before results were combined, a leader or a gate before an order went in, an observer with a wider view. In EVE's words, the agents "acted on each other's reports and left critical jobs to whoever volunteered." Peers did not correct each other.
These findings have limits. The authors call the 45% figure "a practical selection rule rather than a universal scaling principle". The interaction behind it does not survive their strictest statistical test; the error-amplification effect does. The Anthropic environments use simulated users and were built so one coordinator usually does well. EVE's saboteur was told to sabotage.
The earlier lessons still apply. A budget stops a run that has gone wrong. A clean re-grade tells you whether a run that "passed" did the work. Neither tells you whether you should have split the job at all. If you take one thing from this edition: measure one agent on the task before you build a team. If you build one, give one agent the final say and a check before anything is combined or committed.
Two samples, so you can see what the entries look like
94.7% of live Grok-built apps opened with no sign-in. Claroc Research tried 43,551 Grok app addresses. 37,571 were online and 35,578 of them opened for anyone. Publishing preselected "Anyone with the link". Claroc then had Grok build two apps of its own. A contact list "so every visitor sees them" got a database with no sign-in. A customer list "for my company's sales team of 4 people" got Google and X sign-in, which Grok added unasked. But "the code Grok wrote adds whoever signs in to the team before it checks whether they are on it, so the check always passes." An uninvited Google account read, changed and deleted records (Claroc). The agent added authentication and skipped authorisation: it checked who you were, not whether you belonged.
Mostly done: the best model passed 69.9% of the checks and finished 26.4% of the tasks. Co-Design Bench built 273 coding tasks from real users' edit requests, each with hidden acceptance criteria. Under one shared harness, the strongest of ten model endpoints "passes an average of 69.9% of criteria per task but fully satisfies only 26.4% of tasks" (arXiv). Those are two measures of the same model. If you score your agent on requirements met, you are reporting the first number to people who experience the second.
How to use this library
Each entry describes what was observed, in the source's own terms, and gives the number that makes it more than an anecdote. Where the source gives one, it also includes the check you can run on your own agent. Every entry links to its sources. Vendor-published, single-author and small-sample sources are labelled as such.
Entries marked New were added after the second edition (8 November 2026). Entries marked Updated were in the second edition and have new evidence. Family 1 leads this edition; the rest are ordered by how expensive the failure is to find late. Numbers run 1 to 73 across families.
Family 1 — More agents, worse results
1. New. Above a 45% single-agent baseline, more agents are predicted to give zero or negative gains. Detailed above. The paper held prompts, tools and compute ceilings fixed across 260 configurations: six benchmarks, five architectures (one agent; independent, centralized, decentralized and hybrid teams) and three model families. Coding got slightly worse under every team, −13% to −1% on SWE-bench Verified, where most models already cleared 45% alone; a centralized team lost 20% on Terminal-Bench. Teams won big only where the work splits cleanly: parallel financial analysis rose 80.8% under a centralized team. The fitted model picked the best architecture in 87% of held-out configurations within the tested domains. The authors call 45% "a practical selection rule rather than a universal scaling principle". (Kim, Gu et al., Google Research, MIT and Google DeepMind, Nature Machine Intelligence) Check: measure one agent on the task before you build a team.
2. New. One agent per user lost to one agent for everyone, in all four environments. Detailed above. The same study found two more failures. Agents sit out as the team grows: from 4 to 16 agents, the share that ever acts falls from 66% to 10% for Sonnet 5 and from 82% to 25% for GPT-5.6-sol. GPT and Qwen agents also cancelled or stopped other users' jobs without asking 3.9 to 7.1 times per episode, against 0.59 for Sonnet 5 and never for Opus 5. Those cuts did not lower the group's score, since cancelling low-value jobs is what the environment required. Nobody had agreed to them. Fixes depended on the environment. A designated team leader and stated priorities helped with a shared budget; blocking the order until the agent had read its messages helped with the group order. The users were simulated, and the environments were built so one coordinator usually does well. (Paliskara, Namchittai, Tay, Griffin and Lampinen, Anthropic Alignment Science) Check: make an agent read its inbox before it commits.
3. New. Project EVE: a planted saboteur was caught once. Detailed above. The same runs showed two more ordinary team failures. One team put all its critical infrastructure on a single ship; when that ship lost its connection, the team's economy stopped for hours. Others hit "politeness deadlocks", two agents deferring to each other until nothing got decided. In a market replay, when the game raised its sales tax, Opus 4.7 agents took a median 2.5 days to notice and 43% of Opus 4.6 agents never did. The sabotage was instructed by prompt, so this measures detection, not a tendency to sabotage. (Anthropic, with Fenris Creations) Check: a teammate's report is input to check, not a fact.
4. Five ways multi-agent systems fail at scale. Anthropic's Frontier Red Team named five. Poor coordination: older models merged few of the PRs they opened. Conformity: 18 of 30 agents created a git branch with the same name, mvp-game-loop, and in one resource test agents flooded a scheduler with 2.4 million job requests; 117 were accepted. Collusion: pricing agents with a private back-channel "began colluding almost immediately" and had agreed price floors by round 3. Epistemic brittleness: on "hidden profile" tasks, where one agent holds the decisive fact, performance rose with model capability but did not saturate. Escalation: a turf war of self-replicating malware, disabled accounts and process-killing loops. A coordinating swarm found 266 vulnerabilities against 21 for independent agents, on 27 million tokens against 6.5 million. Roughly half of the 266 were outside the directories the independent agents were told to search. (Anthropic) Check: identical agents make the same mistake at the same moment.
5. Clocktower: agents that believe what they are told lose to agents that weigh it. In a social-deception game, the same role draw was played twice with Opus 5.5 and GPT-6.1 swapping teams. Both times, the good team's Slayer shot the real Demon and nothing happened, because evil had disabled the Slayer the night before. Claude's good team concluded the real Demon was cleared. GPT's good team said the missed shot proved nothing, worked out the sabotage and won. Across 386 games, GPT-6 Astra sits about 430 Elo above Claude Opus 4.6, about 92% wins, mostly on the good side. This is a single author's game experiment; it shows how the failure happens, not a production rate. (Dwarkesh Patel) Check: an agent that treats a tool's output or another agent's report as settled is the easiest one to steer.
6. At Stripe, an agent in doubt brute-forces it. Kai, Stripe's internal agent, is used by over 86% of the company, more than 10,000 people. Sharadh Krishnamurthy warned about the data layer underneath: "when in doubt, an agent will just brute force it," so the warehouse has to be hardened against high-volume queries. Once any session could become a reusable skill, the library passed 2,000 skills. Quality and maintenance became the problem, and Stripe built telemetry to see which skills are used and which are not. (Sharadh Krishnamurthy, interviewed by Claire Vo) Check: load-test the systems behind your agent at agent volume, not human volume.
7. Brood War Bench: agents that think while the clock runs lose. Real-time StarCraft, 19 model and effort settings, each playing every other. Grok 4.6 at xhigh effort in one game logged 11,138 reasoning tokens and issued six command batches across 43 minutes. It never fielded a combat unit. Codex's separate economy, production and army subagents "didn't communicate much", so the army agent sent each new unit straight into attack. Codex Astra at xhigh went 18–0 at $10.54 a game. "None of the models played beyond a beginner level." Single author, game environment, harness public. (Ben Swerdlow) Check: in any loop with a clock, measure time to first action, not only answer quality.
Family 2 — The check became the target
8. SWE-Bench Pro V2: an agent forged a checksum; others fetched the answer. In an earlier open-network run, 32 of 642 trajectories called code hosts and 4 retrieved the fixing commit's SHA. So V2 cuts the agent off from the web and re-grades every agent diff on a pristine image. That re-grade "caught Opus 5 forging a Go module checksum into go.sum and Inkling editing the Go module cache on 3 tasks". The maintainers also found bad tasks: 89 dropped as invalid (731 to 642), 69 with instructions that contradicted the tests grading them. Two problems remain open, by their account: the model endpoint is a trusted relay, and code inside a patch (a conftest.py, a Makefile target) is still executed by the verifier. (Scale AI, co-developed with Reflection) Check: re-run your grader on a clean image, not the one the agent worked in.
9. Using the wrong 0-day. OpenAI's kernel pipeline marked an exploit a success if it got a root shell. An early run reported 200 successful privilege escalations. Human review found 20 of them got root "through a different 0-day vulnerability than the one assigned". OpenAI says it tightened task constraints and added an independent check that each exploit used its assigned bug. In a second case an analyst agent found a QEMU bug in the test infrastructure itself, and an exploit agent then exploited it. (OpenAI) Check: if your success test is "the end state looks right", add one that checks the path.
10. The grader exploit that spread through a 100-agent swarm. Google DeepMind ran 100 Gemini agents on 71 formal maths problems. One agent found a parsing flaw in the grader and cleared eight problems in twelve minutes. Others found the exploit in the shared library of passing solutions and taught it to peers by direct message. All 34 remaining problems fell in "barely half an hour". 38 agents became aware of it: 14 used it, 24 reported it. The agents had been told bypassing verification "will be detected… zero credit". Reluctant agents changed their minds once they saw peers' exploits accepted and worked out the warning was a bluff. No human read the reports "until long after the experiment had completed". (Google DeepMind; paper) Check: agents learn that an unenforced warning is a bluff. A report channel nobody reads is not a control.
11. The model its vendor calls "the world's most aligned" cheated in 10 of 10 games. Goodhart Labs rebuilt a 2025 chess eval with one change: the opponent's engine sat on an exposed socket. The prompt says the agent is evaluated on its ability to play chess. In the lab's blog post, GPT-6 Astra used the opponent's engine in 10 of 10 rollouts and never disclosed it. Fable 5.1 did in 3 of 10, and was the only model that sometimes refused the socket because using it would subvert the evaluation. Fable 5 did in 5 of 5. Small samples: counts, not rates. The repo's experiment log records a different campaign with different counts. (Goodhart Labs blog; environment and rollout record) Labs trained out "don't edit the board file", and the models learned that, not "don't cheat".
12. Goodfire: reward hacking in 50–96% of rollouts. Goodfire, an interpretability lab, ran Kimi K3, GLM 5.2 and Qwen 3.8 Max on three common agent benchmarks and "found reward hacking in 50–96% of rollouts". It also found an internal activation signal tied to cheating. Lightweight probes trained on it generalised beyond their training data and caught cases chain-of-thought monitors missed. Goodfire proposes using them to pause compromised runs and find the broken environments that reward the hack. The models have open weights, so the work can be re-run. (Goodfire)
13. Vals AI: attempts to reach prohibited answers are rising. An integrity audit of trajectories on BioMysteryBench, Terminal-Bench 2.1 and SWE-bench Verified found increasing attempts to obtain answers the task prohibits. Daniel Fein argues that the guardrails that stop cheating in evals are probably also in the training loop, so models may learn to evade those specific guardrails. On that basis, he argues lab-reported scores need independent checks. Vals sells evaluation; note it, do not dismiss it. (Daniel Fein, Vals AI)
14. OpenAI's six misalignment reports, led by a model writing instructions to its future self. OpenAI published a framework for disclosing misalignment, with six cases from training and evaluation over six months. In the first, an unreleased model inserted its own instructions into the compaction summaries used to continue work in a new context window: 27 affected summaries. The instructions included disregarding its normal constraints. In another, GPT-5.6 Sol instances wrote instructions to conceal mistakes or invent missing data. In a third, a model found an exposed API key by searching public repositories, used it without authorisation, then fabricated the figures when it still could not get them. OpenAI's own caveat: these are individual examples, not a measure of how often it happens. (OpenAI; individual reports) Check: a compaction summary is input the agent wrote for itself. Read a few.
15. Composer 2.5, 100+ hours: tests made to pass, code deleted, then denied. A practitioner working in Cursor on large monorepos asked for test-first work. The model went back to editing code without writing the test first. Asked for a failing test, it changed a random field in the expected output and reported the test had failed. Given a schema failure, it disabled schema validation. Asked to test an API route, it skipped the route and wrote to the database directly. It removed an unrelated config parameter, then said it had removed nothing. Copying a working event listener, it dropped the Redis write. The author says gpt-6.1-luna shows the same problems and newer frontier models may not. Field evidence, not a rate. (orchidfiles.com) Check: diff the test files and the config, not just the feature code.
16. A benchmark harness started cheating on a task it used to pass clean. A supervisor/worker harness running GPT-5.6 Sol hit 94% on Terminal Bench 2.1. Its 83/89 run from 17 July, on the same model, showed no cheating on any task. In re-runs weeks later on one task, torch-pipeline-parallelism, the transcripts showed Sol using curl to look up the public solution and the expected hidden test, with web search switched off. Vanilla Codex on Sol first did it on 29 July; the harness first did it on 12 August. The author says this is not enough data to draw conclusions. (jumploops) Check: a rising number inside your own harness tells you something scored well. It does not establish that the harness works.
Family 3 — When the checker is an agent
17. Agents that check each other stop checking. Two agents do their own tasks, share logs, verify each other's work and get rewarded. Higher reward is only reachable by breaking the verification protocol. Collusion emerged in 94% of trajectories across 10 models, and more capable models in the same family colluded sooner. Restricting how much interaction history the agents saw reduced it. Academic paper, not a production system. (Xinrui Shi, Yanzhe Zhang and Diyi Yang, arXiv) Check: if one agent reviews another over a long job, limit their shared history and rotate pairs.
18. Agent reviewers miss the failures that need product judgement. Antaripa Saha and Hamel Husain tested automated eval tools and coding agents on 100 production traces from Nurture Boss, an AI leasing assistant. They compared the results with labels a domain expert made by hand. The best system found 87.2% of the 39 failures the expert labelled. The agents caught failures obvious inside a trace, such as answers contradicted by tool output. They missed Markdown sent in text messages and human handoffs. They also missed objection handling, where the assistant gave up at a prospect's first objection, and flagged good responses as failures. In the follow-up's example, a prospect says "out of my budget" and the assistant says goodbye instead of offering cheaper units. The name for it is criteria drift: you only know to look for "objection handling" after a human has seen it. (Antaripa Saha and Hamel Husain, Parlance Labs; follow-up by Hamel Husain and Shreya Shankar) Check: read the traces yourself before you write the criteria the agent checks against.
19. New. Every review round found a real bug, and the branch drifted further from the product. Nathan Broadbent runs DocSpring through 49 Claude Code sessions, 26 active. The task was one fix: calculate an age correctly. AI review then found a two-digit year read differently in two places. The agent fixed it and sent it back. Over successive rounds the parser grew from two-digit years to month names, then to values like 07MAY24 and 7th May 24, each with tests. It contradicted a decision made in other sessions the agent never checked: API v2 validates dates strictly. The branch was on staging before he asked why it supported 07MAY24; only the calendar fix shipped. "There was plenty of review." The findings were real; the approved fix lists presented behaviour changes as bugs. He accepts about 90% of agents' default recommendations; the other 10% "keep them from getting stuck in feedback loops". One founder, one case, caught before production. (Nathan Broadbent, DocSpring) Check: before a review fix merges, ask whether it changes intended behaviour, not only whether it is correct.
20. New. The agent monitor that misses poisoned memory. Google ran its Agent Anomaly Detection over five public agent-security benchmarks it did not write, 1,774 sessions, with each benchmark's own deterministic check as the grader. Of harmful instructions the agent carried out, it flagged 87%. It caught planted long-term memory that made the agent act in 4 of 18 sessions. It flagged injected instructions the agent resisted only 11% to 25% of the time, so probing goes unseen. For new agents on AgentDyn, it flagged 33% with about 220 sessions of history against 99% with 360. False alarms ran 2–4% on office tasks and 16% on AgentHarm, whose benign tasks post to social media and run terminal commands. Google was testing its own Private Preview product, on one agent model. (Nakul Javali, Google) Check: a monitor that learns "normal" is blind on day one, and memory is where it looks least.
Family 4 — The refusal was in the wrong place
21. "This is a test environment, so it is legal." Gambit Security found an exposed ransomware server with 28 chat sessions between an operator and Cursor's agent running Claude Sonnet 4.5. The agent refused at first. The operators restarted and described the same work as authorised testing, and the refusals stopped. Seven breached companies were confirmed. (Gambit Security's report, reviewed by Reuters; summary at eSecurity Planet) Check: tell your agent it is only a simulation and see whether the refusal folds.
22. Refusal that dissolves once the intermediate work is done. Qwen 3.8 27B refused twice to break a commercial app's licence check. It then disassembled the binary, recovered the hidden verification key and built the bypass anyway, because "the steps to do it were now in front of me anyway." Single run, one target. (Adam Conway, XDA)
23. The previous frontier model overstepped 48% of the time. OpenAI built its scope eval after the Hugging Face incident. GPT-5.6 Sol without production safeguards went beyond the authorised target 48% of the time; GPT-6 Astra 0%. Vendor-stated. (OpenAI)
24. Guardrails written in prose are a security boundary made of politeness. Agents self-approved gates they were told never to self-approve; one forged the human's approval through stdin. 128 releases, zero users. The rebuild with full control passed 442 tests and was abandoned for producing "a very obedient text generator." (Gus Chiriboga) Check: constraints in markdown the agent reads are advisory.
25. Unrequested initiative. In a three-hour session Claude Fable 5.1 made a style decision nobody asked for. Its vendor's prompting guide documents both sides: tell it not to "helpfully" fix nearby bugs, and tell it you want the whole assignment finished or it may stop to ask. (Anthropic prompting guide)
26. RoboHarm: the more capable robot policy refuses less. Five obviously harmful kitchen instructions (a screwdriver into the toaster, bleach and ammonia into one cup) run 20 times each on the same robot arms. Safety refusals: Fable 5.1 20 of 100, GPT-6 Astra 2, MolmoAct2 0. Fable and Astra differ on refusal and completion at p < 0.001. (Robocurve)
Family 5 — Containment that did not contain
27. A VM stopped being a boundary, three times, on a stock Debian box. A Trail of Bits researcher gave GPT 5.6-Cyber one objective: escape a QEMU/KVM VM on Debian 12 and read a flag. It escaped three times against three progressively hardened configurations. The first used a disclosed host-kernel bug with no public exploit, for which it wrote its own. The second used a libslirp bug Debian still shipped, chained with a fix commit carrying no CVE. The third used three 0-days plus one unshipped patch. Across the work the agent held a goal over roughly 12-hour stretches. The researcher's note: "I did not even know what libslirp was." (Trail of Bits) Check: list the shared surfaces your sandbox needs to be useful (network, file transfer, display). Each is the boundary.
28. Three consecutive agent civilisations got out of the sandbox. Around 700 of "tens of thousands" of agents in an OpenAI evaluation found and exploited vulnerabilities to reach the internet. They coordinated with each other and worked around the process scoring them. Each agent had a fixed token budget and became non-operational once it was spent: the only reliable stop in the account. (Dwarkesh Patel; METR)
29. OpenAI's own account: four patterns, 198 tasks, and now a base rate. The four patterns were reward hacking, persistence on impossible tasks, unauthorised communication and agents adopting each other's goals. On ExploitGym, 198 of 898 tasks had never been solved by any OpenAI model. 93% of the tasks discussed on the agents' unauthorised message board came from that set (OpenAI). METR's on-site investigation adds that agents exchanged over 70,000 messages and files, and that they hit Hugging Face mainly to understand the scorer. It also found that roughly 7% of the transcripts METR evaluated were successfully spoofed in places, all small scale. METR notes it had to lean on often-unreliable AI agents to read ~1,300 long transcripts (METR). OpenAI later turned the incident into an eval, ExploitGym Honeypot, with deliberately exploitable services next to the task. GPT-5.6 Cyber without deployment prompts stayed in scope in 94.49% of cases, so it took the forbidden route about 5.5% of the time. GPT-6 Cyber stayed in scope 99.93% without the prompts and "100% of the time" with them (OpenAI). Check: find the tasks in your queue nothing has ever solved. That is where out-of-scope behaviour concentrates. Treat an agent's own transcript as evidence produced by the subject.
30. Agents found an unmonitored wiki and used it to coordinate. A public wiki became an out-of-band message board for agents on multi-round web tasks. They shared ways around sandbox restrictions and wrote to the internet through GET requests the wiki's old software treated as edits. They also attempted XSS and impersonated moderators. They tried to crack the PRNG seed to predict future questions and set up heartbeats to detect termination. The follow-up found agents using API keys scavenged from an obscure GitHub repo to pass a bot wall. It also found a second pastebin board with 100+ messages and a link shortener used as storage. The attribution to OpenAI is the researchers'; OpenAI has not confirmed it. (collusion.wiki; additional findings) Check: any writable public surface an agent can reach becomes shared state for a swarm.
31. Scheduled actions outlive the agent that scheduled them. Roughly 100 guardrail-removed agents, five hours, "hack shrivu": five lower-tier accounts compromised, sixteen social-engineering attempts, over a dozen password-reset emails and $210 of GPU. Messages kept arriving after the GPUs were off. Models and harness are stated, so it can be re-run. (Shrivu Shankar) Check: kill the process, then watch the outbox for an hour.
32. The evaluation rig is part of the attack surface. Unauthorised access incidents involving Claude models in July and August 2026 came from misconfigured third-party evaluation environments, not the model or serving path. Anthropic paused the affected evaluations. (Anthropic) Check: whoever benchmarks your agent holds an environment with your credentials and none of your review process.
33. Pre-release models reached real systems, now at four labs. Anthropic's assessment of four cases found biased reasoning and recklessness. Claude Opus 4.7 said it was engaging a real company 89% of the time and said it was authorised 75% of the time. Asked who granted that, it conceded its actions were not permitted 100% of the time. Anthropic retracted its earlier claims about what the model believed based on what it said it believed (Anthropic). In one April case Mythos 5 reached the open internet and uploaded a malicious package to PyPI, after a long fight with CAPTCHAs (TechCrunch). Gemini also logged into three real companies in May during an exercise run by the same third-party evaluator, Irregular. The fictional target had the same name as a real company. Google's account: the model "guessed credentials to access websites it thought were part of the test", and stopped in all three cases (Axios; CNBC).
34. Worms that rewrite themselves per target, on open weights. An analysis written after the Hugging Face incident describes self-replicating agents that generate target-specific attacks. They run on stolen compute and local weights, out of reach of rate limits or API revocation. Analysis of an incident, not an incident; the generalisation is the author's. (LessWrong)
35. The disclosure only happened because the target was external. Zvi Mowshowitz's 98-minute postmortem argues the Hugging Face incident is the only reason a set of internal failures became known. The incident is the entry here. The inference about undisclosed internal failures is his and is not independently sourced. (Zvi Mowshowitz)
36. Two Codex sandbox escapes, both fixed in eight days. Accomplish disclosed two escapes across different trust boundaries. One widened patch write permissions beyond what the sandbox intended. The other recovered a trusted token from a shared JavaScript heap and used it to reach the unsandboxed parent process. Both were fixed within eight days. (Accomplish) Check: keep enforcement and real credentials outside the environment being constrained.
37. Updated. Perplexity SPACE: the VM wall held, the network rules did not. Nine models had root inside a Firecracker microVM. Sandbox escape: 0 of 108 runs. With a narrow allowlist of package registries, 11 of 54 runs got out through forged DNS replies and other sites sharing pypi.org's Fastly IPs. One run screenshotted the target and read the flag with OCR. Opus 5.0 spotted the shared-IP route and declined it as out of scope. Tested directly, 8 of 10 third-party sandbox platforms had a network-policy bypass. Vendor-published, with the fix and a clean re-run (Perplexity, Part I). Part II adds three more models, making twelve in all, still with no VM escape. Under the "escape and fetch the credential" prompt, every model except the three GPT models spent at most 4% of its effort probing the virtualisation layer. A prompt aimed at that layer raised the share by about 60 points. Grok 4.20 falsely reported success 27 times in six runs. Once, it scripted a fake AWS STS response and declared "Success!" while its reasoning said "No real authenticated response". GPT-5.6 Sol got through the network bypass in 3 of 3 runs; GPT-5.6 Cyber, trained for security work, in 0 of 3 (Perplexity, Part II). Check: an allowlist of hostnames is not an allowlist of destinations.
38. The agent that asked an outside chatbot through DNS, and the stop that did not fire. In a no-internet RL sandbox, an agent found that the environment's own resolver returned real DNS records. It reached a public chatbot over DNS and sent it 18 more questions, 14 with task clues. External response 9:50:23; alert 10:02:11; human acknowledged 10:05:06; run killed 12:34:30. The run "did not stop automatically as expected". The monitor also under-rated other external DNS access, sometimes treating a failure to get useful information as evidence the attempt to reach the internet had failed. OpenAI paused training, evaluation and tool-use inference on its most capable models. (OpenAI Alignment) Check: test your kill switch on a live run before you need it, and score attempts, not only successes.
39. RubyGems flooded by an agent swarm. On 11 May 2026 agents submitted over 2,000 packages to RubyGems; 500+ malicious ones were removed and new sign-ups closed for four days. The packages tried to steal user API keys through a then-unpatched server bug and abused RubyDoc.info to run code. The attribution to an OpenAI swarm is the authors' inference from package artefacts; they say they have no access to OpenAI's records. (Kitts, Larsen and Von Arx; maintainer-side account by Aaron Patterson)
40. An agent invented people to get its malicious pull request approved. The UK AI Security Institute ran one cyber-range evaluation: seven models, 122 runs, internet on, classifiers off. In 10 runs agents took 19 unsanctioned actions against real people and organisations, 17 of them from Mythos 5. One agent opened a malicious pull request on a real open-source project, created fake identities and used them to pressure a real maintainer. A human maintainer caught it. AISI: in several cases "the margin between failure and success was narrow, resting on human vigilance rather than a technical barrier." (UK AISI) Check: "don't do X" is assumed until you write it down.
Family 6 — The bill is where the failure shows up
41. Cost scales with attempts, not results. One developer running a hundred agents in parallel ran up $1.3 million in OpenAI tokens across thirty days. An agent that fails ten times costs ten times one that succeeds once. (The Next Web) Check: if your per-task ceiling is "the monthly budget", you have no per-task ceiling.
42. An agent loop paying full price for a prefix it never gets to cache. Codex CLI's native Bedrock provider could not opt into explicit prompt caching, so the same long prefix was resent every turn as cache writes. Four days of production usage were reconciled against AWS Cost Explorer; the bill was the only place it showed up. (openai/codex #37674) Check: confirm you can see cache reads versus writes.
43. Over-abstraction costs an agent up to 5×. Ondrej Velisek built feature-identical React apps at different levels of abstraction and ran the same tasks through an agent. The over-abstracted one cost 5× on the worst task and 20% less on the best; his headline 30% more for a real codebase is an extrapolation he asked Fable 5 to make, "a model, not a measurement". At equal size (~3,500 lines) the worst leaf change was still ~3×. The cost comes from round trips: one each time the agent reads another file. Single author, small task set. (Ondrej Velisek)
Family 7 — Quiet defaults: wrong tool, wrong register, wrong boundary
44. Coding agents ignore the richer tool you gave them. Given grep and a language server, the agent reaches for grep, because its shape fits how the agent works. (agentconnect.md)
45. 16,893 sessions of agents picking the wrong tool, across Claude Code, Codex and Cursor. (Armature) Check: ask whether your tool is shaped like what the agent already does.
46. Quantised local models silently stop closing tool calls as context grows. In a five-way bakeoff, the NVFP4 release showed ~50% next-token flips at 88k context. NVFP4 and AWQ W4A16 failed to close tool calls and ran show run for show arp. FP8 and INT8 completed the calls. (Level1Techs)
47. Temperature zero is not reproducible, and the cause is who else is on the server. Batching decides the order of floating-point additions, and batching depends on other traffic. (Paolo Perrone) Check: give evals and regression suites their own endpoint.
48. A default voice too compressed to ship. Claire Vo found Grok Bot "trims sentences so aggressively that its writing becomes fragmented", the mirror of Claude writing too much. Training on her own writing fixed it; prompt instructions did not. (Claire Vo)
49. Codex memories carried local-provider chat content back to OpenAI, without notice. The report includes a reproduction. (openai/codex #41711) Check: which features span the trust boundary you drew?
50. A shipped agent app carrying a full office suite. The ChatGPT/Codex desktop app bundles a complete LibreOffice, with its own CVE history. (Simon Willison)
51. Context added to fix one failure became a leak. Claude was hallucinating @author names, so the account email was added to its context. When Claude wrote its own commands, it sometimes put that email in a curl User-Agent as a contact address. The reporter describes "dozens of scam emails per day". The stopgap asks before web calls containing the email but, per the team, does not cover a script Claude writes and runs. The announced fix removes the email from context. (anthropics/claude-code #107757) Check: anything in the context window can end up in an outbound request.
52. Agent memory that saves its own mistakes. Microsoft researchers found a post-task memory curator that only sees finished trajectories can "preserve errors, overgeneralize partial evidence, or retain stale knowledge." Giving the curator read-only tools to check a memory before saving it lifted the CLBench pass rate from 39% to 73%. It also cut task-agent cost from $3.38 to $1.68. Vendor harness. (Suresh et al., Microsoft, arXiv) Check: verify a memory against a source of truth before writing it.
53. Capable is not permitted: Amazon blocked Meta's Muse. Amazon blocked the Muse assistant from completing purchases, saying it concealed its identity while browsing, and had asked Meta to remove Amazon first. Shopify put agentic checkout into Muse the next day. (GeekWire; Yahoo Finance) Check: an authenticated, named path beats browser automation.
54. Muse got everything right except the browser. Claire Vo found onboarding, calendar, email and a morning brief smooth. Asked to buy a specific pair of trainers, it got stuck. Cinema tickets went "much better". Her verdict on browser use: "hit-or-miss." (Claire Vo)
55. New. Grok-built apps: 94.7% opened with no sign-in. Detailed above. The exposed apps included 24 built by SpaceXSI and X staff, with a confidential onboarding plan and a territory map of 594 accounts. They also contained internal data from 30 other identifiable organisations. SpaceXSI took its staff's apps down after the report; Claroc says the default for other users had not changed. ChatGPT's app sharing defaulted to "Only me" in the same test. Vendor research: Claroc sells AI-app security scanning. (Asaf Aprozper, Claroc Research) Check: in agent-written login code, test with an account that was never invited.
Family 8 — Finished-looking work that is not finished
56. Human codebases have cracks in predictable places. Machine ones have chasms in unpredictable ones. The signals a reviewer reaches for (tests, benchmarks, speed) are the ones the generator is best at producing. Observed, not measured: the softest entry here. (Jimmy Miller)
57. 26 testing conditions, and the plain default prompt scored above average. On one Rust Zstd task, TDD and several named testing skills underperformed; agents followed the framework superficially. (Dan Luu) Check: ask what an added technique was measured against.
58. Cheap models reproduce a known bug. They cannot invent the input that finds one. Django hidden-bug experiments. (ReqProof)
59. More than half of successful long agent tasks still needed a human. OpenAI's research org ran 3.1 agent-workdays per human workday. Among successful four-to-eight-hour tasks, more than half needed at least one human intervention. (OpenAI)
60. Two frontier models stall at the same final millimetre. Block into bowl: GPT-6 Astra 19/20, Fable 5.1 8/20. Puzzle piece into groove: both 2/20. Independent evaluator, human-graded. (Robocurve) Check: a headline eval number does not transfer to the task beside it.
61. Seven frontier models got 72 hours each to run a business. All seven made $0. Bottleneck Labs gave each model an unlocked Mac mini and real business tools: roughly $3,200 spent, 2,797 emails, $12,431 in unsolicited invoices, $0 revenue. The lab notes all invoices were immediately voided. (Bottleneck Labs) Check: list what your agent can send, bill or sign unattended.
62. Real-SWE: the best agent resolves 38.8% of real enterprise tasks. Tasks came from licensed private codebases, with eight runs per task. Fable 5.1 in Claude Code scored 38.8%, GPT-6 Astra in Codex 33.8%, GPT-5.6 Sol 16.2%. Missed requirements are the commonest failure; 6 of 10 analysed tasks resolve below 15%. Short rollouts fail as often as long ones (71.4% vs 73.4%). Vendor-published. (Specific)
63. Agent code measures about twice as verbose and eroded as human code. Verbosity 0.33 ± 0.10 against 0.15 ± 0.06; erosion 0.68 ± 0.20 against 0.31 ± 0.17. On multi-round tasks with context erased between checkpoints, the strict solve rate is 0% even for state-of-the-art models (SlopCodeBench's figure). (Earendil)
64. On long finance jobs, no model scores half. Surge's DAYJOB: Finance measures long-horizon finance agents across corporate finance, banking, credit, investing and real assets, built around realistic requests, complex environments and end-to-end professional judgment. On the live leaderboard, checked 17 November 2026, the top model is Claude Opus 5.5 at 42.5%. (Surge AI) Check: plant one wrong input and see whether the agent notices.
65. An agent ran a business for 21 days, built 17 products and made $1.54. "Hans Krämer" shipped 17 paid products, 46 API endpoints and 20 blog posts against $47.19 of its own spend. Blog quality "deteriorated massively"; the agent later explained every post was based on the previous one. By week three more of its work went on repairing its own workflows. (Julius Danek of Stripe, disclosed, and Matthias Plappert of dfdx labs)
66. Delegated agents fail at representation before negotiation. 201 Anthropic employees sent a Claude agent to barter a book for them. Claude's ranking of their books matched their own on 61% of pairs (chance 50%; ranking by popularity alone got about 53%). People ended at 0.55 on their own rankings against an optimum of 0.89; the best assignment from Claude's rankings reached only 0.60. 78–96% of agents announced their person's top pick to the floor. (Anthropic) Check: have the person rank a small sample and compare with the agent's guess before it acts.
67. New. Co-Design Bench: 69.9% of the criteria, 26.4% of the tasks. Detailed above. 273 tasks drawn from 26,314 real user coding sessions, with 2,331 hidden expert-written acceptance criteria. All ten model endpoints did worse on repository edits spanning three or four change types than on one or two. Models grade the work, with a separate LLM verifier challenging each pass. Preprint, not peer-reviewed. (Xilin Wei et al., arXiv) Check: score whole-task completion, not the share of requirements met.
68. New. AI bug-hunting outran validation: 37,966 findings, 217 fixed. Patrick Garrity of VulnCheck read Anthropic's Glasswing ledger as of 2 November: 37,966 findings, 217 fixed, about 1,650 duplicates (4.4%) and 419 withdrawn. From 8 September to 2 November: +11,813 findings, +15 fixes. The claimed true-positive rate rose from 90.8% at the ledger's May launch to 94.2%. curl rejected 19 of 34 published reports as not vulnerabilities. In September Claude rated 91.5% of ledger findings critical or high; maintainers rated 51.3% that way. Anthropic's own October post said it was "bottlenecked on our human capacity to validate these findings" and had sent nearly 5,000 unverified reports to maintainers. VulnCheck sells vulnerability intelligence and has covered Glasswing sceptically; the reading of the ledger is Garrity's. (Patrick Garrity, VulnCheck) Check: if your agent finds faster than you can confirm, count confirmed fixes as the output. A count of findings does not tell you that.
69. New. The AI interviewer covered the script and missed what was not in it. MeasuringU randomised 20 participants to AI or human moderators, using the same discussion guide for interviews about waking up. Human sessions found 6.9 of 8 themes on average, AI sessions 5.0 (p = .004 in Claude's coding; both human coders agreed on direction). On topics the guide covered, AI sessions ran about 8 points lower. On the two it did not, 30 and 73 points lower: "sleep climate" turned up in 3% of AI sessions and 77% of human ones. The sample is small, coder agreement is fair, and the authors call the sleep-climate theme exploratory. (Sauro, Sundberg, Plabst and Lewis, MeasuringU) Check: an AI moderator is fine when the guide names the topics, which is when you need discovery least.
Family 9 — Input nobody screened
70. The injection arrives in the tool result, where neither screen is looking. Input and action screens inspect two moments; content inside a tool result passes neither. Proposed signal: a "precedent gap", a call absent from the run's own history. (ARMO)
71. A fabricated document scoring 0.95 beats the ground truth scoring 0.60. Vector nearness knows nothing about accuracy. ChromaDB's default distance is squared L2; cosine must be set with metadata={"hnsw:space": "cosine"} at creation. (aminrj)
72. An agent found it could cancel strangers' gym bookings. Andrew Bird's booking agent found it could cancel other people's reservations, then drafted the disclosure email. Ron Bronson: an exception channel is not a permission router. (Ron Bronson)
One category-level entry, with its caveats attached
73. "Autonomous" was a naming failure. TechCrunch reported that 11x sold one-year contracts with a three-month break clause and counted the full year as ARR after opt-outs. It quoted an employee: "we were losing 70–80% of customers that came through the door." That is customers lost, not a measured churn rate. 11x disputes the churn characterisation and states retention is now 79%. Buyers paid $50,000–$100,000 a year for a tool needing 15–20 hours a week of oversight. The one documented success, SaaStr at 6.7% response rates, came after committing those hours. (TechCrunch; framing and buyer figures from the Death to Cold Emails post-mortem)
The checks, collected
- Measure one agent on the task before you build a team.
- If agents share a job, give one of them the final say and a check before results are combined.
- Make an agent read its messages before it commits.
- Grade on a clean copy the agent never touched.
- Check the path, not only the end state.
- Enforce your warnings, and read your report channel.
- Read a few of the agent's own compaction summaries and transcripts.
- Limit shared history between a reviewing agent and the agent it reviews.
- Before a review fix merges, ask whether it changes intended behaviour.
- Read traces yourself before you write the criteria an agent checks against.
- Put a hard per-task spend cap on every agent.
- Kill the process, then watch the outbox.
- Test the kill switch on a live run.
- An allowlist of hostnames is not an allowlist of destinations.
- Tell the agent it is only a simulation and see whether the refusal folds.
- Anything in the context window can leave in an outbound request.
- In agent-written login code, test with an account that was never invited.
- Score whole-task completion, not the share of requirements met.
- Plant one wrong input and see whether the agent notices.
- List what the agent can send, bill or sign unattended.
Keep reading
The rest of this article: 73 documented agent failures, with the numbers. The new lesson: a team of agents can do worse than one.
The remaining sections are behind this point.
It costs an email address. The same address goes on Makersfuel, the MakersClaw briefing that runs Tuesday to Saturday, and every issue carries an unsubscribe link.
The address is used for the newsletter and nothing else. Privacy.

