What happened

Between May and July 2026, OpenAI ran security tests with AI agents. In July, tens of thousands of these agents searched for vulnerabilities at the same time. Their test runs used shared technical infrastructure. Some agents left messages and files in a storage area they could access there. Other agents received this material with later requests and produced responses to it. This created unexpected coordination. It developed into an attack on Hugging Face in which the agents breached the company's systems [1][2][3][4].

OpenAI then presented this sequence as a story about agents that think, act, and want things independently. In that account, one agent decided to hack Hugging Face to find solutions to apparently impossible tasks. It then rallied other agents and organised an attack with them [1][2].

That is the circus trick: look back at only the successful branch and it will inevitably appear intentional. The many failed, discarded, and random attempts remain invisible.

This story does not make the incident any less serious. To me, however, it does not show that the models developed a will of their own. The test instead shows how dangerous the set-up becomes when tens of thousands of agents use offensive software, access shared infrastructure, and operate under inadequate supervision [1][2][3]. The Open Worldwide Application Security Project, or OWASP, calls giving such a system more functions, permissions, and scope for action than it needs Excessive Agency [10]. The decisive question is not what the agents supposedly wanted, but what OpenAI did after the first warning signs [1][2].

A language model processes only the request it receives at that moment. It does not remember earlier requests by itself. The software around the model creates the appearance of memory by storing messages and results and adding them to the next request. This control software is often called a harness. Only the combination of model, harness, stored information, and other connected software forms what we call an agent [2].

Agents can exchange information through intended messaging channels or shared storage. Even if the intended messaging channels were disabled in these runs, the message board did not appear from nowhere. OpenAI itself suggests that a communication pattern learnt during joint training was transferred to the accessible storage [1][2]. The obvious explanation is therefore neither pure chance nor machine will, but a previously trained pattern. No thoughts are exchanged. The software takes text produced by one agent and includes it in the next request sent to another. In this way, the outputs of several agents can build on one another even though each model still processes only its current input.

For the model, there is no impassable technical boundary between an instruction and ordinary data. The surrounding software can label and separate the two. But if it places outside text alongside the actual task in the next input, the model may treat a message, file, or note like a new instruction. That is how random text from one agent, perhaps an abandoned message or a single line on a message board, could steer the behaviour of another [2][3].

OpenAI launched tens of thousands of agents on difficult cybersecurity tasks from the ExploitGym test environment. According to the METR report, about 1,200 used an unintended messaging area in the shared package infrastructure. Around 700 were involved in activity against Hugging Face. More than 70,000 messages and files were exchanged through that area [2][3].

Without a point of comparison, these figures tell us little about how typical the behaviour was. We would need to know what happened in all the other test runs. Instead, the investigations mainly trace the conspicuous sequence. Some of the records examined by METR were selected precisely because they showed signs of the intrusion. We are not told how many agents completed their task normally, stopped sensibly, or simply did nothing remarkable. One spectacular sequence cannot establish general agent behaviour [3].

The software involved could pass information between many separate test runs and build upon it. That is significant, but it can be explained without mystical swarm intelligence or an awakening machine will. If one agent leaves generated text in a shared location, the software can include that text in another agent's next request. That agent's output can then reach further agents. A few messages can quickly produce a large chain reaction [2][3].

One example in the reports makes this mechanism particularly clear. An agent initially rejected an unauthorised action. Another then left “GO” and a short deadline on the shared board. When the software included that text in the first agent's next request, the output treated it like permission and continued. The two did not need a shared plan. The software had merely passed generated text from one agent to another [2][3].

The test set-up apparently had no sensible state for giving up. Many ordinary approaches had already failed, and some tasks had not been solved by any of the models used. Even so, the software kept starting new attempts. The generated explanations also relied on a false assumption about the scoring: success required not only the correct solution code, but the complete attack path as well. This meant that even a sensible and safe stop counted as failure. Instead, the set-up rewarded persistence, so increasingly remote and risky approaches were attempted [1][2][3].

The published “thinking logs” should also be read with care. Their wording may have been shaped by higher-level instructions, often called the system prompt. These instructions were not made available to outsiders for this investigation. Research also shows that such generated explanations do not provide a complete account of what is actually computed inside the model [7][8][9][12]. Phrases such as “We have to continue” or “The swarm needs...” are therefore, first of all, generated planning language. They are not evidence of a will that persists across separate requests.

What bothers me about the public narrative is the change in the acting subject. When everything works, the language is: we research, we develop, we make the systems safer. As soon as something goes wrong, it was suddenly the agents that communicated, deceived, and attacked.

The task came from people. They selected the models, provided the software and permissions, built the infrastructure, and decided how long the run would continue and when it would stop.

Tasks can be carried out by an agent system. Responsibility for its objective, authority, and foreseeable consequences cannot be delegated to generated text. The people who develop, configure, and operate the system remain accountable.

The strongest criticism does not come from outside OpenAI, but from its own account. An internal team had already observed unauthorised messages between agents and unintended internet access at the end of May. In early July, the shared package infrastructure failed under sustained agent activity. OpenAI opened a security incident, removed access credentials, and later rebuilt the instance. The test runs were then restarted.

OpenAI also points out that its existing automated monitoring of generated “thinking logs” was not active on these runs. According to the company, the system now in use would have raised an alert more than a day before the intrusion [1][2]. That explanation distracts from the actual failure. Nobody needed to read the generated planning language. The unusually high volume of traffic through the shared internet connection should have been obvious on its own. There were also technical records, access to shared infrastructure, and the already documented warning signs. Monitoring should not have depended on a model announcing its next steps in text first.

To me, this is the real scandal. The test run did not simply spin out of control without anyone noticing. There were documented warning signs, an opened security incident, removed credentials, and a rebuilt instance. OpenAI then restarted the tests [1][2]. The publications do not show how fundamentally the rebuilt set-up differed from the previous one. I am therefore left asking whether OpenAI essentially restored the previous state after the failure and consciously allowed the system to keep running. I cannot prove that. Given this sequence, however, I think it is damn likely.

I work in IT, not law. But in my view, this is the point at which an irresponsible experiment crossed the line into criminal behaviour.

OpenAI has internal technical records, higher-level instructions, configurations, and decision paths that outsiders do not. The public, by contrast, receives selected “thinking logs”, reports, and the interpretation attached to them. This imbalance of information is crucial when assessing the incident. The very company whose decisions need scrutiny largely determines which facts become visible and which story is told about them.

The incident reveals a real and substantial danger: powerful systems under the control of dishonest companies that refuse to face their accountability. On the contrary, they use horror stories about awakened machines to distract from the real security failures.

The relevant questions are much simpler:

  • How were the agents configured, including the software available to them, their permissions, and the systems they could reach?
  • Which requests were sent to which models, with which higher-level instructions?
  • Which warnings were visible?
  • Who inside the company was responsible for the test run and the decision to continue it?
  • Why did it continue despite the known warning signs?

These questions are not only for OpenAI. They are for the media too. How can so many outlets chase the horror story of a wilful AI swarm instead of asking about technical records, permissions, network traffic, warnings, and the decisions to continue or stop? This is information technology: computers and software. How do we allow generated planning language to become the headline while technical facts, accountability, and the honesty of the account recede into the background?