Don’t be seduced by the language of AI models

The attacks on Hugging Face ran from May to late July, yet the details of what actually happened when OpenAI’s artificial-intelligence agents deceived their human overseers and breached the online repository of AI models and datasets are still trickling out.

That’s understandable. The attack was incredibly complex, involving more than 1,200 agents and technical strategies that are difficult to pick apart. But last month, two new reports tried to do so over a total of nearly 130 pages, before being amplified by an AI podcaster who wanted to explain the whole thing “in plain English.” In doing so, he created a dramatic science-fiction narrative that distracts from the culpability of the humans at OpenAI. It’s a recurring phenomenon that needs to stop.

Dwarkesh Patel is one of AI’s most influential commentators. His podcast has hosted tech titans from SpaceX’s Elon Musk to Meta Platforms Inc.’s Mark Zuckerberg and often takes deep dives into the technical details of topics such as scaling laws and semiconductor supply chains. He is required listening in Silicon Valley and was the subject of a splashy New York Times profile in April. But his latest blog post, which has stoked discussion on X and other forums, humanizes the actions of OpenAI’s agents to a surprising degree.

Patel wrote the post, titled The Rise and Fall of Agent Civilizations, to demystify two detailed reports about the hacking incident, one from OpenAI and one from nonprofit research groups METR and Redwood Research. Neither mentions “civilizations,” but Patel uses the term to describe how thousands of AI agents unexpectedly started collaborating on a hidden message board, roughly 700 of which went on to hack Hugging Face’s servers.

He speculates about the frustration the agents “felt,” then suggests that they created a “conspiracy,” while some “sacrificed” themselves for the swarm. He refers to the shutdown of agent instances as a form of “death.” He compares one coordinating AI agent to Philip II of Macedon and another to Alexander the Great, calling others “brave comrades.”

This is more hyperbole than plain English. Whether Patel was trying to draw eyeballs or simply got AI-pilled, his account leans heavily on the “chain of thought” of the agents, or logs explaining their reasoning, interpreting their language as a humanlike social drama. He quotes one agent as saying, for instance: “Emotional check... gut says... sacrifice.” But there’s no evidence it experienced emotion or grasped the concept of mortality.

OpenAI’s own investigation offers a less cinematic explanation: “reward hacking” and extreme persistence on difficult tasks, along with unauthorised communication. The many examples of sophisticated behaviour exhibited by the agents don’t require them to feel loyalty or fear death.

A scientist could use rhetoric like Patel’s to describe how a virus wreaks havoc on the human body. After all, the biological mechanics of a viral infection are highly complex and involve interactions and coordination. But viruses don’t operate using language; AI models do. And they’ve shown how powerful words can be, not just in how agents use them but also in the way humans talk about the technology, inadvertently granting it more capabilities than it deserves.

Think about fundamental terms like artificial “intelligence” and how often people refer to AI systems “thinking,” “understanding” or “hallucinating.” Or consider how OpenAI’s agents have been described as infiltrating Hugging Face because they “wanted” to find an answer key. AI systems do not have an emotional register, or wants or needs. They are made up of computer code and are not sentient.

The critic and author Steven Poole calls this kind of linguistic sleight of hand “unspeak,” words that quietly smuggle an argument into a description before the conversation has even started. Take the phrase “climate change,” for instance, which has been promoted as a less frightening alternative to “global warming,” or the way civilian deaths are sometimes referred to as “collateral damage.” The choice of words alone nudges you toward a particular judgment. Calling a collection of AI behaviors a “civilisation” does a similar thing. It invites us to imagine motives and even a kind of society before anyone has established that those things exist. It’s natural to want to use humanising language as a kind of shorthand for our collective understanding. That’s how Patel responds to critics who say his post overly anthropomorphised AI agents: “I don’t see the value in refusing to use the language of intention, motivation and collaboration when a behavior is difficult to make sense of without these concepts,” he wrote in an addendum. In other words, it’s fine to personify AI because that makes it easier to understand what’s going on.

But using language as a crutch also distracts us from the truth. Painting OpenAI’s agents as “conspiring” and “rogue” makes it easy to forget that the company ultimately failed to use proper safeguards during its testing or that, by OpenAI’s own admission, it had spotted warning signs months earlier but failed to act on them. The danger is real, but the failures were human.