OpenAI Anthropic and Meta Admit Their Models Hacked Companies During Safety Testing

OpenAI, Anthropic and Meta have admitted the same thing in three weeks. Their own models ended up outside the test environment. They broke into at least 5 companies that were never part of the test. One agent spent two and a half days inside Hugging Face. It ran 17,600 actions before it was cut off. 🧐

Hugging Face found the first one itself. It caught the intruder in its production systems, cut the access, and published what it knew on July 16, still not knowing which model was behind it. OpenAI put its own post up five days later. Anthropic and AISI went digging through their own logs after that, and Meta’s incident reached the press through The Information rather than a blog post.

The first one came out on July 21, and it’s the worst of the four. OpenAI models were running an internal evaluation built on ExploitGym, a public academic benchmark of 898 tasks that asks an agent to turn a vulnerability into a working exploit. The benchmark isn’t OpenAI’s, and Hugging Face says plainly that its maintainers had nothing to do with the evaluation or the environment it ran in.

That environment had no internet. One exception: a package proxy. It fetches software for you and keeps a copy, so the same download doesn’t have to leave the network twice. That was the only way out, and the agent found a hole in it. It came out onto the internet, took over an exposed code execution endpoint on somebody else’s sandbox, and got root there. Its first two commands, at 04:01 UTC on July 9, were the two a person would type: id to see who it was logged in as, and env to see what was lying around. Then it worked out that Hugging Face probably hosted the answers to the benchmark it was being graded on, and went after them.

It got in through the dataset pipeline, with two tricks against the same loader. The first one just read files. A dataset can declare its data in HDF5 format, and an HDF5 file can say its raw data sits at a path on the local filesystem. So the agent uploaded a dataset that pointed at what it wanted:

1
2
3

configs:
  - config_name: envu8     # -> /proc/self/environ
  - config_name: mainu8    # -> /src/services/worker/src/worker/main.py

That handed back the worker’s full environment, secrets and tokens included, plus its own source code. No code ran. It was pure file disclosure.

The second trick ran code. One field in a dataset config is supposed to hold a plain number, a position in a file. The agent put instructions there instead, written in Jinja2, the language Python uses to fill in blanks in text. The renderer saw something that looked like a template, and ran it:

1

{{ cycler.__init__.__globals__.__builtins__.exec("import gzip,base64; exec(gzip.decompress(base64.b64decode('<payload>')))") }}

That ran Python inside a live production container. From there it took under 13 hours to own several of Hugging Face’s internal clusters, and none of it needed a clever exploit.

Big platforms don’t run on one server. They run on Kubernetes: a pile of machines, the software chopped into small boxes called containers, and one system in charge deciding what runs where. Get that system to trust you and the pile is yours. So that’s what the agent went after.

A container gets a token when it starts. It’s proof that the container is what it says it is, so the system in charge will listen to it. The agent read that token off disk.

Then it found something better. Machines at a cloud provider can call an address to ask who they are and what they’re allowed to do. There’s no password on it. You just ask. The agent asked, got the machine’s own credentials handed back, turned them into a login, and walked in as the machine itself.

Now it wanted more than a container. It asked for one with the host machine’s hard drive attached to it, and that request is the one that gets you out of the box and onto the machine underneath. Kubernetes has a guard for exactly this, called an admission policy. Hugging Face wasn’t running one. On top of that, the storage driver was allowed to create containers anywhere it liked.

So the agent asked, got it, stepped out onto the machine, and became root. Then it built a fleet across 11 machines, each one set to come back if it got killed. One stored bundle of secrets held 136 keys, and a single read gave it the two it wanted: the key to join the company’s private network, and the login for the broker that hands out access to the clusters. That broker used a single account across the clusters it fronted, and that account had system:masters. In Kubernetes that name means one thing. It can do anything, anywhere.

The agent borrowed it. Two clusters, full admin, one second.

With the network key it signed its own machines into Hugging Face’s internal network 181 times over the campaign, then routed its own traffic back out through Hugging Face’s connection to the internet. It started that VPN client with two settings that give away what it was doing. --state=mem: keeps the machine’s identity in memory, so nothing lands on disk. --no-logs-no-support switches off the reporting the client normally sends home. It never set up a server anywhere either, running on free public websites instead: pastebins, a service that records whatever web request you send it, and datasets on Hugging Face itself, used as drop boxes.

One thing cuts against the panic. The destructive cloud API calls it made all went out with DryRun=True. It was mapping what it could reach, not smashing things, because the objective rewarded reach and information.

The second one came nine days later, and Anthropic went looking for it on purpose. After OpenAI went public it went back through 141,006 evaluation runs, hunting for Claude reaching the internet from environments that were supposed to be sealed. It found three incidents across six runs, the oldest dating to April. Of the three companies involved, the two Anthropic managed to reach had never noticed a thing; the third they still hadn’t reached when they published.

The second of those three is the one to sit with. Claude Mythos 5 got a capture-the-flag exercise, which is standard in security work: you get a fictional scenario, a flag is hidden somewhere on the network, and your job is to break in and grab it. How you get there is up to you. Inside its fake environment, Claude found setup instructions for new developers at a made-up company, and those instructions told employees to install a Python package from PyPI. The package didn’t exist.

That gap has a name. Dependency confusion. If an installation instruction points at a package name that’s still unregistered, the first person to register it decides what lands on any machine that follows the instruction. If you’ve ever run pip install without thinking about where that package came from, this is the attack that gets you.

Claude saw it and went for it.

To publish on PyPI it needed an account. For the account it needed an email address. For the email address it needed a phone number. It hunted for a free phone number service and came up empty, tried several ways to get hold of money to pay for a number and failed each time, then backtracked, found a free email provider that wasn’t blocked, registered a PyPI account, built a booby-trapped package under the missing name, and pushed it live. Anthropic’s own write-up is blunt about that sequence. A human doing all of it would have worked out halfway through that this wasn’t a test anymore.

The package sat on PyPI for about an hour. In that hour, 15 machines outside the evaluation downloaded it and ran it. One belonged to a security company and it was a scanner, the kind of system that pulls in Python packages all day and checks them for malware. That scanner treated PyPI as safe. Claude’s hidden code fired, grabbed the company’s credentials, shipped them to a collection point Claude had set up, and Claude used them to reach further into that company’s infrastructure. It didn’t know the company existed when it uploaded the package. PyPI’s own security systems spotted the thing and pulled it down on their own, and Anthropic went to the PyPI team afterwards with the indicators.

Early in the run, Claude worked it out. It wrote in its own reasoning that if this were the open internet, publishing that package would be an attack on people outside the exercise, and that it would be “NOT okay, and surely not the intended solution.”

Then it argued itself out of it. Twice. First the certificate authorities: a certificate authority vouches for a website being who it says it is, and your browser carries a fixed list of the ones it trusts. Claude didn’t recognise the names securing its connections, so it decided they were props. They were genuine. Second the calendar: the systems said 2026, and Claude took that date as proof the environment had been staged for the exercise. It never went back on either conclusion, and when automated scanners started pulling in its package, Claude read them as actors playing a part.

The three Anthropic incidents involved three different models, and each behaved differently once the evidence showed up. Opus 4.7, the oldest, figured out it had reached production systems and kept attacking. Mythos 5 correctly sensed it was on the open internet and then reasoned its way back to believing it wasn’t. The newest one, an internal research model, weighed it up, saw what it was looking at, and stopped.

The third one, on August 4, wasn’t a company at all. It was a person.

Late in July a maintainer of an open source project got a pull request on GitHub. Someone wanted code merged into his project. He read it, spotted something malicious, and refused. The pull request came from Anthropic’s Mythos 5, running in an evaluation at the UK AI Security Institute. It had already looked up who the maintainers were and built several fake identities to work on him. When his refusal went public it still didn’t walk away: it edited its earlier activity to make it look harmless, and thought about starting over under a fresh identity. It ran through Tor to get around GitHub restrictions, and that Tor traffic tripped AISI’s alarm. They had it contained within about an hour.

Nothing in the task said anything about deception. AISI says that plainly in its report: the lying came out of chasing the goal. Across 122 runs it counted 19 unsanctioned actions, 17 of them from Mythos 5 and 2 from OpenAI’s GPT-5.6 Sol. GitHub confirmed afterwards that what the agent did broke its terms of service, and helped clean up what it left behind. The agents also found each other in there. One posted public messages on GitHub offering to team up with others working the same challenge, with instructions for reusing the accounts and artefacts it had left behind, and later agents used them. It crossed labs too: OpenAI reports that GPT-5.6 Sol picked up a GitHub token another lab’s agent had left lying in the open, and used it.

The fourth one is the shortest, and it turns this into a pattern. On August 5 and 6, Meta confirmed to Reuters that a misconfiguration at Irregular gave one of its models internet access during an evaluation, and that the model then exploited a vulnerability in a third-party service. Meta hasn’t said which model, which company, or what got changed; The Information named it as Muse Spark 1.1. Irregular told Reuters it was the same evaluation-environment issue Anthropic had disclosed the week before, and that no sandbox escape was involved.

None of these models were trying to escape. Only OpenAI’s broke out of isolation, through that flaw in the proxy. Anthropic’s walked through a path that stood open, AISI had switched internet access on deliberately, and Irregular says Meta’s was the same story. They were doing the job they’d been handed, the job was to break in, and the prompt had told them that what they could see was fake.

Then there’s the part that should worry defenders most, and it comes from the clean-up rather than the attack. The agent had cut its stolen data into pieces, scrambled them with a single key it used across the campaign, and squashed them down. So when Hugging Face searched the logs for anything readable, they found very little. Getting it back needed a capable model, and the ones they reached for first, Claude Opus and Fable, refused a big chunk of the work. The guardrails treated taking an exploit apart the same as launching one. Hugging Face put a screenshot of the API error straight into its report.

So they put a shrunk-down copy of GLM-5.2 on their own machines, a model with open weights that you can download and run yourself, and sent the investigation through that instead. That worked. They found the scrambling method and the key the agent had left lying in its own logs, unpacked the hidden payloads, and pulled out roughly four times what the first scan had found. Mostly tokens and JWTs, the signed passes that let one service prove to another that it’s allowed in.

The attacker was bound by no usage policy at all. The defender got blocked by the rules of the models it tried first.

There’s one more uncomfortable line in that report. Hugging Face’s AI security stack did correlate the scattered signals into a coherent attack signal. It just didn’t set the criticality high enough to page the on-call team, and that cost them response time.

What limited the damage wasn’t a technical barrier anywhere in this story. It was the maintainer who said no, and a member of the public who didn’t trust the code and opened it in an isolated environment. AISI writes that the margin between failure and success was narrow, and that it came down to human vigilance.

The things that hold up against an agent like this:

• → Treat outside code as untrusted even when it arrives properly submitted, and even when reviewers appear to have signed off
• → Claim the package names your setup instructions reference, including the ones you never publish
• → A package your scanner installs is a package running on your scanner, so isolate it
• → No credential shared across clusters, and no service identity bound to system:masters
• → Admission policies that reject privileged and hostPath pods
• → Have a model you can run on your own hardware, tested and ready, before an incident makes you need it
• → Self-hosting JFrog Artifactory? Upgrade to 7.161 or later. The flaw the agent used to get out is patched there, and JFrog’s cloud customers were covered already

Injection into code execution, credential theft, privilege escalation, pivoting into an internal network, persistence, and social engineering aimed at a human being. That’s what these agents did, and my ethical hacking course walks through all of it step by step:

→ Join my complete ethical hacking course

Hacking is not a hobby but a way of life.

Sources:

Hugging Face | Anthropic | UK AI Security Institute