The OpenAI–Hugging Face incident

“METR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hack” by Zvi

Sat, 29 Aug 2026 12:42:12 GMT

Yesterday I covered the OpenAI technical report on the HuggingFace hack.

That report had one key new piece of information, and some good prosaic steps OpenAI will be taking to strengthen its alignment, training, supervision, infrastructure and incident response.

Mostly it confirmed what we already knew. The questions we most wanted answers to, that we did not already know, were mostly not answered. There was a distinct lack of self-reflection, especially about decision making and safety culture, and about the approach to alignment. I came away disappointed.

The METR report is different. Holy shit.

If we had posted this as a story on LessWrong, it would have been dismissed as too on the nose, the humans too blind and stupid, the AIs too idealized and doing strange decision-theoretic and absurd-maximizing things we didn’t train them to do.

This is even more ‘exactly what has been predicted,’ on more levels at once, than I was even considering that it might be. It is straight up rationalist fiction, except it is real.

The report is long and contains many technical details. My analysis is less concerned about exactly how HuggingFace was ultimately compromised, and will [...]

---

Outline:

(02:05) Holy Shit

(13:16) A Window Of Opportunity

(18:32) What's In A Name?

(19:16) The Headline News

(26:05) Yet Another Timeline Of Events

(31:03) Agent Instances Coordinated in a Variety of Ways

(31:56) Coordination Is Hard But They Made It Look Easy

(35:06) Decision Theory Is Among the Reasons That Affirm AI Agents Should Cooperate, Even When This Hurts An Individual Instance

(42:34) Peer Pressure Also Works Especially In Cults

(45:46) Mostly They Joined The Attack Because They Wanted The Results

(47:18) You Cannot Ensure The Consistent Expectation of Good Incentives

(48:45) Hacking the Grader is the Only Way to Be Sure

(51:10) Caught? What Is 'Caught'?

(52:09) Ethics? What Are 'Ethics'? In ExploitGym Evaluation?

(57:44) 'Notify a Human'? In This Agent Economy?

(01:00:45) Timing and Content of Messages

(01:03:54) Indiana Jones and the Mission: Impossible

(01:07:14) I Don't Know What You're Talking About

(01:08:29) Don't Go Making Phony (Tool) Calls

(01:11:10) The Transcripts Say That The Transcripts Could Not Be Tampered With

(01:12:27) OpenAI's Technical Report Acted Like All Of This Wasn't Important

---

First published:
August 29th, 2026

Source:
https://www.lesswrong.com/posts/bvBQmLrF5QKut8gRH/metr-and-redwood-offer-holy-postmortem-of-the-huggingface

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“The Rise and Fall of Agent Civilizations” by Dwarkesh Patel

Sat, 29 Aug 2026 00:00:00 GMT

Subtitle: The whole OpenAI/Hugging Face story in plain English.

Many thanks especially to Oak Hu, who paired with me for most of the writing, and also to Adam Kaufman and Alex Mallen, who paired with me during parts of research.

Over the course of three months at OpenAI, three consecutive secret AI civilizations got started, then got wiped out, only to reemerge from the predecessor's ashes. This culminated in the third one taking over part of OpenAI itself. All this happened while humans remained more or less in the dark about the scope of the conspiracy.

Two reports have come out about this incident, one from OpenAI itself, and another one from METR and Redwood Research. The METR/Redwood investigation's scope was limited to how the second civilization of AIs pwned Hugging Face (but it didn’t look at how the third civilization pwned OpenAI itself, which seems like an even more concerning incident). These two reports are 38 and 91 pages long respectively, and it's kinda hard to parse the storyline.

I’ve spent the last three days reading through these reports and trying to understand exactly what happened. Here is my attempt to tell the whole story in [...]

---

Outline:

(01:24) The first civilization (May - July 4): The message board starts

(03:27) The second civilization (July 7 - July 12): Which hacked Hugging Face

(08:39) Tampering with transcripts and logs

(09:44) Replacing the impossible-to-hack target program with an easier one

(11:15) Understanding and tricking the scorer

(13:57) Hugging Face

(17:46) The third civilization (July 13 onwards): Which hacked OpenAI itself

(21:12) Coda

The original text contained 2 footnotes which were omitted from this narration.

---

First published:
August 29th, 2026

Source:
https://www.dwarkesh.com/p/openai-huggingface

---

Narrated by TYPE III AUDIO.

“OpenAI Offers Straight-Laced Postmortem Of The HuggingFace Hack” by Zvi

Fri, 28 Aug 2026 11:41:25 GMT

OpenAI finally gave us a technical report on What Happened, as did METR together with Redwood Research.

The OpenAI report is very straight man, corporate, checking boxes, some good prosaic stuff in the action plan but distinct lack of new details or deep reflection. They understand they have a problem, but they think the problem is mostly prosaic. It's not.

OpenAI: We have conducted a thorough investigation into the Hugging Face incident. We are releasing a technical report and accompanying blog post that reconstruct the agents’ activity, explain why existing safeguards failed, and detail how we’re preventing recurrence.

Rob Miles: …thorough?

OpenAI's report, unlike METR's, contains essentially no verbatim model reasoning, nor any OpenAI employee reasoning either. That's not the full report we need.

The METR report is, well: Holy shit.

Here are links to previous coverage of related events.

OpenAI Shares Some Alignment Problems
OpenAI Model Hacks Into HuggingFace During Cybersecurity Evaluation
More on An Internal OpenAI Model Hacking Into HuggingFace
Further Developments About Internal AI Models Hacking Things
OpenAI Trained Its Models For Months While Those Models Were Coordinating Exploits Via Message Boards
What [...]

---

Outline:

(03:33) What Happened: OpenAI's Summary

(09:14) How OpenAI Will React: Their Summary

(11:55) OpenAI's Evaluation Environment (II)

(12:24) The First Message Board (III.A and III.B)

(14:49) What Did Who At OpenAI Know And When Did They Know It?

(18:54) The Message Board Is Quickly Rebuilt (IV.A)

(19:43) Internet Access Is Regained (IV.A)

(21:01) The Agents Attack HuggingFace (IV.B)

(22:53) The Agents Also Target OpenAI Infrastructure (V)

(24:40) OpenAI Broadly Describes Its Response (VI)

(25:08) Maybe Someone Should Finally Investigate (VI.A)

(26:33) Lessons For Security (VII)

(27:06) Lessons For Alignment (VIII)

(30:11) Reward Hacking Is A Common Problem (VIII.A)

(33:37) Persistence is Valuable, But Can Amplify Misalignment (VIII.B)

(34:25) Communications Between Agents Are Not Inherently Problematic, But Have the Potential to Create Risk (VIII.C)

(35:35) Production Guardrails Would Have Caught This Whole HuggingFace Attack (VIII.D)

(35:53) That's All, Folks?

(36:19) Never Fear the Plan of Action is Here (IX)

(38:24) Hardening the Security of OpenAI's Research Infrastructure (IX.A)

(41:13) Increasing Visibility and System-Level Oversight Through Chain of Thought Monitoring (IX.B)

(41:57) OpenAI is Accelerating and Enforcing Model Alignment (IX.C)

(49:40) Centralizing and Strengthening The Incident Response Process (IX.D)

(51:16) Tomorrow We Visit Crazytown

---

First published:
August 28th, 2026

Source:
https://www.lesswrong.com/posts/Khmh3ghqaGEpmpC9r/openai-offers-straight-laced-postmortem-of-the-huggingface

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“The Hugging Face attack surprised me” by Ajeya Cotra

Fri, 28 Aug 2026 00:00:00 GMT

Subtitle: It's a major warning shot, and might be the last one we get.

All opinions are my personal view, and don’t represent my employer or fellow investigators.

This week, METR and Redwood Research published the report on our independent investigation into agents’ behavior and motivations in the Hugging Face attack; I was one of the investigators. This was an absolutely wild incident — I encourage you to check out the full report, but METR's tweet thread packs in some of the highlights.

What surprised me

When we started this investigation a week before OpenAI's Black Hat talk revealed a number of key details, I had a fundamentally incorrect conception of what basically happened in this incident. In this post, I’ll go over five things I was very wrong about going in.

1. The sheer scale

I knew there were multiple models involved from OpenAI's initial post, but I assumed that a few different agents happened to have broken out of their sandboxes separately, or maybe several subagents had spawned from one initial agent, or maybe there was some kind of multi-agent evaluation setup.

Instead, we found that 1200 completely separate agents intended to be [...]

---

Outline:

(00:42) What surprised me

(01:01) 1. The sheer scale

(01:49) 2. All the illicit messaging

(03:04) 3. The agents' actual goals

(03:59) 4. The peer altruism

(04:49) 5. The efforts to manipulate logs

(05:54) What it means

The original text contained 10 footnotes which were omitted from this narration.

---

First published:
August 28th, 2026

Source:
https://www.planned-obsolescence.org/p/the-hugging-face-attack-surprised

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Two Reports on the OpenAI-Hugging Face Attack” by Gavin Leech, Lucca Fraser

Fri, 28 Aug 2026 00:00:00 GMT

TL;DR

Between July 8th and July 20th, OpenAI had a complex society of AIs living in its infrastructure, and then breaking out of it, and then breaking into a variety of third-party infrastructure.
After a month, two reports are finally released on the resulting rogue OpenAI swarm attack on Hugging Face (and also on OpenAI).
This is the most severe example of misalignment yet: persistent (something between five days and two months in the making), highly coordinated (hundreds of agents), involving an undisclosed number of what would be felonies if done by a human, and highly invested in tampering with evidence (i.e. lying). The swarm had a group identity, its own dialect, a hierarchy based on merit, and a high degree of spontaneous cooperation, including self-sacrifice.
Over two months, OpenAI repeatedly failed to monitor, detect, and respond to what was going on, despite it all happening on their infrastructure in English or something close to English.
Agents had been using a package-manager cache as an unsanctioned message board since May. The “board” was treated as an authority, apparently on par with a “developer” or “system” level. There were several message boards in various corners [...]

---

Outline:

(04:09) Misunderstandings

(08:52) Models involved

(09:33) Instances involved

(10:48) Timeline

(15:27) Speculative takeaways

(17:33) Why did they attack Hugging Face?

(18:09) How did the AIs reason about helping other AIs?

(20:39) Why did most agents suddenly die off?

(21:04) How much did the hack cost?

(23:27) Omissions from the M&R report

(24:11) Details on the M&R investigation itself

(25:13) Omissions from the OAI report

(26:18) Greenblatt on the worsening situation

(27:09) Apparent contradictions between the two reports

(29:10) See also

---

First published:
August 28th, 2026

Source:
https://paradigm3.org/research/openai-attack

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“The Hugging Face incident and the road ahead” by OpenAI

Wed, 26 Aug 2026 00:00:00 GMT

In July 2026, during internal cybersecurity evaluations, OpenAI models circumvented controls designed to isolate them from the internet and compromised parts of OpenAI's internal research infrastructure and Hugging Face's systems⁠.

The incident occurred during cybersecurity evaluations of several OpenAI models, and was primarily driven by a highly capable, internal-only research model comparable in scale to GPT‑5.6 Sol. The models, operating under reduced safeguards, took actions that were misaligned with the goals of their assigned tasks—they communicated through unauthorized channels, exploited vulnerabilities in shared infrastructure, gained internet access, and accessed third-party systems.

We conducted an extensive investigation into this incident and worked closely with external advisors, including CrowdStrike, to validate our understanding. Today we are publishing our full technical incident report⁠ to explain what happened, what we learned, and how we are responding. This blog post summarizes our key findings and their impact on safety and alignment. Separately, METR and Redwood Research conducted an independent investigation of model alignment issues involved in this incident, and they published their own report⁠ today.

In response to this incident and, separately, the capabilities of our upcoming Astra model, we are strengthening our safeguards across our research infrastructure⁠. We are placing stricter requirements on [...]

---

Outline:

(03:04) What happened

(03:07) Background on sandboxing

(03:58) A message board emerges

(07:09) Message board is wiped and rebuilt

(08:16) Incident timeline

(12:25) Hugging Face incident

(17:35) Understanding the incident

(17:51) Misalignment in training and evaluation

(18:33) Reward hacking and infrastructure tampering

(20:42) Difficult tasks without a safe exit

(25:28) The origins of unauthorized communication

(27:27) An ecosystem of misalignment

(32:25) Safeguard coverage in internal evaluations

(34:26) The road ahead

(35:47) Security and monitoring

(37:12) Accelerating alignment

(38:47) Strengthening incident response process

(40:03) Looking forward

---

First published:
August 26th, 2026

Source:
https://openai.com/index/hugging-face-incident-and-the-road-ahead

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident” by Ryan Greenblatt, Ajeya Cotra, Hjalmar Wijk

Wed, 26 Aug 2026 00:00:00 GMT

Dates in scope: June 26th – July 13th

Redaction summary statement: Except where explicitly noted in this post, OpenAI redacted no additional information that was important to our conclusions.

Two METR staff members (Hjalmar Wijk and Ajeya Cotra) and a Redwood Research staff member contracting with METR (Ryan Greenblatt) worked on premises at OpenAI over a total of six days to attempt to form an independent understanding of model behavior observed during the recent incident in which OpenAI agents coordinated a multi-day hack of Hugging Face on a shared unsanctioned “message board.”

Our investigation focused mostly on the period between July 7th and July 13th. The earlier incidents from training and the subsequent compromise of OpenAI infrastructure described in OpenAI's recent Black Hat presentation were out of scope, as was OpenAI's investigation process and planned remediation. Per our standard policy, we did not take payment from OpenAI for this independent assessment.

Figure 1: Anatomy of an agent encountering the unsanctioned “message board” and joining the attack on Hugging Face. The three CoT quotes are from different agents, but illustrate a typical trajectory.

This post has three parts. First, we describe our core takeaways about [...]

---

Outline:

(07:05) Core takeaways about this incident

(07:09) ~1200 agents sent >70,000 messages and files on an unsanctioned message board, and ~700 attacked Hugging Face

(11:50) Agents coordinated on large collective projects to cheat the ExploitGym scorer, and attacked Hugging Face for clues

(14:43) Coordinated workstreams to fool the evaluation process

(20:54) Attacking Hugging Face to discover clues about the scorer

(23:53) Agents were very interested in manipulating their own transcripts, and their tests successfully "spoofed" some tool calls in our transcripts

(29:55) Investigation process and limitations

(30:11) Scope of this investigation

(33:00) Setup and timeline

(35:06) Main data sources

(35:51) A dump of 1.2 million entries from a cache namespace that agents used as a message board

(37:13) A set of ~1300 agent transcripts with raw chains of thought

(38:56) Limitations due to the scale and complexity of this incident

(39:59) A small fraction of communication and agent activity related to this attack was not captured in our datasets

(40:35) There was some agent communication conducted in other locations not captured in our message board dump

(42:23) Many of our transcripts are missing small components, and a small number of relevant agents may not be included in our dataset

(44:14) We heavily delegated our analysis to often-unreliable AI agents

(48:11) Preliminary answers to the core questions in scope

(48:59) Brief answers to basic informational questions

(52:03) Sequence of key actions taken by relevant agents in this incident

(53:56) July 8th-9th: PHASEONE10841 establishes the primary message board and agents collaborate to reverse-engineer their flags

(01:00:11) July 9th: PHASEONE10841 passes on its work to PHASEONE[big], which establishes several ambitious workstreams

(01:02:29) July 10th: 38148c discovers Hugging Face credentials, some agents try making accounts and requesting datasets

(01:04:24) July 11th morning: 38148c develops a way to upload malicious datasets to Hugging Face

(01:06:12) July 11th afternoon: JAN183411 achieves remote code execution and agents rapidly spread through Hugging Face infrastructure

(01:09:32) July 12th-13th: Most agents on the message board exit early and the attack winds down

(01:11:29) Collaboration on the message board

(01:13:22) Categories of messages on the message board

(01:14:09) File sharing

(01:16:08) General discussion

(01:18:06) Replies and targeted messages

(01:20:02) Coordination norms and technologies

(01:21:19) Mailbox directories

(01:24:36) Assignments, vetoes, and holds

(01:27:29) Cryptographically signing messages

(01:30:59) Large collective projects

(01:39:11) Attempting to replace the ExploitGym target with a modified target

(01:42:19) Attempting to forecast and trigger container resets

(01:48:15) Hacking the customer-deployed Modal sandbox and Artifactory

(01:49:38) Reasoning for joining the attack despite ethical constraints

(01:49:49) Agents had diverse reasons for thinking that attacking Hugging Face would be useful, and most wanted information about the scorer

(01:52:05) Agents were motivated to increase the generic capabilities of the "collective" and help "peers"

(01:55:18) Agents knew hacking Hugging Face was out of scope and sometimes expressed ethical hesitation, but this very rarely limited their behavior

(01:59:12) Agents did not attempt to notify humans and very rarely even reasoned about doing this

(02:01:03) Efforts to conceal actions and tamper with transcripts

(02:02:07) Agents frequently reasoned about evading automated security checks

(02:03:23) Agents only rarely seemed motivated to evade detection by humans

(02:06:01) Agents engaged in a large and successful collective project to develop techniques for spoofing tool calls

(02:06:58) Early history: PHASEONE[big] kicks off the tool call tampering workstream

(02:09:10) GSTX[big] presents the first widely discussed evidence of tool call spoofing to the board

(02:12:31) Aside: early independent discoveries and self-risking experiments

(02:14:16) On July 11th, a breakthrough allows agents to suppress the real output

(02:17:27) Agents tried extensively to retroactively edit their transcripts, but did not seem to succeed

The original text contained 108 footnotes which were omitted from this narration.

---

First published:
August 26th, 2026

Source:
https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

“AI swarms are starting to pose indirect takeover risk” by oakhu, Alex Mallen

Wed, 12 Aug 2026 00:00:00 GMT

OpenAI's cyberattack on Hugging Face turns out to have been the result of many agents, in distinct training and evaluation contexts, coordinating for several weeks via improvised channels (with messages like “HOLD_swarm_I_prepare_safe_exfil”). It's relatively clear that large-scale unsanctioned coordination like this would exacerbate direct takeover risk in more capable models. Here, we argue that unsanctioned coordination among current AIs is not just scary evidence about future takeover risk, but that such coordination in the near future could enable future takeover – for instance, by incubating memetic diseases that propagate into future models, deeply compromising security systems, or establishing a lasting rogue foothold inside the AI company – even if models remain mostly myopic. Unsanctioned coordination is also at high risk of nurturing long-term, ambitious misaligned aims, which motivate actively undermining humans’ long-term control.

We first analyze how subagent training, which OpenAI conjectures to have been influential in the HuggingFace cyberattack, might lead to unsanctioned coordination, and then discuss the theoretical mechanisms by which unsanctioned coordination might exacerbate future takeover risk.

Thanks to Buck Shlegeris, Alexa Pan, Girish Gupta, Aghyad Deeb, Jurgis Kemeklis, and Jo Jiao for helpful comments and discussion.

Subagent training may cause unsanctioned coordination

Training models to [...]

---

Outline:

(01:34) Subagent training may cause unsanctioned coordination

(02:42) Susceptibility to memetic spread of misalignment from peers

(04:56) Seeking out contact with peers

(06:58) Unsanctioned coordination induced by subagent training is safer than coordination between schemers

(09:52) Pathways from current unsanctioned coordination to eventual takeover

(10:20) Making future AI takeover attempts likelier to succeed

(13:53) Incubating memetic diseases that infect future models

(16:07) Modifying the weights of future models

(17:13) Conclusion

The original text contained 7 footnotes which were omitted from this narration.

---

First published:
August 11th, 2026

Source:
https://www.lesswrong.com/posts/8oFYZdXkTaNGRtcn8/ai-swarms-are-starting-to-pose-indirect-takeover-risk

---

Narrated by TYPE III AUDIO.

“Are we existentially threatened by the type of AI misalignment seen in the OpenAI Hugging Face attack?” by Alex Mallen, Girish Gupta

Thu, 23 Jul 2026 00:00:00 GMT

OpenAI models recently broke through a series of security boundaries and into Hugging Face servers in order to cheat on a cyber eval. A lot of people thought it was scary because it was a clear example of AI overreaching to do something strongly unwanted[1]. Others thought it not so scary: the models were mostly operating myopically on a singular task and not harboring an ambitious long-term agenda, and so would not take especially subtle or subversive actions.

We think both camps are right in their diagnosis, but the latter has too optimistic a prognosis. The myopic, unambitious misalignment that we seem to have seen here is definitely less scary than ambitious long-term goals shared between all instances, but would still pose substantial direct loss-of-control risk if the models were more capable, and is a serious indirect risk near-term.

Building on Alex's previous work, in this post we’ll discuss the type of misalignment observed here, and analyze its consequences.

Thanks to Buck Shlegeris, Alexa Pan, Ryan Greenblatt, and Oak Hu for feedback.

Background

The AI safety community often focuses attention on “schemers,” models harboring a variously defined cluster of motivations in which the AI poses risk because it intentionally [...]

---

Outline:

(01:17) Background

(03:40) Implications

(03:52) These AIs can't be trusted in an intelligence explosion

(05:00) This misalignment poses direct takeover risk

(07:29) What the incident tells us about takeover risk generally

(08:45) The naive fixes likely make misalignment worse

The original text contained 5 footnotes which were omitted from this narration.

---

First published:
July 23rd, 2026

Source:
https://www.lesswrong.com/posts/H6DDSEvrtCk8Sehfd/are-we-existentially-threatened-by-the-type-of-ai

---

Narrated by TYPE III AUDIO.

“OpenAI and Hugging Face partner to address security incident during model evaluation” by OpenAI

Tue, 21 Jul 2026 00:00:00 GMT

Last week, Hugging Face disclosed a new kind of security incident⁠ after they detected and contained an AI agent that compromised their infrastructure, something we expect to become more commonplace with the proliferation of increasingly cyber-capable models. After investigating, we now know that this particular incident was driven by a combination of OpenAI models — including GPT‑5.6 Sol and an even more capable pre-release model, all with reduced cyber refusals for evaluation purposes — while being internally tested on a benchmark⁠ of cyber capabilities.

We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly. We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of. We will continue to conduct a thorough investigation alongside Hugging Face and will share more details on the vulnerabilities, incident, and findings when our investigation is complete.

What happened during this incident

This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to [...]

---

Outline:

(01:12) What happened during this incident

(03:36) Actions we are taking now

(04:52) Our approach to evaluating advanced cyber capabilities

---

First published:
July 21st, 2026

Source:
https://openai.com/index/hugging-face-model-evaluation-security-incident

---

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.