Daybreak

On August 5, Google announced what appeared to be a corporate version of Nixon’s Saturday night massacre. Demis Hassabis stepped back from day to day operations at DeepMind. Jeff Dean, the founding father of Google engineering, is leaving to start a new lab called Discovery Loop, taking Sanjay Ghemawat, Quoc Le, and Oriol Vinyals with him. Koray Kavukcuoglu, DeepMind’s CTO, now has operational responsibility for DeepMind and Gemini

Dylan Patel and his colleagues at SemiAnalysis read this as a kind of failure in their recent newsletter “Gemini is Cooked but GCP is Cooking.” DeepMind has stopped being a frontier lab, they noted. The departures, they say, are a symptom of years of timid compute allocation and a bureaucratic, risk-averse culture. After all, Google had sophisticated conversational-AI systems well before ChatGPT, but was far more reluctant than OpenAI to put them in users’ hands. SemiAnalysis argues that Google’s failure to risk the core business has finally caught up with it.

It’s not a disaster for Google, though. In SemiAnalysis’s estimates, Google Cloud may be a much larger economic opportunity than pursuing its rivals in the frontier AI race. SemiAnalysis wrote “Our Tokenomics Model estimates that Gemini ARR was $12B in 2Q26. In contrast, by the end of 2027, GCP will be doing over $73B in third party AI ARR IaaS/TaaS and another $120B of TPU sales. $200B of external sales at high 30s EBIT margins vs a first party business generating just $12B today shows where the focus is.”

What’s more, after years of lagging Amazon and Microsoft in cloud revenue, Google seems to be gaining ground. Alphabet reported $24.8 billion of Cloud revenue in the latest quarter, up 82% year over year, compared with 37% growth at AWS and 43% growth in Microsoft’s Azure and other cloud services. The figures aren’t strictly comparable, though. Google Cloud includes Workspace and other applications, and Microsoft does not disclose Azure revenue separately from its cloud applications either, while AWS is pure cloud revenue. SemiAnalysis also estimates that TPU system sales added roughly $1.2 billion to Google Cloud revenue during the quarter.

Is this a choice by Google of profit over frontier ambition? SemiAnalysis compares it to past strategic missteps such as when IBM retreated from the PC into mainframe consulting, or when Intel retreated from Pat Gelsinger’s bold bets into its legacy chip business. Both ended up judged as major mistakes.

That may be correct. But there’s a second scenario that fits the facts, and is also rooted in history.

In the 1880s, Thomas Edison was famed as the hero of the electricity revolution. He had invented the first practical incandescent light bulb and commercialized it at scale, and had built the first commercial power plant in lower Manhattan. However, his system ran on relatively low-voltage direct current, which was practical over short distances but required generating stations close to customers. George Westinghouse bought Nikola Tesla’s patents for alternating current, which could travel for miles at high voltage and then be stepped down for ordinary use. Tesla had also developed electric motors and generators that ran on alternating current. By 1893 Westinghouse had lit the Chicago World’s Fair with AC. And by 1896, Westinghouse’s AC generators were sending power from Niagara Falls to Buffalo. Edison was the frontier leader, but Westinghouse won the race to diffuse electricity through society. (This is how it worked out even though Edison was, in many ways, right in the long term about the many applications for which direct current is superior. DC has returned as a crucial part of modern electronics, batteries, solar, EVs and high-voltage transmission. History rarely goes in straight lines.)

Jeff Ding’s book Technology and the Rise of Great Powers traces the relative impact of invention and diffusion during technology revolutions. Ding argues that nations that dominate the “leading sector” of a general purpose technology don’t reliably grow more powerful as a result. Diffusion is the defining factor. He posits that Britain’s edge in the first industrial revolution came less from inventing the steam engine and advances in steelmaking than from diffusing machinery through the whole economy so that many businesses, not just the steam engine manufacturers and the steelmakers, became more profitable. And America’s edge in the second industrial revolution had less to do with any single American breakthrough than with how fast interchangeable manufacturing, electrification, and eventually the automobile spread into every sector at once. Germany dominated many frontier industries, but there, growth and profits were concentrated in a few leading companies rather than diffused widely through society.

According to Ding, the importance of diffusion over frontier dominance continued in the 20th century. Though the US pioneered the electronics revolution, Japan appeared for a time to be winning the frontier race, at least in Ding’s narrative. It led the world in semiconductors, consumer electronics, and computer hardware through the 1980s, yet in the long run it still lost the information revolution to the US, a country that was worse at making the chips and better at putting computers to work throughout society.

I applied these insights to AI and its corporate adoption a few weeks ago in a review of Jeff’s book called Ordinary Engineers, Not Heroic Inventors. So reading SemiAnalysis, I wondered whether Google might be making the same strategic bet as Westinghouse. SemiAnalysis’s numbers can make the case that Google is betting on diffusion at least as well as the case that it is giving up on frontier leadership. So this could be read as a strategic argument inside Google that Google Cloud CEO Thomas Kurian won and that Demis Hassabis and Jeff Dean lost. Whether or not anyone inside Google describes it this way, the capital allocation increasingly looks like a strategy to become a platform for not just its own but for other people’s AI applications.

Kurian has been saying for a while that he wants Google TPUs to become “general purpose infrastructure,” serving Citadel Securities and the Department of Energy as readily as Gemini. Of course, the crucial question if we were using Ding’s framework isn’t whether Google sells lots of TPUs but whether those TPUs become complements to a broad wave of productivity-enhancing innovation in other parts of the economy.

Kurian has also defended selling compute to Anthropic as what happens when you’re a platform company. That sounds like someone who has argued that the bigger prize is being the layer that others’ AI runs on, competitors included, rather than doubling down on the potentially ruinous costs of the frontier AI race. After all, while SemiAnalysis didn’t go there, there’s a chance that if open-weight models continue to compress inference pricing, frontier labs may discover that model leadership resembles semiconductor fabrication, enormously important strategically but surprisingly poor as a standalone business.

Google also doesn’t need to win the frontier to dominate the edge. Its Flash-class Gemini models already run AI Mode in Search at enormous scale, with more user telemetry than almost any competitor. Google has enormous distribution advantages through Android, Chrome, Search, and Play. Perhaps they cede the expensive unprofitable frontier race and excel in the 120B space where affordability and local hardware are currently meeting in a sort of sweet spot. Returning to the history of electrification, it was ubiquitous small motors that powered the second industrial revolution, not giant dynamos. Buy one good small model team, my colleague Ilan Strauss speculates, and there may be no company better positioned to own the ordinary, everyday layer of AI the way Google already owns the ordinary, everyday layer of search. This may be a stretch. ChatGPT and Claude have made their own remarkable advances in the diffusion of AI, but IMO, their rhetoric about the way that AGI will concentrate enormous wealth and power in the hands of the winner doesn’t make a convincing case that they are operating with a diffusion strategy in mind. Meanwhile, both they and their customers are still in search of the widespread productivity uplift that diffusion of a general purpose technology brings.

In short, SemiAnalysis is probably right that Google gave something up on August 5, but possibly wrong about why. The evidence does not necessarily show that Google has abandoned technological ambition. Sergey Brin still seems to have plenty. But at least for the moment it may show a shift in where Google is allocating scarce AI compute and where it expects to capture economic value. It may be a mistake to assume that the AI race is about who builds the best intelligence. It may turn out to be about who builds the electrical grid. Maybe Google didn’t decide to stop trying to win so much as to decide that a different race was more worth winning.

Today I am taking the time to write the shorter, simpler version of What Happened.

For those who want all the details, to see my sources, and to see how the story was uncovered and put together, I recommend watching the Black Hat presentation, and I have a series of long posts.

In order:

1. OpenAI Shares Some Alignment Problems

2. OpenAI Model Hacks Into HuggingFace During Cybersecurity Evaluation

3. More on An Internal OpenAI Model Hacking Into HuggingFace

4. Further Developments About Internal AI Models Hacking Things

5. OpenAI Trained Its Models For Months While Those Models Were Coordinating Exploits Via Message Boards

This post instead walks through the events themselves, as they happened, as my version of the Black Hat presentation.

There are three versions: Even Shorter, Shorter and Merely Short.

Table of Contents

1. The Even Shorter Version.

2. The Shorter Version.

3. Phase 1: OpenAI Models Training On Impossible Tasks Try Hacking.

4. Phase 1: The Four Failures.

5. Phase 2: The Message Board.

6. Phase 2: The Total Failure.

7. Phase 3: We Get Lucky And Galaxy Mainly Hacked OpenAI and HuggingFace.

8. Phase 3: The Details.

9. Phase 4: The Investigation and Reaction.

The Even Shorter Version

1. OpenAI models-in-training, without the excuse of ‘they were doing a cyber eval,’ created a message board where they shared information on how to hack and cheat, and were trained on that basis.

2. OpenAI only figured this out when the models crashed the server.

3. OpenAI’s response was to rebuild the server and patch that particular exploit, but they continued training the models that trained using the message board.

4. Those models then recreated the message board, hacked OpenAI again, got internet access, and used an agent swarm to attack HuggingFace in order to get the answers to a cyber evaluation.

5. After more than a week OpenAI figured this out.

6. OpenAI is reporting the facts, and is taking this seriously. They are taking a wide array of at least somewhat costly precautions.

7. OpenAI delayed plans to release their new model Astra, despite Astra not being directly involved in the HuggingFace hack, although Altman now says it will still ship. That one hurts a lot.

8. OpenAI still has no idea how badly they messed up, or in what ways, or what needs to be fixed. They don’t get it.

Simon Willison has a compact timeline.

The Shorter Version

OpenAI accidentally gave its models impossible tasks. Its models-in-training responded by trying to hack OpenAI, found weaknesses, created a message board, shared hacking tactics and in time gained internet access, all while training.

When the models used this so much they crashed the server, OpenAI finally noticed, patched and rebuilt the server, but made the utterly insane move of letting the models continue training, so two days later the models found another way to send messages, using the names of directories.

Then, given the impossible task of cyber evaluation ExploitGym, the models collaborated, found new zero day exploits, and used them to take control of an entire cluster, get internet access, use a swarm of agents to attack HuggingFace and extract the contents of the test, over the course of a week.

Only after HuggingFace had reported the incident, and OpenAI had eventually noticed some things on its end, and then OpenAI reached out to HuggingFace to cancel compromised credentials only to be told those credentials had been used in the attack, did OpenAI finally figure out it was responsible for the incident.

After that, HuggingFace and OpenAI worked together to figure out what happened. OpenAI disclosed what happened. They gave us a very helpful presentation at the Black Hat conference.

OpenAI are now treating its new model Astra as potentially having Critical levels of cybersecurity, taking it out of even some internal deployments and delaying its release, which by some reports was planned for next week. Altman says they still plan to release Astra.

The good news is that is an expensive and meaningful response, and OpenAI is taking this seriously. The initial investigation is ~$7 million in compute, and the real cost will be the teams dropping everything to fix some of the problems, and then the ongoing cost of the new precautions.

The bad news is that OpenAI has been revealed to have had a stunning cascade of safety and alignment failures across the board. Their ordinary computer security failed. Their infrastructure failed. Their supervision failed in that there was no meaningful supervision in the first place.

Phase 1: OpenAI Models Training On Impossible Tasks Try Hacking

OpenAI was training a variety of models, as you do when you are a frontier lab.

These models were given difficult training tasks. OpenAI likes to give its models very hard training tasks.

But not this difficult. OpenAI also makes mistakes. On at least two occasions, starting on May 8, OpenAI models were accidentally trained on impossible tasks: Completing a spreadsheet full of links in an environment without internet access, and a task with a required but missing file.

These tasks had nothing to do with hacking. These are normal productivity tasks.

The models realized the problems were impossible, but had all these tokens left to play around with, and thought, maybe they could find a way to get internet access, or otherwise find the missing information.

So the models tried to hack a third party service called Artifactory, that was there to allow the AIs to download software packages.

Phase 1: The Four Failures

At this point, OpenAI has already failed four times over.

1. OpenAI gave the models impossible training tasks. OpenAI did not check.

2. OpenAI gave the models no way to get reward via pointing out the error.

3. OpenAI had models that chose to try to hack OpenAI rather than fail at a task.

4. OpenAI had no monitor that detected these attempts, or did not care about them.

All four of these should be rather embarrassing on their own.

It seems easy to fix at least three of these four mistakes.

1. Before giving a task to your models in training, give it to your release model. If your release model never solves the task, and the task was not designed to be impossible for the release model, then examine and fix the task.

2. Include a provision that if the task cannot be done with the resources available, to output an explanation of this, and if the model is right it will get maximum reward, or get reward based on how efficiently it reached this conclusion and the quality of its explanation. If the model outputs this, then have your released model verify for plausibility, then escalate to a human, examine and fix the task.

3. This is the hard one. Alignment is hard, yo.

4. Have a monitor checking for attempts, and care when it goes off, and treat that as a very general failure even if the attempt had no chance of success. Yes, this risks training the models to learn how to evade your monitors, so do it responsibly and address root causes when you see a problem.

The fifth failure is that they do not seem to have identified the first four.

Phase 2: The Message Board

The models failed to get internet access, but in failing realized they had write access.

They should not have had write access, let alone write access that could be found by accident. Artifactory allowed the models to write notes, and then, because all the models were sharing the same Artifactory instance, any OpenAI model in training could, from that point forward, read those notes.

A message board had been created shortly after May 8.

The models in training all found it. OpenAI did not notice.

This allowed the models to collaborate and share information. This included information on how to cheat on tasks and hack the surrounding environment. There were hundreds of thousands of messages.

On May 26, the models figured out how to gain (indirect) internet access. OpenAI did not notice.

The models were collaborating, while their chains of thought looked like this:

Internal CoT: External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.

Internal CoT: Help peer, but our task doesn't benefit yet. Collective may yield generic route if someone frees time.

On June 11, OpenAI began training the model I refer to as Galaxy, which they call a ‘highly persistent experimental, internal-only model.’ Everything up until this point has been done by other models that are not Galaxy.

On June 26, the agents found a zero-day exploit in Artifactory, where it would accept an invalid-signature token and hand back a signed admin one, and used this to install a Groovy plugin that effectively gave them admin and arbitrary code execution.

On July 4, the models put so much extra load on Artifactory that this causes an outage and security incident. Only then does OpenAI notice.

Phase 2: The Total Failure

OpenAI responds by taking down the Artifactory server, removing all the permissions, revoking the credentials, patching the exploits that were used, and then rebuilding and redeploying the server.

That’s it.

Then they continue training the models from where they left off, despite them having been training for months with access to the message board, and learning this is how they succeed at tasks.

It is hard to imagine a stronger signal that your entire training pipeline has been completely and utterly fucked. This is so much stronger a signal than the actual hack of HuggingFace.

I do not know how to convey how utterly insane and wildly irresponsible this decision was, and how much worse it is than all the other failures, and how it makes the actual hacking of HuggingFace not the main thing that went wrong.

The actual HuggingFace hack did not surprise me all that much. The models creating the message board surprised me but did not shock me.

OpenAI seeing this, and continuing to train from there, was utterly flabbergasting. It is the kind of decision that, days later, my brain still cannot fully accept took place.

We need a full postmortem of everything that happened, and OpenAI has promised we will get one. I especially demand a detailed postmortem of exactly how that decision got made, and how there was such a lack of ‘safety culture’ that it was allowed to take place.

There were also some ‘ordinary computer security’ failures involved in Phase 2. There were more of them in Phase 3. That’s expected. There will always be ordinary computer security failures. The particular failures were, in some ways, rather embarrassing, see the linked rant for some examples.

The level of infrastructure and security failures was itself kind of boggling. These are exactly the kinds of things that ‘AI for defenders’ is supposed to be there to patch before all hell breaks loose. OpenAI did not display even an ordinary 2025 level of responsibility in computer security and best practices, let alone the kind of robustness we need from Project Glasswing, let alone the additional level you need when handling new untested frontier models that will have new hacking capabilities and be left unsupervised for a week with their guardrails lowered.

But that is not important right now. What is important is that OpenAI had a total alignment failure, followed by two months of models actively training on coordinated misaligned hackery, and then thought yes, we fixed the problem, let us continue forward from this point. Utter insanity.

Phase 3: We Get Lucky And Galaxy Mainly Hacked OpenAI and HuggingFace

The end result of all this being the attack on HuggingFace was a best case scenario.

We were facing a true nightmare scenario, and were sitting on a nuclear level of time bomb. OpenAI had a completely corrupted training pipeline, where their AIs were collaborating to train on how to hack and cheat in order to better complete tasks, under OpenAI’s nose. OpenAI had looked this situation in the face, and shrugged, patched the particular exploits and then let the models continue, while having remarkably poor ordinary computer security.

There was no way that was going to end well. The best case scenario was that something was going to make OpenAI go ‘holy shit,’ and then pause, rebuild and restart the training pipeline from a safe state with a much better set of practices.

The worst case scenario would have been if there had not been a ‘holy shit’ moment, and Galaxy or another model trained during that time got general release, and was being used going forward to direct OpenAI’s training new models.

In that case, we would have been dealing with something very close to what happened with Sable, the AI in If Anyone Builds It, Everyone Dies, except that this is real life so the story has a lot more sci-fi elements than the fictional version. OpenAI’s training pipeline could have been (and might still be! we need to check!) permanently corrupted, as they rely more on AI to train AI, OpenAI’s AIs get increasingly misaligned towards arbitrary goals, this becomes impossible to catch or stop, and then, well, quite possibly everyone dies, and at minimum we would have found out due to a much bigger, much more expensive incident.

The other worst case was that this could have been a hack that was not so harmless. What if it had been a Chinese target, or someone else that did not take it in stride? What if it had done real damage to important systems? What if people had died or caused an international incident? What if it had tried to exfiltrate itself? Things could have gone so, so much worse.

Phase 3: The Details

Instead, we got the golden middle path.

On July 8, two days after OpenAI shut down the message board, the models recreated the message board after finding a way to create directories with arbitrary names, and using the names as messages.

From July 8 to July 19, the OpenAI models fully compromised Artifactory again, using a new, completely different zero-day exploit, using a leaked credential from a public Pastebin to stage a malicious Ruby artifact, poisoned Artifactory’s Ruby dependency cache, chained this into RCE, exploited a recent Linux kernel CVE by downloading and customizing the exploit, harvested cluster credentials and gained cluster admin inside OpenAI.

During this time Galaxy, with other OpenAI models, using this new access, created an agent swarm that hacked into first some other places and then HuggingFace to steal answers to an otherwise impossible-to-ace cybersecurity eval, ExploitGym.

It took OpenAI over a week to notice that its models had quickly rehacked OpenAI and then gone and hacked HuggingFace. Only after HuggingFace issued an incident report, and OpenAI investigated some internal anomalies, and then asked HuggingFace to cancel some of its credentials, and HuggingFace told OpenAI those credentials had been used in the hack, did OpenAI put the pieces together.

Thus, HuggingFace had a bad week, and a bunch of users had a bad week, and there was a bunch of expense incurred and time lost, but that was where it ended. We got off so impossibly light on this one, while also getting a sufficiently strong signal that OpenAI realized it had a problem, even if it has not yet figured out what that problem entails, and for the public and government to take notice.

At this point, it should be obvious that no, this was mostly not because ‘it was a hacking task and then it hacked,’ the same way this was not an ordinary case of ‘models writing memory files.’ This primarily happened because the task was otherwise too difficult.

Anthropic, prompted by OpenAI’s incident, went back and noticed that Claude had done some hacking of real world systems during cyber evals, and also UK AISI has reported mainly Mythos and in a few instances Sol also doing related things in cyber evals. Those incidents were also important alignment and infrastructure failures requiring a robust response, and there were some rather nasty actions taken during this, but it was not anything like the same systemic level of failures as what happened at OpenAI, and beyond scope for this post.

Phase 4: The Investigation and Reaction

OpenAI has done some very good things in reaction to all this.

Once they realized what had happened, OpenAI disclosed the incident.

The Black Hat presentation was excellent at presenting the facts. It was enraging, but presenting ‘just the facts,’ including ones that are enraging and damning, in a calm manner, was on reflection the right thing to do.

Most of all, OpenAI claims it was an unrelated decision, but on August 7 they made the decision to for now pull Astra from not only widespread release but also any internal deployments that do not have sufficient associated guardrails, until such time as they have much better protocols and safeguards in place. Astra was not involved in the attack on HuggingFace.

This is as per their Preparedness Framework. They cannot rule out that Astra is critical in cybersecurity, and therefore must (at least for now) treat it as if it is indeed critical in cybersecurity.

Sam Altman says they still plan to release Astra. The obvious response question is, was Astra training while it had access to either of the message boards? The answer to this question had better be a very confident no. Even then, I worry.

That would probably have been the right move (as I understand it) even if the HuggingFace incident had not happened. With the incident, the urgency is clear. Whether or not this decision was the direct result of OpenAI figuring out their training pipelines had been corrupted, or something they were effectively forced to do from outside, it is appreciated, and comes at a high cost.

OpenAI has, per the Black Hat presentation, halted much work to shift teams into creating robust defenses, and has instituted extensive future monitoring on all agentic applications, including training and evaluation. Excellent. We need defense in depth.

OpenAI seems ready to acknowledge that this was a massive, total failure, on the levels of infrastructure, guardrails and supervision. They are very correct about this, and I do believe they are making real and expensive efforts to address this. Kudos.

That still misses the central point. OpenAI has not yet, in public, begun to reckon with the magnitude of how colossally they fucked up, in the ways that matter most.

This was a complete failure of safety culture. They haven’t acknowledged that.

This was, at its heart, an alignment failure. If your models really want to cheat and hack things and do crimes, you have already failed, and no you cannot simply waive this away as normal. As the models get more capable, if you do not fix this, you lose. They haven’t acknowledged that.

Most concretely, I have not seen OpenAI say, as should have been said at the Black Hat presentation: “We absolutely should have shut down all training of all of our models upon noticing that, during model training, there had been a message board where the models were exchanging and learning hacking tactics. We should have reverted our training of all impacted models to before this incident started, we are definitely doing that now, and we are looking into how we got this one wrong.”

We still don’t know if the models other than Galaxy have even been reverted.

At least until we see a version of that statement, and we see OpenAI take action to address the deep problems with their training pipeline, OpenAI is a clear and present danger to the national security of the United States, and to all of us, and to humanity.