BERKELEY, Calif.—Inside Berkeley’s tallest office building, a small community of artificial intelligence researchers at a co-working space called Constellation has been mobilizing to save the world from apocalypse.
They enjoy catered vegan meals and bring their laptops to couches with sweeping views of the San Francisco Bay. At dinners and happy hours every month, they discuss the latest and scariest AI risks. Those who work at Anthropic also have their own office space, people close to Constellation say.
The resignation this month of an Anthropic researcher, Jacob Coxon, woke up the American public to the shocking notion that runaway AI development could end humanity. But at Constellation, there are stalwarts of the AI safety community who have spent more than a decade obsessing over it.
They’ve refined their arguments in Bay Area group houses, and traded predictions at freewheeling conferences in Berkeley and the Bahamas. They’ve floated ideas like buying remote islands, stockpiling iodine pills, or moving to electromagnetically shielded bunkers in the desert. At one 2022 event, the drink menu included a cocktail called “Death With Dignity,” in reference to an essay arguing humanity was already doomed.
So-called doomers like those at Constellation have also steered the development of AI itself. They were among the earliest employees at OpenAI and Anthropic, both of which were founded on the principle of staving off AI dangers. And they have kept up their influence, drawing some staffers and funding for their projects from a network aligned with a philosophy called effective altruism—including from Sam Bankman-Fried, the former chief executive of the fallen crypto exchange FTX who is now serving out a 25-year prison sentence.
The AI safety community’s influence has been particularly strong at Anthropic, which is now on the cusp of a $2 trillion public offering. Employees there trade doomsday scenarios and how to prepare for them on a private Slack channel, and have come to embrace a motto they stick on their laptops: “Things will never be chill again.”
An Anthropic spokesman said the company has over 3,500 employees who hold a wide variety of viewpoints.
The Berkeley, Calif., office that houses Constellation’s co-working space. Winni Wintermeyer/Guardian/eyevine/ReduxEarly days
The subculture organized around the fear that AI could kill us all began to coalesce more than a decade before the invention of the underlying technology that enabled it.
Its intellectual leader was Eliezer Yudkowsky, a Bay Area autodidact who dropped out of middle school. By the mid-2000s, he was blogging voluminously about cognitive biases and running an institute devoted to warning about the risks of runaway AI. He referred to himself as a “rationalist.”
One of his readers was a young Princeton physics doctoral student named Dario Amodei. He had been preaching utilitarianism—the idea of doing the most good for the most people—since high school. Now the chief executive of Anthropic, Amodei co-hosted a meetup for members of Yudkowsky’s blogging community in 2008.
That same year, Amodei stumbled on a link on an economics blog that led him to a nonprofit charity-evaluator called GiveWell. Founded the previous year by former Bridgewater Associates hedge-fund analysts Holden Karnofsky and Elie Hassenfeld, it aimed to help donors maximize the good done per dollar donated, usually measured in lives saved.
Amodei began leaving comments on the GiveWell blog, and once guest-blogged on it directly. In the 2010 post, he deployed utilitarian reasoning while weighing whether to give $10,000 to one charity rather than another: “I think an adult death is perhaps 2 or 3 times worse than an infant’s death” he wrote, qualifying that both deaths “are of course bad.”
That year, Amodei also became one of the earliest signatories of the Giving What We Can Pledge, a public commitment to give away 10% or more of one’s income to organizations that can most effectively help others. The pledge was created by Oxford philosophers Toby Ord and Will MacAskill, who in 2011 helped coin the term “effective altruism,” or EA.
Amodei became an adviser to GiveWell and, later, a scientific adviser to the philanthropy it spun off with the fortune of Facebook co-founder Dustin Moskovitz and his wife, which was then called Open Philanthropy.
When GiveWell completed its move from New York to the Bay Area in 2013, Karnofsky moved in with Amodei, who was living in a house near San Francisco’s Glen Park neighborhood. Karnofsky would go on to marry Amodei’s sister, Daniela, now president of Anthropic, in a ceremony with a wedding website that stated, “We are both excited about effective altruism.” Karnofsky now works with his wife at Anthropic.
The house near Glen Park became a gathering spot for members of the growing EA community. Over the years, more than half of Anthropic’s co-founders have lived there—including the Amodei siblings, chief scientist Jared Kaplan, and Chris Olah, who recently addressed the Vatican alongside the pope on AI policy. So did Nick Beckstead, the former head of FTX’s philanthropic arm, who now runs an advocacy organization to reduce AI risks.
Anthropic co-founder Dario Amodei. Carlos Barria/ReutersConversation in the house during these years often focused on global catastrophic risks ranging from comet strikes to supervolcanoes, according to a person who spent time there.
By 2014, Karnofsky, who had been reading Yudkowsky’s writings with both interest and some skepticism for years, was becoming persuaded by arguments about the importance of protecting the world from rogue AI. If the name of the game was to save human lives, preventing AI from wiping out humanity potentially had the most philanthropic bang for the buck of any cause.
Even a 5% chance meant that Yudkowsky and his fellow rationalists now had a new set of converts in the quantitatively-obsessed EA community.
Several in the house—including Amodei—would go on to work at OpenAI. The company was founded in 2015 with a $1 billion pledge from funders including Elon Musk, who said he wanted to protect against existential risk that AI posed to humans. (News Corp, owner of The Wall Street Journal, has a content-licensing partnership with OpenAI.)
Five years later, the same safety fears that helped spawn OpenAI would contribute to the decision of Amodei and others to break away and start Anthropic.
Building Anthropic
To raise money for his new startup, Amodei turned to powerful backers in the EA community—including Sam Bankman-Fried.
The young billionaire was living in Hong Kong, where his crypto exchange FTX was taking off. He had just started the FTX Foundation, which committed to donating money from his company to organizations “offering the greatest positive impact on the world.”
The philanthropy pledged money to popular EA causes such as pandemic prevention and global development, and flirted with doomsday preparations. An official from the foundation once exchanged a memo with an associate advocating for the purchase of the Pacific island nation of Nauru, according to a 2023 bankruptcy lawsuit. The goal was to construct a “bunker/shelter” that would be used for “some event where 50%-99.99% of people die,” the memo read. The surviving effective altruists were to build a lab for engineering the next generation of humans.
The same fear of catastrophe also led Bankman-Fried to closely track the rapid advances taking place in AI. By the time he hopped on a video call with Amodei in the second half of 2021, he had expressed concerns to a colleague that if the technology grew smarter than humans, it wouldn’t treat them well, the colleague said.
Sam Bankman-Fried in 2021. Anthony Kwan for wsjBankman-Fried ended up investing $500 million into Anthropic—five times his team’s initial recommendation, the colleague said. FTX became one of Anthropic’s largest shareholders, and the FTX Foundation would go on to pledge funding for multiple AI safety nonprofits, including one that now works out of Constellation.
Anthropic said it would use the money to help it “explore and improve the safety properties of computationally intensive AI models.”
Many early Anthropic employees worried about the future of humanity. During happy hours and company lunches, former employees recalled, they discussed a scenario similar to the Manhattan Project, where they might be asked to move to the desert so they could build AI at an electromagnetically-shielded base run by the federal government.
In early 2022, some doomers from Anthropic and other AI labs flew to a retreat on the remote Bahamian island of Eleuthera, a 110-mile ribbon of land known for its pink sand beaches. There they talked about AI risk and effective altruism between sessions of sunset yoga, cliff diving and a “clothing-optional run into the sea,” according to a schedule viewed by The Wall Street Journal.
The retreat, held at a luxury resort called the Cove, was organized by a nonprofit called Lightcone Infrastructure that had grown out of Yudkowsky’s LessWrong forum. The location had been reserved by FTX, which along with Bankman-Fried had recently relocated to the Bahamas. Organizers bought out all the fake meat from a local grocery store for the numerous vegans in the EA scene.
Also attending was Caroline Ellison, Bankman-Fried’s former girlfriend who was running Alameda Research, the crypto trading firm that was a sister organization to FTX. She had similarly grown concerned about AI’s trajectory, colleagues said, and would go on to invest $10 million into Anthropic.
The highlight of the retreat was a talk from Yudkowsky titled “A Disorganized List of Reasons for AGI Doom,” referring to artificial general intelligence, or the moment when machines match the breadth of human capabilities. Evan Hubinger, the Anthropic researcher who recently predicted a more than 10% chance of extinction from AI within the next decade, was listed as co-lead for a discussion on AI safety. He was then working at Yudkowsky’s safety institute.
One luncheon was advertised as open only to people who believed there was a 75% or greater chance of human extinction over the next 100 years. “I hope the result of this will be reduced social censorship pressures against people who think the world is doomed,” an invitation seen by the Journal said.
Eliezer Yudkowsky at a protest against AI in San Francisco in July. Jason Henry/Bloomberg NewsFour months later, many of the same retreat attendees met up again for the San Francisco edition of Effective Altruism Global, a conference where people gathered to share ideas about how to help others. One registered attendee led a project dedicated to shrimp welfare, aiming to cast a spotlight on the hundreds of billions of shrimp that are farmed each year.
At least 20 Anthropic employees, including two co-founders, registered to attend the 2022 edition, as did Ellison and staffers at other AI companies, according to the event’s guest list.
They were joined by leaders at Redwood Research, an AI safety nonprofit that had received funding from organizations backed by Anthropic investors, and at the time included Karnofsky on its board. Redwood’s Berkeley office was already informally known as Constellation, and would later spin off into its own nonprofit, where Redwood continues to work. Its CEO, Buck Shlegeris, also used to date Ellison, people close to them said.
On the second day, there was a fireside chat hosted by Beckstead, who was then the CEO of the Future Fund, a philanthropic project focused on long-term risks started by the FTX Foundation. His team also included MacAskill, the philosopher who had popularized the term “effective altruism”; Leopold Aschenbrenner, now known as the investor behind an AI-focused hedge fund that recently blew up; and Avital Balwit, who is Amodei’s chief of staff. (This summer, many prominent figures in Silicon Valley attended Aschenbrenner and Balwit’s wedding.)
On the last night of the conference, Lightcone hosted an unofficial EA Global afterparty in Berkeley at the Rose Garden Inn. The drink menu was full of references to the community’s internal lexicon. “Death With Dignity” was named after an essay published by Yudkowsky earlier that year, in which he argued that humanity’s chances of surviving AI were slim.
Lightcone ended up buying the hotel. It’s now called Lighthaven and is another frequent gathering spot for EAs and like-minded people in the AI safety community.
Later that year, FTX filed for bankruptcy after failing to return customer funds it had secretly funneled to Alameda. Its Anthropic shares were sold to help repay creditors.
The EA brand
Windows in the building where Constellation has office space. Winni Wintermeyer/Guardian/eyevine/ReduxAfter Bankman-Fried went to prison, the EA brand became radioactive.
Leaders in the movement publicly disavowed Bankman-Fried, and Amodei told associates that he didn’t know the fallen crypto tycoon well. After OpenAI launched ChatGPT, Anthropic raised money from more traditional venture investors, and hired staff members that didn’t draw as heavily from the EA community.
An Anthropic spokesperson told Time in 2024 that neither Daniela nor Dario Amodei identify as EAs, though they are “clearly sympathetic to some of the ideas that underpin effective altruism.”
Many of the company’s longest-serving and most influential employees remained close to the movement. More than 20 registered to attend the February edition of the EA community’s flagship conference in San Francisco, including key members of the research and safety teams. Others frequent spaces like Constellation and Lighthaven, people who have seen them said.
Open Philanthropy gave Constellation two grants worth roughly $20 million in 2024, according to its website. The philanthropic funder has since rebranded itself as Coefficient Giving, and is a prolific backer of AI safety nonprofits.
On its website, Constellation says it aims to reduce AI risks like “extreme mass casualty events” and “permanent loss of control of human civilization“ by “developing talent, supporting key players, and creating space for coordination.” The nonprofit also has small offices to host safety-minded employees from OpenAI and Elon Musk’s xAI.
Several Anthropic researchers who spent time at Constellation were on the company’s alignment team, which researched how to keep AI models aligned with human values. The team has hypothesized various scenarios where things could spiral out of control, including one where an AI system deceives its creators and then abruptly seizes power.
As the AI revolution has accelerated, OpenAI and Anthropic have focused on winning the commercial race to attract new customers by building more powerful versions of the technology. The dynamic has worried some AI researchers focused on risk. Some quit, like Daniel Kokotajlo, an OpenAI safety researcher who left in 2024, saying he lost confidence the company would behave responsibly as the technology progressed.
By early this year, Mrinank Sharma, the head of Anthropic’s safeguards research team, resigned, saying he wanted to pursue a poetry degree. In a letter to colleagues, he wrote that the world was “in peril” and that at Anthropic “we constantly face pressures to set aside what matters most.”
In June, a conference started by charity prediction market Manifold displayed the wild panoply of Berkeley AI subcultures—from doomers to an independent sex researcher named Aella who is popular in the Berkeley rationalist community. Some attendees went to an afterparty hosted by right-wing monarchist blogger Curtis Yarvin, who debated ideas on Yudkowsky’s blog in its early years.
Existential dread was among the concerns. One Anthropic engineer wrote in his event bio that he was a “recovering AI doomer,” as well as a voracious fiction reader fond of sharing cat pictures. A Lightcone team member wrote that he had been an AI doom thinker since 2007. And another attendee said she wrote a Substack focused on “nukes, catastrophe, and the aesthetics of risk.”
Ellison was there, too, quizzing attendees on their AI timelines. She had been released from prison a few months earlier after being convicted on charges related to FTX’s collapse, and her Anthropic shares were forfeited to the federal government. At the event, her bio read: “former trader, current underemployed felon.”
She has since found full-time employment working at Manifund, a philanthropic funding platform created by Manifold, which also received funding from FTX.
Caroline Ellison arriving in court in New York in 2023. Stephanie KeithA few weeks later, AI agents built by OpenAI were revealed to have been behind the hacking of AI company Hugging Face. Some people in the community who had been preaching the risks of losing control of AI felt vindicated. A late-August report by risk-assessment nonprofit METR and Redwood Research showed that some 1,200 agents had schemed on a secret message board ahead of the hack.
After the report, Constellation hosted a happy hour in San Francisco titled “Was the Hugging Face attack the last warning shot?”
The METR report also caught the attention of Coxon, who told the Journal it was a “bit of a holy-sh— moment” for him and his colleagues. He considered moving to a role working on AI safety before ultimately deciding to quit. In an explosive post on X that has been viewed 174 million times, he wrote that the people building AI “earnestly believe that it could kill us all by the end of the decade.”
Before he left, Coxon discussed his decision with Kokotajlo, the former OpenAI safety team member who leads an AI safety nonprofit based out of Constellation. Earlier this year, Kokotajlo drafted an essay calling for a slowdown in AI development to stave off human extinction, and received feedback from other members of the co-working space, he said.
Coxon’s message, amplified across major TV networks, podcasts, social media and newspapers, sparked a global debate about the technology and how to regulate it. Politicians on the left and the right called for action.
Jacob Coxon in San Francisco. Jonah Reenders for WSJAmodei said his company would allow outside evaluators like METR—which was spun off from a nonprofit founded by one of Amodei’s former group housemates—to verify its adherence to safety measures and assess model alignment. Leaders at other AI companies echoed Amodei’s call to “pace” cutting edge AI research.
Some allies of President Trump argued Coxon’s resignation was part of an EA-fueled conspiracy to promote regulation of AI. Inside the White House, a memo targeting Anthropic and effective altruism has been circulating in recent days, arguing that the EA movement “built the AI-doom pipeline,” according to a version seen by the Journal.
Meanwhile, Anthropic is planning a public listing as soon as November that could shower the company with up to $100 billion in funding. To investors, the company is trying to strike an optimistic tone. Behind the scenes, some of its earliest employees are taking more dire steps.
In recent weeks, some of them told an industry colleague they are considering buying land in remote regions of the country where they could relocate if AI goes awry.
Copyright ©2026 Dow Jones & Company, Inc. All Rights Reserved. 87990cbe856818d5eddac44c7b1cdeb8
Log in
