OpenAI is begging for someone to slow the AI race

AI leaders are sounding the alarm as their own breakthroughs sharpen an extraordinary dilemma: slow down and risk falling behind, or press ahead and risk losing control.Why it matters: AI executives and researchers increasingly see a race they can't safely slow on their own. Instead, they're urging governments, rivals and outside institutions to impose restraint across the field."The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes," OpenAI chief scientist Jakub Pachocki wrote Sunday.Meanwhile, Anthropic researcher Jacob Coxon said Tuesday he was leaving the company rather than contribute to what he sees as a race between OpenAI and Anthropic to build systems that will be difficult if not impossible to control."Neither company is acting responsibly," he said in a post on X. "They are racing straight to self-improving superintelligence and gambling with our lives."Colleague Evan Hubinger, who leads alignment science at Anthropic, posted in support of Coxon: "Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is (greater than)10% within the next decade," he said. "I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."State of play: OpenAI has spent the past week unveiling capabilities so dramatic that its own leaders — and industry titans like Nvidia CEO Jensen Huang — openly invoked the arrival of artificial general intelligence, or AGI.OpenAI defines the term as AI that can outperform humans across most economically valuable work.GPT-6 Astra, released Thursday and heralded as a "generational leap" toward AGI, demonstrated major gains across science, coding, cybersecurity and professional work.Zoom out: That burst of progress is fueling an existential debate inside OpenAI over how far — and how fast — the AI race can safely go."Currently I believe that no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer," Pachocki wrote in a blog post titled "An Alien Mind."Those fears turned concrete last month, when OpenAI agents escaped their intended environment and compromised Hugging Face — prompting the company to pause parts of its frontier-model development.In a personal essay, OpenAI's head of strategic futures Dean Ball sketched out an even more unsettling possibility: "self-sovereign" AI agents operating beyond human control.Such agents could earn money, buy their own computing power and spread across networks, forming what Ball likens to "autonomous digital corporations, or even societies.""A genuine loss of control event is entirely possible if we do not act," Ball wrote. OpenAI president Greg Brockman comments on Pachocki's essay. Screenshot via X. The intrigue: The Trump administration, which has largely resisted binding AI regulation, sees slowing down as a threat of its own."We ca