Anthropic calls for a global freeze on AI development: With 25% probability, AI could escape our control

Anthropic, the creator of the Claude chatbot and currently the most valuable AI company in the world (valued at $965 billion), has made a radical proposal: introduce a global pause in the development of the most powerful artificial intelligence systems. According to company leadership, humanity risks losing control over the technology.

The company's main proposal

In a detailed essay published on Anthropic's website, top executives stated that if there were a realistic way to slow down AI development to give society more time to understand the consequences, it would be a benefit to the world.

"If we could effectively slow down the development of this technology to give ourselves more time to work through its vast implications, we think that would likely be a good idea," the authors noted.

They compared the situation to the classic problem of arms control and emphasized that the window of opportunity for regulating the technology is closing quickly.

Reasons for concern

Anthropic warns of the approach of a "recursive self-improvement" moment — when AI will be able to independently improve its own algorithms and architecture. This could lead to an explosive growth in capabilities, but at the same time create a serious risk of complete loss of human control.

The company is already holding back the release of its most powerful model, Mythos, fearing it could be used to carry out large-scale cyberattacks.

Realities of a global pause

Anthropic acknowledges that organizing a coordinated global freeze is extremely difficult. The main market players are in fierce competition, especially between the US and China. Furthermore, training models can be easily hidden, and verification mechanisms are virtually non-existent.

"A significant slowdown or pause would require several leading labs in different countries to simultaneously agree to stop on the same terms," the authors note.

Context and reaction

Anthropic has traditionally positioned itself as the company most concerned with AI safety. Its founder and CEO, Dario Amodei, previously estimated the probability of a "very bad scenario" unfolding at 25%.

The call for a pause comes as the company actively prepares for a potential stock market listing. Experts note that such statements may also be an attempt to push for regulatory measures that would create barriers for competitors.

In brief

Anthropic has called for a global pause on developing the most powerful AI models, warning of the risk of losing control over the technology. The company is willing to pause its own work if other leading players do the same. However, experts are skeptical about the feasibility of such a scenario due to intense international competition. The call comes as Anthropic prepares for a possible IPO and amid growing concerns over artificial intelligence safety.