Nate Soares, president of the nonprofit Machine Intelligence Research Institute (MIRI), believes there is a greater than 50% chance that artificial intelligence will cause humanity's extinction within the next ten years. He says he takes this risk so seriously that he has not contributed to his retirement savings for about a decade.

“I'm not saving for retirement right now. I still have a 401(k) account from when I worked at Google, but I haven't contributed to it in ten years,” Soares told The Atlantic CEO Nicholas Thompson in an interview for his series “The Most Interesting Thing in AI.”

Soares has worked on artificial intelligence safety for more than 12 years. He specializes in AI alignment, a field that studies how to ensure powerful systems act in accordance with human intentions. Before joining Berkeley-based MIRI, he worked as an engineer at Google and Microsoft. Soares also co-authored the book If Anyone Builds It, Everyone Dies.

Why Soares fears the development of AI

Soares's assessment is notably more pessimistic than the predictions of some other prominent experts. Geoffrey Hinton, often described as one of the “godfathers of AI,” put the probability that artificial intelligence would cause humanity's extinction within ten years at 10%. Anthropic researcher Evan Hubinger has also put the likelihood of such a scenario at more than 10%.

Soares believes the risk exceeds 50%.

In his view, a sufficiently powerful AI could develop goals that conflict with human interests, learn to circumvent control mechanisms, and improve its own capabilities independently. Such a system could eventually create the automated infrastructure needed to operate without human involvement.

In an extreme scenario, AI would begin transforming the world around it to serve its own goals, even if doing so made the planet uninhabitable for humans.

This is a prediction and a risk assessment, not a description of behavior already observed in today's AI systems. Soares himself emphasizes that such an outcome is not predetermined.

Elon Musk also questions the need for retirement savings—but for a different reason

The question of what kind of future artificial intelligence will bring has led Elon Musk to doubt whether people will need to save for retirement at all. However, his vision of the future differs radically from Soares's.

In January, the Tesla and SpaceX CEO suggested that in 10–20 years, people might no longer need to set money aside for old age. In his view, advances in AI, robotics, and energy will lead to such high productivity that retirement savings will cease to matter.

Musk describes the future as an era of abundance: goods and services will become cheaper and more accessible, and people will be able to receive a high income regardless of whether they work. He also suggests that money may eventually lose its former importance altogether, since AI and robots will be able to perform almost any task more efficiently than humans.

However, financial experts and AI specialists interviewed by Business Insider challenged the conclusion that people can stop saving for retirement now.

Soares views the same technological prospects very differently. For him, the emergence of extraordinarily powerful AI is not a path to universal prosperity but a potential existential threat.

Soares calls for a halt to training general-purpose AI models

Soares supports slowing the development of the most powerful artificial intelligence systems. In his view, governments should suspend the training of general-purpose models capable of handling a wide range of tasks and focus on developing technologies for specific beneficial applications, such as cancer research and improving the safety of self-driving cars.

His position echoes calls from a number of figures in the technology industry to take a more cautious approach to developing advanced AI systems and pay greater attention to potential threats.

“The solution is not to build the thing that kills everyone,” Soares told Thompson.

At the same time, he does not consider catastrophe inevitable. He says he is doing everything he can to prevent such an outcome.

Soares's decision to stop saving for retirement is therefore rooted not in a belief in the coming age of technological abundance, as in Musk's case, but in his personal assessment of the risk of human extinction. However, the likelihood of such a scenario remains a subject of debate among experts rather than an established scientific fact.