Two weeks after Pope Leo XIV published an encyclical warning artificial intelligence companies against constructing “a new Tower of Babel”, the multibillion-dollar AI company Anthropic is calling for a global pause or slowdown in development. Source: EWTN News.
Anthropic co-founder Jack Clark and Anthropic Institute head Marina Favaro published a blog on June 4 warning about a risk of “humans losing control over AI systems” as its own system Claude is reaching the potential to autonomously design its own successor without any human contributions.
“This is called recursive self-improvement,” they wrote. “We are not there yet, and recursive self-improvement is not inevitable. But it could come sooner than most institutions are prepared for.”
The blog post did not mention the encyclical, but a separate Anthropic co-founder, Chris Olah, met with Leo and sat alongside the pope when the encyclical Magnifica Humanitas was revealed on May 25.
Anthropic has engaged in outreach to the Vatican and other religious leaders to help address ethical questions related to AI development.
In the blog post, Anthropic leaders explained that its AI system is taking over a large portion of writing code that designs AI – with its workload growing eightfold every quarter. AI will “become much more capable in coming years,” they wrote, and “these trends have huge implications.”
“If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behaviour all grow much more important,” they wrote.
Although Mr Clark and Ms Favaro acknowledged AI has not reached this level yet and they cannot say for certain it will, they wrote: “We do not have good intuitions for what this world would look like” if this occurs, and AI capabilities “eclipse those of humans.”
Anthropic’s leaders wrote that AI companies should come together to either pause or slow down development “to give ourselves more time to deal with its immense implications.”
However, this would require global international cooperation among countries and AI companies because “if a slowdown simply lets the least cautious actors catch up technologically, it could leave everyone less safe,” they wrote.
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” they added.
FULL STORY
Anthropic urges ‘pause’ or ‘slowdown’ of AI development after Leo’s encyclical (By Tyler Arnold, EWTN News)
