Anthropic and Claude warn of the “recursive self-improvement” of artificial intelligence and its serious implications for humanity

(ZENIT News / Rome, 06.14.2026).- For years, warnings about the risks of artificial intelligence came primarily from philosophers, ethicists, and religious leaders. Now, some of the strongest cautions are emerging from within the industry itself.

Just weeks after Pope Leo XIV published Magnifica Humanitas, his first major document on artificial intelligence, one of the world’s leading AI companies has publicly raised concerns that the technology may be advancing faster than society’s ability to govern it. The convergence is striking: the Vatican and Silicon Valley, often perceived as distant worlds, are increasingly expressing similar anxieties about the future relationship between humanity and intelligent machines.

Anthropic, the company behind the AI model Claude, issued a detailed warning on June 4 through its co-founder Jack Clark and Anthropic Institute director Marina Favaro. Their concern centers on a possibility known as “recursive self-improvement,” a scenario in which artificial intelligence systems become capable of designing increasingly powerful successors with little or no human intervention.

According to the company, current AI systems are already assuming a growing share of the coding work used to develop future models. Anthropic noted that the amount of work its systems can perform autonomously is increasing at an extraordinary pace. While the company stressed that fully autonomous self-improvement has not yet been achieved and may never occur, its researchers acknowledged that many institutions appear unprepared for the possibility.

The implications are profound. If machines eventually acquire the capacity to develop new generations of artificial intelligence on their own, traditional methods of oversight could become increasingly difficult to maintain. The challenge would not merely be technical but civilizational: ensuring that human judgment remains capable of guiding technologies that may surpass human abilities in specific domains.

Rather than calling for a permanent halt, Anthropic proposed that governments and technology companies explore mechanisms to slow or temporarily pause the development of the most advanced systems if necessary. Yet the company also recognized the central obstacle to such a proposal. Any meaningful slowdown would require unprecedented international cooperation. Otherwise, nations or corporations that continue advancing could simply gain a strategic advantage over those exercising restraint.

The debate echoes themes recently raised by Pope Leo XIV. In Magnifica Humanitas, the pontiff warned against creating what he described as a new “Tower of Babel,” a technological culture capable of immense power but detached from a shared moral vision. Leo has repeatedly argued that the central question is not whether innovation should continue, but whether humanity can ensure that technological progress remains ordered toward authentic human flourishing.

That concern has found unexpected support among some scholars working directly with the technology sector. Charles Camosy, a moral theologian who has collaborated with Anthropic on ethical questions, argues that the Pope’s call to “disarm” artificial intelligence should not be understood as opposition to innovation. Rather, it is an appeal to prevent technology from dominating the human person or replacing essential human relationships.

The issue extends beyond technical safety. Critics increasingly worry about the growing tendency to delegate uniquely human responsibilities—including education, caregiving, mentorship, and even moral formation—to artificial intelligence systems. The risk, according to many ethicists, is not that machines suddenly rebel against humanity, but that human beings gradually surrender activities that cultivate wisdom, responsibility, and genuine interpersonal bonds.

At the same time, new academic research suggests another challenge that has received far less attention: AI’s difficulty in engaging religious perspectives. The Consortium for Faith and Ethics in AI, bringing together researchers from Baylor University, Brigham Young University, the University of Notre Dame, and Yeshiva University, recently found that most major language models largely omit religious viewpoints when addressing ethical questions.

The consortium’s All Faith Benchmark examined responses from fourteen leading AI systems developed by companies including Anthropic, OpenAI, Google, Meta, and xAI. Researchers concluded that while religion continues to shape the moral outlook of large portions of the world’s population, AI systems frequently fail to reflect that reality. The study also identified measurable differences in how various religious traditions are represented, raising questions about fairness, bias, and cultural understanding in machine-generated responses.

The findings reinforce a broader argument increasingly heard in both academic and religious circles: artificial intelligence cannot be treated merely as an engineering challenge. It also raises questions about anthropology, ethics, culture, and the meaning of human dignity.

Perhaps the most significant development is that this conversation is no longer confined to churches, universities, or think tanks. Major technology companies themselves are beginning to acknowledge that the future of AI may require not only more powerful machines, but also deeper reflection on what it means to be human.

That is precisely the question Pope Leo XIV has placed at the center of the debate. The ultimate danger, he suggests, is not that machines become too human, but that human beings forget the unique gifts that distinguish them: moral responsibility, wisdom, wonder, and the capacity to recognize truth, beauty, and the dignity of every person.

Thank you for reading our content. If you would like to receive ZENIT’s daily e-mail news, you can subscribe for free through this link.