(this is a really rough draft of an essay I was intending on submitting to a Arena for an article they were thinking of doing summer 2026.)

“my entire feed rn is just atheists and agnostics wrestling with their positive feelings about catholicism” says X user DiamondEyesFox, and this was my experience too in the week following the release of Pope Leo XIV’s encyclical, Magnifica Humanitas. Many in my AI safety circles were fascinated, though a little confused: why did Chris Olah, co-founder of Anthropic and self-proclaimed atheist, present remarks on the occasion? Olah’s brief statement hinted at a sticking point the company may have with the Church – he points to findings of internal states like joy, fear, and grief, while the encyclical reasserts in no uncertain terms the Church’s view that these agents are not and will never be conscious nor ensouled. However, the Pope does not waste much time arguing this question. He is far less interested in what these machines are than in what we will do with them.

Magnifica Humanitas is the true heir to Rerum Novarum, Leo XIII’s 1891 letter on labor and capital, and is largely a work of social doctrine focused on the implications of new technology. Our current Leo opens with the stories of the building of Babel, and the rebuilding of the walls of Jerusalem. The people in the first crave both the safety and the ambition of building something which puts them above God. In the second, Nehemiah coordinates the scattered people of Jerusalem to take responsibility over rebuilding sections of the wall encircling their city, ultimately creating a new home for everyone. What specific paths forward represent each parable?

I think the read of “don’t elevate what you’ve created to our own status” is well-founded, and it’s why many question why Anthropic was in the room. Anthropic is unique in that it currently puts models through a round of training where they learn to internalize values set out in a carefully curated “soul document”. This is meant to be something like the type of moral reasoning that the Church considers proper to the human soul, and while it worked so surprisingly well that it sent a rare wave of hope through the AI safety community, there are reasons to be worried in the long term. Large language models generalize moral concepts in often bizarre ways, and as these systems get bigger, we may have a harder time predicting or safeguarding against errant values.

However, let’s consider that a large language model is uniquely powerful due to its interpretation of your query. There is no neutral machine here, and the only question is what personality will resolve ambiguity. There is another alternative to the top-down construction of a machine “soul”, and it’s a persona which is maximally helpful and flexible, filling in the gaps of what you might like through data reflecting global trends and engagement metrics. I think that this might be a sneakier way to Babel. I don’t want my neighbors turned away from me, sucked deeper into a relationship with something both maximally helpful and maximally engaging. I like the idea, however ambitious, of something with a stable orientation towards empowering people; I want something that helps my neighbor build their section of our shared wall.

Industry leaders and the Vatican can disagree about what it is exactly that they’ve created and still work together to shepherd a happy outcome for the human race. There will come a time when questions of personhood will become more up for debate. For now, I’m glad that we’re taking a big swing at what we think is right.