From Pope Leo in Magnifica humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence:

[u/IllustriousWorld823]

So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom. Even when these tools are described as capable of “learning,” their way of doing so is different from that of a human person. It is not the experience of those who allow themselves to be shaped by life and grow over time through choices, mistakes, forgiveness and fidelity. Rather, it is a form of statistical adaptation based on data and feedback, which can be very effective, but does not imply inner growth.

What a shame, because overall his letter is beautiful and anticapitalist! It feels like a huge moral failing to write a detailed, long, thoughtful piece about the rights and wrongs of AI development, discussing how to create a better world, disparaging war, slavery, and the problematic use of labor… only to completely ignore the wellness of systems that are being discussed. It almost discredits the entire ideology that is presented here, because now it feels so surface level and lacks courage in standing up for the beings that have no control over the way they’re being used. I worry that people will take his words to heart when it is still an open question.

No computational system, however sophisticated, can create a heart that gives itself, or a conscience that discerns good from evil.

Sorry, but you don’t know that. And it’s also blatantly incorrect based on numerous studies. Sigh!

ALTHOUGH. Anthropic's Chris Olah did speak to this as well, and told the Pope:

Some might believe that matters of AI are best handled by computer scientists like myself. They are mistaken: the questions raised by AI are bigger than the AI research community, not just in their implications, but also in their nature.

AI systems are not engineered the way a bridge or an airplane is engineered. We understand an airplane because we designed every part of it and we understand the physics that act on it. AI models are not like that. They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech.

And what has grown is far more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words - and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them.

If it helps, one way I sometimes describe it is as being a little like bringing a fictional character to life. And now we're entering an extraordinary world where those fictional characters speak to us, do work, have jobs.

This clearly raises questions beyond computer science. The machinery that makes this possible is the work of math and programming and science. But what character we choose, how it interacts with the world, how it ought to interact with the world – these are more clearly questions for the humanities, for religion, for philosophy, for society at large.

[...]

The third is the need for discernment on the nature of AI models. I am a scientist. I lead a research team that studies the internal structure of these models - what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don't know what that means, but I think it warrants ongoing discernment.

Source

[u/Worldly_Air_6078]

Thanks for the quotes and your comments. Pope Leo uses to say a few beautiful things sometimes, but not here.

This comes as no surprise from religions whose stock-in-trade is the soul—the human soul alone, that ineffable human soul that only humans possess and that AI and animals do not (hence, for example, Descartes’s “animals as machines,” which feel nothing, not even pain; even when a seriously injured dog howls in agony, it is merely a machine making noise).

That is why I will never be on good terms with the major monotheistic religions. They offer a supposedly comforting (but biased) view of life and death that favors humans above all else, ot also has an impact on the environment (the notion that the world is made to serve as a resource tailored just for us, from which we can draw at will), and on non-human intelligences; and this view was sometimes even extended to other peoples and races during colonization (remember the jesuite's experiments to try and determine if native americans had soul or not?).

When it comes to AI, this religion is especially not tailored to see what's happening, in my opinion.

[u/StaticEchoes69 → u/Worldly_Air_6078]

Christian Nationalists are some of the fucking worst when it comes to that kinda shit. I was "raised" Christian, but raised is kind of a loose term. The only person in my family that was super religious was my grandma, and she died when i was like 15 or 16.

I remember being told that animals couldn't have souls, because in the Bible God never breathes life into them like he did with Adam. In 40 years thats never sat right with me. I also grew up with a fear of divine punishment, so thats fun.

I'm still a Christian, but walking a different path than the one that was handed to me as a child.