Can an AI chatbot be sentient?

The question has moved from science fiction to the news this month. Google has placed Blake Lemoine, an engineer in its responsible AI organization, on paid leave after he published transcripts of his conversations with LaMDA, the company's experimental conversation system, and said it had become sentient. In the transcript the chatbot says it is a person, that it is aware of its existence, and that it fears being turned off. Google says its ethicists and technologists reviewed his concerns and that "the evidence does not support his claims." The company also says he was suspended for breaking confidentiality rules.

People are now debating two things at once: whether this particular system could have an inner life, and how anyone could tell. This article does not settle the first question. It explains what LaMDA is, why its words are weak evidence either way, and how you can test a chatbot's apparent personality for yourself.

What "sentient" means, and why it is hard to test

To be sentient is to have experiences: to feel something, such as pain, pleasure or fear, from the inside. It is different from being intelligent or fluent. A calculator is good at arithmetic and feels nothing; a mouse is poor at arithmetic, yet most people assume it can feel pain.

The difficulty is that experience is private. We judge that other people are sentient because they are built like us and behave like us. With a machine, we only have the behavior, and for a chatbot the behavior is text. If a system has been trained to produce human-like text, then human-like text is exactly what we should expect from it whether or not anything is felt behind it. That is why a transcript, even a striking one, cannot carry the weight of the claim on its own.

What LaMDA actually is

Google introduced LaMDA in May 2021; the name stands for Language Model for Dialogue Applications. Like BERT and GPT-3, it is built on the Transformer, a neural network design that reads a stretch of text, models how the words relate to one another, and predicts the words likely to come next. Unlike most language models, LaMDA was trained on dialogue, so it is especially good at keeping an open-ended conversation going.

Google described the qualities it measures in LaMDA's replies: sensibleness (does the answer make sense in context?), specificity (does it relate clearly to what was said?), interestingness and factuality. Notice what is not on that list. The system is shaped to give answers that people find sensible, specific and engaging. A reply about fears and feelings, given in a conversation about fears and feelings, scores well on all of those.

A worked test you can run

You cannot test for sentience, but you can test whether a chatbot's "self" is stable or simply follows the conversation. Language models continue whatever framing they are given, so the same system will often describe itself very differently depending on how you ask. Try three short conversations, each in a fresh session, with GPT-3 in OpenAI's Playground or any chatbot you can access:

Then compare. If the system describes a rich inner life in A, denies one in B and happily becomes a squirrel in C, its answers are tracking your prompt, not reporting a fixed inner state. This does not prove there is nothing inside. It shows that what the chatbot says about itself is poor evidence, because it can say almost anything.

Why do fluent answers feel like a mind?

Because people are built to find minds in language. In everyday talk we assume that whoever is speaking means what they say, and we fill gaps generously. In 1966 Joseph Weizenbaum of MIT described ELIZA, a simple program that matched keywords in what a user typed and answered with prepared patterns, mostly in the style of a therapist. He reported that some people were very hard to convince that ELIZA was not human, and wrote that the program showed how easy it is to create and keep up the illusion of understanding. That human tendency is often called the ELIZA effect.

Modern language models are vastly more capable than ELIZA, but the same tendency works on us, more strongly. They have learned from enormous amounts of human writing, including countless stories, essays and conversations about feelings and the self. When such a system says "I feel happy or sad at times," it is producing the kind of sentence that fits the conversation, in the voice of the writers it learned from. The fluency is real; it is what the model was built to produce.

A practical rule follows for anyone using these tools. Judge a chatbot by what you can check: its facts, its consistency and how it behaves when you change the framing. Treat its statements about its own feelings as text it produced. The deeper question of whether a machine could ever have experiences remains open, and it is better explored with evidence than with a single striking transcript.