# What is Google's LaMDA, and how is it different from other chatbots?

Date: 2021-05-19 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/what-is-google-lamda/

**Answer:** LaMDA, short for Language Model for Dialogue Applications, is an experimental Google language model unveiled on 18 May 2021 at Google I/O that was trained on dialogue so it can hold open-ended conversations that drift naturally from topic to topic. Google judges its replies on whether they make sense and are specific, and is also working on making them interesting and factually correct, but calls it early research that is not yet in any product.

Yesterday, 18 May 2021, at its annual I/O developer conference, Google introduced **LaMDA**, a research model built for conversation. In a short demo, the model spoke as the dwarf planet Pluto, and then as a paper airplane, answering questions in character.

Most chatbots today follow narrow, scripted paths: ask about your bank balance or the weather and they cope; wander off topic and they stall. LaMDA aims for the kind of conversation people actually have, where a chat about a TV show can drift to the country where it was filmed and end with a debate about that country's food.

## What Google showed at I/O

Google CEO Sundar Pichai presented LaMDA during the keynote with two videos of short conversations. In the first, the model played Pluto. According to The Verge, it brought up real facts, such as the New Horizons probe that flew past Pluto in 2015. (Both Pichai and the Pluto persona called it a planet; astronomers classify it as a dwarf planet.) In the second, LaMDA answered as a paper airplane.

The Verge notes that the model apparently had no special training for either role. Pichai called the conversations impressive but was careful to add a warning: "But it's still early research, so it doesn't get everything right."

The same day, Google published a blog post by Eli Collins, a vice president of product management, and Zoubin Ghahramani, a senior research director, explaining how LaMDA works.

## How LaMDA works

The name stands for **Language Model for Dialogue Applications**. A **language model** is a program trained to predict which words come next in a piece of text. Like Google's BERT and OpenAI's GPT-3, LaMDA is built on the **Transformer**, a neural network design that Google researchers invented and released in 2017. A Transformer reads a whole passage, works out how its words relate to each other, and uses that to predict what follows.

The key difference is the training material. Most language models learn from general text such as web pages and books. LaMDA was trained on dialogue, so it picked up the patterns of back-and-forth conversation. Google says it builds on the company's 2020 research showing that Transformer models trained on conversations can learn to talk about almost anything. After this main training, Google **fine-tunes** LaMDA, giving it extra, targeted training to improve the quality of its replies.

## A worked example: what makes a good reply

Google grades LaMDA's answers on two main qualities. **Sensibleness** asks whether a reply makes sense in context. **Specificity** asks whether it clearly relates to what was just said. Google's own example starts with one person saying, "I just started taking guitar lessons." Here is how that reply, Google's sensible and specific example, compares with two weaker ones of the kind Google describes:

| Reply | Sensible? | Specific? | Why |
|---|---|---|---|
| "I like turtles." | No | No | It ignores what was said |
| "That's nice." | Yes | No | It fits almost any statement, so it says nothing |
| "How exciting! My mom has a vintage Martin that she loves to play." | Yes | Yes | It responds to guitars in particular and keeps the chat going |

The middle row is the trap. Generic replies like "that's nice" or "I don't know" are always safe, which makes them tempting for a chatbot, but they kill a conversation. LaMDA is trained to avoid them.

Google is also exploring two more qualities. **Interestingness** means replies that are insightful, unexpected or witty. **Factuality** means sticking to the facts, something Google admits language models often struggle with. The company says it wants answers that are not only compelling but correct.

> **Try it:** Pick an ordinary opening line, such as "I just got back from a camping trip." Write three replies: one that makes no sense, one that is sensible but generic, and one that is sensible and specific. Then write a fourth that is also interesting. Notice how much harder each step is; that is exactly the ladder Google is trying to teach LaMDA to climb.

## Why it matters, and the concerns

For Google, better conversation is tied to its core business. Pichai pointed out that much of the company's AI work is about finding information, from translating languages to understanding what people mean when they search. If a computer can follow a natural, wandering conversation, searching the web or using a phone could feel more like talking to a knowledgeable person. Google says it plans to bring conversational abilities into more of its products.

Google also stresses the risks. Its blog post notes that models trained on language can absorb biases, echo hateful speech or repeat misleading information, and that even a carefully trained model can be put to bad use. The company says LaMDA has been scrutinized at every step under its AI Principles, and that minimizing these risks is its top priority.

Some observers are skeptical of that message. The Verge points out that the emphasis on responsible AI comes after Google fired Timnit Gebru and Margaret Mitchell, two of its best-known researchers on AI ethics, in a dispute linked to a paper on the dangers of large language models, and that this went unmentioned on stage.

## When will you be able to talk to LaMDA?

Not yet. LaMDA is a research project, and Google has not announced a public version, an app or a date. Its blog post says the early results are encouraging and that it expects to share more soon.

The most likely path, based on what Google said, is that LaMDA's abilities will appear inside existing products rather than as a standalone chatbot. The Verge reports that Google sees it as a way to make its conversational assistants more natural in future. Before that happens, Google says it wants to improve factual accuracy and keep testing for bias and misuse.

In the meantime, the useful lesson for anyone building or using chatbots is the scoring idea behind LaMDA. When you judge any AI conversation partner, ask the same questions Google asks: Does this reply make sense? Is it specific to what I said? Is it interesting? And is it true?

## Sources

- LaMDA: our breakthrough conversation technology: https://blog.google/technology/ai/lamda/
- Google showed off its next-generation AI by talking to Pluto and a paper airplane: https://www.theverge.com/2021/5/18/22442328/google-io-2021-ai-language-model-lamda-pluto