What is AI Dungeon 2, and how does it use GPT-2?

Classic text adventures, like Zork, describe a scene in words and wait for you to type a command such as "go north" or "take lamp." The catch has always been that the game only understands what its designer planned for. AI Dungeon 2, which Nick Walton has made playable in a web browser in recent days, removes that limit. You can type anything, and a language model writes the result.

The game is drawing attention as a hands-on way to meet GPT-2, the text-generating model OpenAI finished releasing in full on November 5. PC Gamer described playing a wizard exploring a ruin who, a few turns in, met a younger version of himself. That kind of surprise is the point.

From AI Dungeon to AI Dungeon 2

Walton's first AI Dungeon used the smallest GPT-2 model, with 124 million parameters. To keep computing costs down, it generated a few possible actions and let the player pick one.

AI Dungeon 2 drops the menu. According to Walton's write-up on the blog of a Brigham Young University research lab, it makes three main changes:

Fine-tuning means taking a model that has already learned general language and training it a little more on a specific kind of text, so it adopts that style.

How the game keeps a story going

Each adventure starts when you pick a setting and a character. Those choices produce two pieces of text: a starting prompt, which is the opening of the story, and a "context sentence," which is fed to the model on every turn so it stays grounded in the chosen world.

Every time you type an action, the game sends GPT-2 the context sentence plus your last eight actions and their results, then asks it to continue. Walton found eight to be a good balance. Anything older drops out of the model's view, which is why a character you met long ago can be forgotten.

There is one more adjustment. GPT-2 tends to repeat itself, so, following an idea from Salesforce's CTRL model, the game penalizes words that have already been generated. That lets it use a lower "temperature," a setting that controls how adventurous the word choices are, without getting stuck in loops.

Here is roughly what the model receives on a single turn, in the style of Walton's own wizard example:

You are Mira, a knight in the kingdom of Larion. You carry a sword and a shield.
[recent turns]
> enter the cave
You step into the cave. Water drips from the ceiling and you hear a low growl.
> light a torch
The torch flares, and you see a sleeping bear curled beside a chest.
> sneak toward the chest

The model's job is simply to write the next few sentences.

Limits you will notice

Walton is candid: the model sometimes shows remarkable understanding, and at other times it confuses which character is speaking or acting, above all in conversations. PC Gamer notes that it can loop, get stuck or crash, and that it takes a couple of minutes to load. It also inherits the weaknesses OpenAI described when releasing GPT-2: the text can be convincing without being consistent, and the model has no built-in sense of truth or of the game's rules.

What can an AI text adventure teach us about language models?

First, it shows what a language model actually does. GPT-2 is not running a game world with maps, inventories and rules. It is predicting the next words given the text in front of it. When the story feels coherent, that comes from patterns learned in training plus the context the game supplies; when it falls apart, you are seeing the edges of that context.

Second, it shows that the input matters as much as the model. AI Dungeon 2's design choices, the always-present context sentence, the eight-turn memory and the repetition penalty, are all ways of shaping what goes into the model and how it picks words. Anyone building with language models faces the same questions: what does the model need to see, and what can be left out?

Third, it is a playful, low-stakes way to build good instincts. In a fantasy game, an invented king or a sudden dragon is part of the fun. In a news article or a homework answer, the same habit of confident invention is a problem. Playing for an hour teaches you to expect fluent surprises, and to check anything that matters.