# How do you turn a plain-English request into working code with Codex?

Date: 2021-09-15 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/turn-plain-english-request-into-working-code/

**Answer:** Write your request as the opening of the file you want, in comments that name the language, the pieces and the edge cases, then let Codex continue it. Build in small rounds: run what it writes, see what is missing, and add one more precise line of instruction each time.

Since August, developers admitted to the private beta of OpenAI Codex can send it a description in ordinary English and get code back. The first attempts often disappoint: a vague request produces vague code. The good news is that a short, repeatable method makes results far more dependable, and it works whether you are a seasoned developer or someone writing their first web page.

The method rests on one fact. Codex is a descendant of GPT-3 that also learned from billions of lines of public code. Like GPT-3, it continues text. It does not "obey" your request so much as guess what a well-written file starting with your words would contain next. Your job is to make that file's opening so clear that the right code is the obvious continuation.

## Step 1: Phrase the request like a specification

A useful request answers the questions a careful programmer would ask before starting:

| Question | Weak wording | Stronger wording |
|---|---|---|
| Which language and format? | "make a calculator" | "a single HTML page with inline CSS and JavaScript" |
| What are the parts? | "with inputs" | "a number box for the bill, a menu for 15%, 18% or 20%, a Calculate button" |
| What should happen? | "show the answer" | "show tip and total, rounded to two decimals, in a paragraph with id result" |
| What about bad input? | (nothing) | "if the amount is empty or negative, show a polite error" |

Every choice you leave out is a choice Codex makes for you, usually the most common one in its training data. That may not be the one you want.

## Step 2: Put the request where code would start

Write the request as comments at the top of the file, in the comment style of the target language, and then begin the code yourself. Those first lines of real code are context lines: they show the model which language, which structure and which style to follow. For a web page, the opening might look like this:

```prompt
<!-- A single HTML page with inline CSS and JavaScript. Title: "Tip calculator". -->
<!-- A number input for the bill amount (id "bill"). -->
<!-- A select menu with tip options 15%, 18% and 20% (id "rate"). -->
<!-- A button labelled "Calculate". When clicked, show the tip and the total, -->
<!-- each rounded to 2 decimals, in a paragraph with id "result". -->
<!-- If the amount is empty or negative, show "Please enter a valid amount." instead. -->
<!DOCTYPE html>
<html lang="en">
<head>
```

The final three lines matter more than they look. They commit Codex to HTML, to a full page, and to starting with the head section, so it is less likely to wander into a different language or a code fragment.

## Step 3: Run it, then add one line at a time

Save what Codex writes as a file, open it in a browser and try it. Then improve it in rounds, changing only one thing each time:

1. **Check the basics.** Enter 50 and choose 20%: you should see a 10.00 tip and a 60.00 total.
2. **Test the edges.** Leave the box empty, then type -5. If no error appears, add a sharper comment about validation and generate again.
3. **Add a feature with a new comment.** For example: `<!-- Add a "Split between" number input; also show the amount per person. -->`
4. **Keep what works.** Paste the good version back in as context, so the next round builds on it instead of starting from scratch.

Small rounds keep each change easy to check. They also play to Codex's strengths: the research paper behind it found the model does well on short, well-described functions and worse when one description chains together many steps.

OpenAI says Codex has a memory of about 14KB for Python code, more than three times GPT-3's, but that space still fills up. If your file grows large, keep the comments that describe the current goal near the end, close to where Codex will write.

> **Try it:** Write the six comment lines above for a page you actually want, such as a unit converter or a reading timer. Generate a version, then test it with one normal value, one empty value and one silly value. Add a single comment line to fix whatever broke, and generate again.

## What kinds of tasks does Codex still get wrong?

The clearest evidence comes from OpenAI's July paper, *Evaluating Large Language Models Trained on Code*. On its HumanEval benchmark of short Python problems checked by unit tests, the research version of Codex solved 28.8% on the first try. Allowed 100 attempts per problem, at least one sample worked for 70.2% of them. In other words, the first answer is usually not the right one, and someone has to tell good from bad. OpenAI describes the API model as improved, but it has not published new scores for it.

The paper also names specific weak spots. Codex struggles when a description lists a long chain of operations, and when it has to keep track of which operation applies to which variable. In practice that shows up as code that handles the first few requirements and quietly drops the rest, or mixes up two similar values, such as the tip and the total.

Some failures come from the request rather than the model. Ambiguous wording leads to code that is correct for a different reading of your words. Missing edge cases lead to code that works only on friendly input. And because Codex learned from public code of every age and quality, it can repeat outdated or unsafe patterns; the paper discusses security as one of the broader risks of code generation. The safe habit is the same as for any code you did not write yourself: read it, test it with awkward input, and never paste it into anything that handles passwords, payments or personal data without careful review.

## Sources

- OpenAI Codex: https://openai.com/index/openai-codex/
- Evaluating Large Language Models Trained on Code: https://arxiv.org/abs/2107.03374