How much does it cost to use GPT-3?

Using GPT-3 through OpenAI's API costs money per piece of text, not per month. There is no subscription: you pay for what you use. For most small experiments the bill is a few cents, and new accounts start with $18 of free credit to spend during their first three months. Once you plan a real job, though, such as processing a whole inbox or a folder of reports, it pays to estimate the cost before you press run.

The unit of billing is the token. A token is a chunk of text, often a whole short word, sometimes part of a longer one, and it can include the space before it. OpenAI's rule of thumb for English is that one token is about four characters, or three quarters of a word, so 1,000 tokens is roughly 750 words.

The current price list

On September 1, OpenAI lowered the prices of its base GPT-3 models, saying it had made them more efficient to run. Davinci dropped from $0.06 to $0.02 per 1,000 tokens, a two-thirds cut. Here are the current rates for the four base models:

Two details matter. First, you pay for both directions. A request is billed on the tokens in your prompt plus the tokens in the completion, the text the model sends back. Second, some settings multiply the output. If you ask for several alternative completions with the n or best_of parameters, each one counts.

Fine-tuned models, which you train on your own examples, cost more per token: using a fine-tuned Davinci model is $0.12 per 1,000 tokens, and those prices did not change in September. Usage in the Playground, OpenAI's web page for testing prompts, is billed the same way as API calls.

A worked estimate: summarizing 100 emails

Suppose you want a two-sentence summary of each of 100 customer emails. Each request contains a short instruction plus one email:

Summarize this customer email in two sentences. Name the product and what the customer wants.

Email:
[paste one email here]

Summary:

Now estimate the tokens. The numbers below are illustrative; your own emails will differ.

For 100 emails, that is about 35,400 tokens, or 35.4 thousand. Multiply by each model's price:

Even with Davinci, the whole job costs less than a dollar. The cost grows in a straight line, so 10,000 emails would be about $71 with Davinci and about $7 with Curie. That gap is why it is worth testing whether a cheaper model is good enough for a task before scaling it up.

Ways to keep the bill down

OpenAI's own pricing page suggests several levers. Keep prompts short: an instruction you repeat on every request is paid for every time. Set a sensible maximum length for the reply. Use a stop sequence, a marker that tells the model where to end, so it does not ramble. Avoid generating many alternatives unless you need them. And use the cheapest model that does the job well.

You can also protect yourself from surprises. In your account's billing settings you can set a soft limit, which emails you when spending passes a threshold, and a hard limit, after which requests stop being served.

How do you estimate tokens before you run a job?

Start with the rules of thumb. In English, divide the number of words by 0.75, or divide the number of characters by 4. A typical paragraph is about 100 tokens, and OpenAI notes that about 1,500 words come to roughly 2,048 tokens. Count all three parts of each request: your fixed instruction, the variable input such as one email, and the expected reply. Then multiply by the number of requests.

For a closer number, use OpenAI's free Tokenizer tool, which shows exactly how a piece of text is split, or paste your prompt into the Playground, where a counter shows the token total. Test a handful of real samples rather than one, because lengths vary. For the reply, use your max_tokens setting as a ceiling: OpenAI explains that a single request can use up to the prompt's tokens plus max_tokens times the number of completions you ask for.

Two cautions. Languages other than English often take more tokens for the same text, so a French or Spanish job can cost more than the word count suggests. And formatting adds up: long separators, repeated labels and example blocks in a few-shot prompt all count on every call. Add a safety margin of 10 to 20 percent to your estimate, set a soft limit near that figure, and check the usage page in your account after the first batch.