# When should you use a GPT, and when the Assistants API?

Date: 2024-01-28 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/use-gpt-assistants-api/

**Answer:** Build a GPT when the people who need your assistant already use ChatGPT and you want something working this afternoon without code. Use the Assistants API when the assistant has to live inside your own website or app, connect to your systems, or serve people who don't have a ChatGPT plan.

Both options come from the same idea. A GPT is a custom version of ChatGPT that you set up by chatting with a builder: you describe the job, add instructions and upload the files it should draw on, with no coding at all. The Assistants API offers the same ingredients to programmers, so the assistant can run inside software you control. The real question is not which one is smarter, since both can use GPT-4, but where your users are and how much control you need.

A simple test settles most cases. If your users will open ChatGPT to get help, a GPT is the shortest path. If they will open your website, your app or your company tool, you need the API.

## The same helper, built two ways

Imagine a small public library that wants a helper to answer questions about its borrowing rules, fines and room bookings, using a 20-page policy PDF.

**As a GPT.** A staff member with a paid ChatGPT plan opens the GPT builder, describes the helper in plain English, uploads the PDF and writes a few rules: answer only from the policy, say so when the policy is silent, and keep replies short. Ten minutes later it works. On a ChatGPT Team plan, OpenAI lets colleagues build and share GPTs inside the workspace, and says it won't train its models on Team conversations. The catch: the helper lives inside ChatGPT, so library patrons would need their own accounts to use it, and the library can't put it on its own website.

**With the Assistants API.** A developer writes a short program. The API has three building blocks:

| Building block | What it holds | Library example |
|---|---|---|
| Assistant | Model, instructions, tools and files | "Library helper" with the policy PDF |
| Thread | One conversation, stored by OpenAI | One patron's chat on the website |
| Run | The assistant working on a thread | Reading the question and replying |

Here is the core of it in Python, using version 1 of the `openai` library and the beta Assistants endpoints as documented this month:

```python
import os, time
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

policy = client.files.create(file=open("library-policy.pdf", "rb"),
                             purpose="assistants")

helper = client.beta.assistants.create(
    name="Library helper",
    instructions=("Answer questions about library rules using only the attached "
                  "policy. If the policy does not cover it, say so and suggest "
                  "asking at the front desk. Keep answers under 80 words."),
    model="gpt-4-1106-preview",
    tools=[{"type": "retrieval"}],
    file_ids=[policy.id],
)

thread = client.beta.threads.create()      # one per patron conversation
client.beta.threads.messages.create(thread_id=thread.id, role="user",
                                    content="How much is the fine for a late DVD?")
run = client.beta.threads.runs.create(thread_id=thread.id, assistant_id=helper.id)

while run.status in ("queued", "in_progress"):   # runs are asynchronous
    time.sleep(1)
    run = client.beta.threads.runs.retrieve(thread_id=thread.id, run_id=run.id)

reply = client.beta.threads.messages.list(thread_id=thread.id).data[0]
print(reply.content[0].text.value)
```

You create the assistant once and save its ID; after that, each patron gets a new thread. The polling loop matters because a run doesn't answer instantly: it starts as `queued`, and your code checks back until it finishes. Messages come back newest first, which is why the code reads item `[0]`. Retrieval is the tool that searches the uploaded files, and OpenAI handles the splitting and indexing of the PDF for you.

> **Try it:** Pick a document you know well, such as a club's bylaws or a product manual. Build it as a GPT first and ask five real questions, then note which answers you would want to control more tightly. That list tells you whether the API is worth the extra work.

## What you control with each

A GPT gives you speed and a polished chat window for free. You don't manage servers, accounts or conversation storage. In exchange, you accept ChatGPT's interface and its rules about who can reach your GPT.

The API gives you control. You choose the screen it appears on, decide who can use it, log questions to improve your documents, and can add function calling so the assistant looks up live data such as whether a meeting room is free tonight. You also take on the work: hosting, error handling, keeping your API key secret and watching your spending.

Many projects use both in sequence. Build the GPT to test the idea with colleagues, learn which instructions work, then move those same instructions into an assistant when the idea proves its worth.

## What does each cost to build and run?

A GPT costs the price of the ChatGPT plan of whoever builds it. ChatGPT Plus is $20 a month; Team is $25 per user per month on an annual plan or $30 month to month. There is no per-question charge, so a busy GPT costs the same as a quiet one. Building it costs only time.

The Assistants API charges by use. You pay for tokens, the small chunks of text the model reads and writes, at the chosen model's rates: for GPT-4 Turbo that is $0.01 per 1,000 tokens read and $0.03 per 1,000 written. Two tool fees come on top. Code interpreter is $0.03 per session, and retrieval storage is $0.20 per gigabyte, per assistant, per day.

The detail that surprises people is that each run is billed for the whole conversation so far, plus any passages retrieval pulls from your files, even though you no longer send the history yourself. Long threads cost more per answer than short ones. As an illustration with made-up but plausible numbers: a reply that reads 5,000 tokens and writes 200 costs about $0.05 for input plus $0.006 for output, roughly six cents. A 20 MB policy file stored on one assistant works out to well under a cent a day at the published rate.

So the choice often comes down to volume and audience. For a handful of colleagues already paying for ChatGPT, a GPT is effectively free to run. For hundreds of patrons on a public website, the API's per-answer pricing is the only option, and a few cents per question is worth estimating before launch.

## Sources

- Assistants API Overview (Python SDK): https://github.com/openai/openai-cookbook/blob/2c441ab9a200070fee204a63b4203628c456e878/examples/Assistants_API_overview_python.ipynb
- Pricing: https://web.archive.org/web/20240110213730/https://openai.com/pricing
- OpenAI debuts ChatGPT subscription aimed at small teams: https://techcrunch.com/2024/01/10/openai-launches-chatgpt-subscription-aimed-at-small-teams/