What is a system message in the ChatGPT API?

Since March 1, developers can build ChatGPT's model, gpt-3.5-turbo, into their own apps through OpenAI's API, the interface that lets one program send requests to another. The new chat endpoint does not take a single block of text the way older GPT-3 models did. It takes a list of messages, and each message carries a role that says who is speaking.

There are three roles. User messages are what the person types. Assistant messages are the model's replies, or examples of replies you write yourself. System messages are instructions from the developer that frame the whole conversation. OpenAI's own examples usually start with a system message, then alternate between user and assistant, though the order is not required.

Why a separate role matters

Before this, anyone building on GPT-3 had to paste instructions and the user's question into one long prompt, and the model had no reliable way to tell them apart. Under the hood, the chat models still read everything as tokens, the small pieces of text a model processes. But the input arrives in a structured format OpenAI calls Chat Markup Language, or ChatML, which marks each part with its role.

OpenAI president Greg Brockman told TechCrunch that this structure, where the developer's words and the user's words are labeled separately, should make apps more robust against prompt attacks: messages crafted to make a bot ignore its instructions. He gave the example of a tutoring app that should always explain an answer and help the student learn, rather than just hand over the solution. That kind of rule is exactly what a system message is for.

There is an important caveat. OpenAI's cookbook, its collection of official code examples, notes that the current model snapshot, gpt-3.5-turbo-0301, tends to give the system message less weight than the user's own messages. It recommends placing important instructions in the user message too. Think of the system message as the job description, and the user message as the specific request in front of the assistant right now.

A worked example: a library help bot

Imagine a small public library that wants a chat assistant on its website. Here is the list of messages a developer would send. The system message comes first:

system: You are the help assistant for the Riverside Public Library. Answer questions about borrowing, library cards, opening hours, room bookings and finding books. Keep answers under 80 words and use plain language. If you are not sure of a library policy, say so and suggest the person call the front desk. Do not answer questions unrelated to the library; politely steer the conversation back. Never ask for card numbers or personal details.

user: How many books can I borrow at once, and for how long?

The model's reply comes back as an assistant message. To continue the conversation, the developer adds that reply and the next user message to the list and sends the whole list again; OpenAI's examples pass the conversation so far with every request.

Now test it. A good test conversation covers the normal case, an edge case and an attempt to pull the bot off task:

The library's real rules matter here. The system message above does not contain the actual loan limits, so a well-behaved bot should say it is unsure rather than make one up. To give correct answers, the developer should add the real policies to the system message or the user message.

How do you write one that keeps the assistant on task?

Start with identity and scope in the first sentence: who the assistant is and which topics it covers. Then add the rules for format (length, tone, reading level) and the rules for limits (what to refuse, when to admit it doesn't know). Short, concrete instructions work better than vague ones; "under 80 words" is easier to follow than "be concise." OpenAI's cookbook illustrates the idea with one question about fractions sent under two system messages: one sets up a friendly teaching assistant who explains in depth with examples, the other a laconic assistant who keeps answers brief.

Next, back it up. Because gpt-3.5-turbo can drift from the system message, repeat the one or two rules that matter most inside the user message your app sends, for example by adding "Answer only about library services" before the person's question. You can also include a few example user and assistant exchanges to show the style you want, a technique known as few-shot prompting.

Finally, watch the token budget. The model handles up to 4,096 tokens per request, and that limit covers the system message, the whole conversation history and the reply. At $0.002 per 1,000 tokens, a long system message is cheap, but it is sent with every request, so trim anything that does not change the bot's behavior. Then run your test conversation again after every change.