Who is responsible when ChatGPT gives bad advice?

Three weeks after ChatGPT opened to the public, a great many people are asking it things they used to ask a friend, a search engine or a professional. Most answers are reasonable. Some are wrong in ways that are hard to spot, because they are written in the same calm, fluent style as the right ones. So a fair question comes up: if you follow a chatbot's advice and it goes badly, whose fault is that?

This article explains what OpenAI's own documents say and what that means in practice. It is a general explanation, not legal advice; laws differ from place to place, and only a lawyer can tell you how they apply to a specific situation.

What OpenAI's terms say

OpenAI's Terms of Use, last modified on December 13, apply to all of its services, ChatGPT included. A few points are directly relevant:

• No promise of accuracy. The terms say the services are provided "as is", and that OpenAI does not warrant they will be accurate or free of errors.
• The user is responsible for content. The terms call what you type the input and what the model returns the output, and together the content. You own your input, OpenAI assigns its rights in the output to you, and you are responsible for that content, including making sure it does not break the law or the terms.
• Limited liability. The terms cap OpenAI's liability at the greater of what you paid for the service over the previous 12 months or $100. For a free preview, that is a small number.
• Age. Users must be 18 or older.

OpenAI is also open about the model's weaknesses. Its launch post says ChatGPT sometimes writes answers that sound plausible but are incorrect or make no sense, and explains part of the reason: during one stage of training there is no source of truth for the model to check against.

"You are responsible for Content, including for ensuring that it does not violate any applicable law or these Terms." (OpenAI Terms of Use, December 13, 2022)

What this means in practice

Read together, these documents describe a tool, not an advisor. A calculator company does not check your tax return, and a word processor does not vouch for the letter you type in it. ChatGPT sits closer to those tools than to a doctor or an accountant: it produces text, and the person who uses that text decides what to do with it.

Stack Overflow, the large question-and-answer site for programmers, gave a vivid example of why this matters. On December 5 its moderators temporarily banned answers written with ChatGPT. Their explanation was not that the answers were always wrong, but that many were wrong while looking good, that they were very easy to produce, and that many people posted them without the knowledge or the willingness to check them first. The result was thousands of answers that each needed an expert to review. The lesson carries beyond programming: the cost of an unchecked answer lands on whoever relies on it.

That does not make the tool useless for serious topics. It changes the job you give it. Ask it to explain, organize and prepare, and leave the deciding to you and to qualified people.

A worked example: asking a health question safely

Suppose your doctor has said your blood pressure is "a bit high" and you want to understand it before your next appointment. Compare two ways of asking.

The risky version asks the chatbot to make a medical decision:

My blood pressure is 145 over 92. Should I stop taking my medication and try diet instead?

A safer version asks it to help you understand and prepare, and leaves the decision with your doctor:

My doctor told me my blood pressure is a bit high. I am not asking you to diagnose me or change my treatment.

1. Explain in plain words what the two numbers in a blood pressure reading mean.
2. List general lifestyle factors that are commonly discussed with doctors about blood pressure.
3. Write eight questions I could bring to my next appointment.

Mark anything you are unsure about.

The second prompt plays to the tool's strengths: explaining terms, organizing a topic and drafting questions. It does not ask for a dose, a diagnosis or a yes-or-no decision. And it keeps personal details to the minimum, which matters because OpenAI's terms say it may use submitted content to improve its services.

How should you use it for health, legal or money questions?

Use it before and after the conversation with a professional, never instead of it. Before, it can explain vocabulary, outline how a topic is usually organized and help you write down what to ask. After, it can help you summarize your own notes in plain language. In both cases, you supply the facts and a qualified person supplies the judgment.

Watch for three warning signs in any answer: a specific number (a dose, a deadline, an amount) you have not seen in an official source, a confident statement about what the law or a rule "says" without a reference you can check, and advice that would be hard to undo. Each is a cue to stop and verify. ChatGPT shows no sources for its answers, so you cannot tell whether a rule it describes is current.

Finally, keep private details out of the chat. You rarely need your full name, account numbers or medical records to get a useful explanation. A question phrased in general terms usually gets a better answer anyway, and it keeps the most sensitive part of the problem where it belongs: with you and the people you trust to advise you.