Society and work

Who is responsible when an AI agent makes a mistake?

· By Robert Scales · 5 min read

Short answer

In most cases the business or person that deploys the agent answers for what it does, not the agent and usually not the AI provider. Provider terms put the responsibility for checking outputs on the customer, and a Canadian tribunal has already held a company responsible for its chatbot's wrong answer, so plan as if every message your agent sends is one you sent yourself.

An AI agent is software that uses a language model to take actions, such as replying to customers, booking appointments or updating records, rather than only suggesting text for a person to send. When an agent makes a mistake, it is tempting to see it as the AI's fault. But an agent is not a legal person. It cannot sign a contract, pay damages or be sued. Responsibility lands on people and organizations, and the question is which ones.

Nothing here is legal advice; the answer depends on where you operate, your contracts and the facts. But three kinds of source give a clear picture of how responsibility is being assigned: the contracts AI providers write, early decisions by courts and tribunals, and new regulation.

What a tribunal said about a chatbot

The best-known case so far is Moffatt v. Air Canada, decided by British Columbia's Civil Resolution Tribunal, a small-claims body, in February 2024. A customer asked the airline's website chatbot about bereavement fares after a death in the family. The chatbot told him he could apply for the discount after travelling. The airline's own policy page said the opposite, and the airline refused the refund.

Air Canada argued it could not be held liable for what the chatbot said. The tribunal rejected that, calling the idea that the chatbot was a separate legal entity "a remarkable submission." Its reasoning was simple: the chatbot is part of the company's website, and the company is responsible for everything on it. In the tribunal's words, "It makes no difference whether the information comes from a static page or a chatbot." It found the airline had not taken reasonable care to make the chatbot accurate and ordered it to pay $812.02.

It is a small-claims decision in one Canadian province, so it does not bind courts elsewhere. Still, it shows how a decision-maker is likely to reason: the business that put the tool in front of customers owns what the tool says.

What provider contracts say

If the business is responsible, can it pass the blame to the company that made the model? The contracts point the other way. OpenAI's Services Agreement for business and API customers, in force since January 1, 2026, makes the customer solely responsible for how it uses outputs and for judging whether they are accurate and appropriate for its purpose. The customer is also responsible for everything done through its account, including by users of apps it builds. The services come "as is," and the customer agrees to cover OpenAI against third-party claims arising from its own applications.

Other providers' commercial terms differ in detail, so read the ones you signed. The pattern to expect is that the provider stands behind narrow promises (OpenAI, for example, covers claims that its services infringe intellectual property), while the results of your particular use remain yours.

What regulation adds

The EU's AI Act uses the word deployer for any organization that uses an AI system under its authority in a professional setting. Most of its rules apply from August 2, 2026. Two duties touch almost any business agent. First, providers and deployers must take steps to give staff who operate AI systems a sufficient level of AI literacy, meaning they understand what the system can and cannot do. Second, AI systems that interact directly with people must tell them they are dealing with an AI, unless that is obvious. Agents used in higher-risk areas such as hiring or credit carry much heavier obligations.

A worked example: an agent that sends emails

Picture a small accounting firm whose agent drafts and sends replies to client emails: confirming appointments, sending document checklists and answering routine questions. Here is an illustrative accountability chart:

Decision or taskOwnerWhat they must do
Decide which emails the agent may send aloneManaging partnerApprove a written list; everything else needs a person
Write and update the agent's instructionsOperations leadKeep a dated copy of every version
Approve any email about fees, deadlines or tax adviceAssigned accountantRead and click "send" personally
Review a weekly sample of sent emailsOffice managerCheck 20 at random; log errors
Handle a complaint or wrong emailPartner on the accountContact the client, correct it, record the cause
Check provider terms and data settingsManaging partnerReview at each renewal or terms update
Train staff on the agent's limitsOperations leadShort session for every new user

The chart does not try to make the agent responsible for anything. Every row names a person, because only a person can be.

Try it: List every action your agent, or one you are planning, can take without a human click. For each, write one name in a column called "answers for this." Any action you cannot assign is an action the agent should not take alone yet.

How should a business set up accountability?

Start by deciding what the agent may do on its own, and keep that list short at first. Separate actions that are easy to undo (saving a draft, tagging an email) from those that are not (sending a message, issuing a refund, deleting a file). Require human approval for the second group until you have evidence the agent handles them well.

Name an owner for every permission, as in the chart above, and make the agent identify itself as automated in its messages. Keep records: the instructions in force, what the agent sent, and who approved it. If something goes wrong, those records let you show you took reasonable care, which is the very thing the Air Canada tribunal found missing.

Finally, review regularly. Sample the agent's work, track errors and the fixes you made, and re-read your provider's terms when they change. Responsibility is easier to carry when it is visible: a business that knows who checks what is in a far better position than one that discovers, after a complaint, that nobody did.

Sources

  1. Moffatt v. Air Canada, 2024 BCCRT 149 — Civil Resolution Tribunal of British Columbia (via CanLII), February 14, 2024
  2. OpenAI Services Agreement — OpenAI, December 1, 2025
  3. Regulation (EU) 2024/1689 (Artificial Intelligence Act) — Official Journal of the European Union (EUR-Lex), July 12, 2024

Get the free Guide to Artificial Intelligence

Sign up for the weekly newsletter and receive the free guide: what changed in AI this week, one lesson, one prompt to try. Unsubscribe in one click.

How we handle your address

Back to top

Display

Your choices are saved in this browser only. No data is sent.

Text size
100%
Colours
Reading
More settings
Accessibility statement