# Can AI detectors tell if a student used ChatGPT?

Date: 2023-08-02 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/ai-detectors-tell-if-student-used-chatgpt/

**Answer:** Not reliably. OpenAI withdrew its own AI text classifier on July 20, 2023 because of its low accuracy, and a Stanford study found that popular detectors wrongly flagged more than half of essays written by non-native English speakers as AI-generated.

An AI detector is a program that reads a piece of writing and estimates whether a person or a language model produced it. Many teachers have tried them since ChatGPT appeared, hoping for a quick answer to a hard question. The evidence so far says the answer they give is too often wrong to support a decision about a student.

The clearest signal comes from OpenAI, the company that makes ChatGPT. In January it released a free classifier for spotting AI-written text. On July 20 it took the tool offline, adding a note to its announcement that cited the classifier's low rate of accuracy. The company says it is now researching better ways to trace where a text comes from.

## How detectors work, and why they stumble

Most detectors look for writing that is statistically predictable. A language model tends to choose likely words in likely orders, so its text often has low perplexity, a measure of how surprised a model is by each next word. Human writing is usually more uneven.

The trouble is that plenty of human writing is predictable too. Short answers, formulaic essays, lab reports and lists all follow familiar patterns. OpenAI's own notes on its classifier listed the weak spots: it was very unreliable on texts under 1,000 characters, much worse in languages other than English, unreliable on code, and sometimes confidently wrong about human writing. It also warned that a few edits to AI-written text could slip past it.

A Stanford team led by Weixin Liang tested seven widely used detectors, including GPTZero and an OpenAI detector. On essays by US eighth-graders, the detectors were nearly perfect. On 91 essays written by non-native English speakers for the TOEFL, an English proficiency test, they labeled more than half as AI-generated, with an average false positive rate of 61%. Eighteen of those essays were flagged by all seven tools, and 89 of the 91 by at least one. A false positive here means an honest student accused. The authors traced the problem to vocabulary: simpler word choices look predictable, so learners of English are penalized for writing the way learners write.

## A worked example: one class, one detector

Numbers like "9% false positives" can sound small. Here is what they mean in a classroom. OpenAI reported that its classifier caught 26% of AI-written texts and wrongly flagged 9% of human-written ones. Imagine an illustrative class of 100 essays in which 10 were written with ChatGPT and 90 were written honestly.

| Group | Essays | Flagged as AI | Not flagged |
|---|---|---|---|
| Written with ChatGPT | 10 | about 3 (26%) | about 7 |
| Written by the student | 90 | about 8 (9%) | about 82 |
| **Total** | **100** | **about 11** | **about 89** |

Of the 11 essays flagged, roughly 8 belong to students who did nothing wrong, and 7 of the 10 AI-written essays pass unnoticed. The fewer students who actually use AI, the larger the share of flags that land on innocent students. OpenAI tuned its public tool to keep false positives low for exactly this reason, and still described it as not fully reliable.

> **Try it:** Rerun the table for your own situation. Pick a class size, guess how many students might use AI, and apply the 26% and 9% rates. Then ask yourself how many of the flagged students you would be willing to accuse.

## What a flag can and cannot mean

OpenAI said its classifier should never be the main basis for a decision, only one input among others. That is the right way to read any detector score. A flag is, at most, a reason to look more closely. It is not evidence of misconduct, and it is especially weak evidence for short answers, for students writing in a second language, and for any text the student may have edited heavily.

## What should schools do instead?

Start by looking at the writing process rather than only the product. Drafts, outlines, notes and the version history in a shared document show how a piece developed over time. A short conversation in which the student explains their argument, or expands on one paragraph, tells a teacher far more than a detector score, and it treats the student fairly.

Next, set clear rules for each assignment. Students need to know when AI help is allowed, what kind (brainstorming, grammar checks, feedback on a draft) and how to disclose it. Some tasks can be done in class, by hand or in conversation, where the question does not arise. Others can use ChatGPT openly, for example by asking students to critique and correct an AI draft. That exercise also teaches them how often these models get facts wrong.

Finally, if a school still uses a detector, it should treat the result as a prompt for a conversation, never as proof, and take particular care with English learners. The Stanford authors go further and caution against using these tools in evaluative or educational settings at all. OpenAI asked educators to send feedback and resources on what works in their classrooms, a sign that even the toolmakers are still looking for the right approach.

## Sources

- New AI classifier for indicating AI-written text: https://web.archive.org/web/20230801010754/https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text
- GPT detectors are biased against non-native English writers: https://arxiv.org/abs/2304.02819v3