How do you test an AI tool on your own tasks before you trust it?

A demo shows you the best case. A test shows you the usual case and the bad ones. Before an AI tool drafts your client emails, sorts your invoices or answers your customers, it deserves the same kind of trial you'd give a new hire: real work, checked against a clear standard, enough times that luck doesn't decide the result.

OpenAI's evaluation guide has a name for what most people do instead: vibe-based evals, judging that a tool "seems to work" after a few tries. The fix is not complicated. It is a small, fixed set of your own tasks that you run every time you consider a tool, a model or a settings change.

What's different about testing a tool in 2026

If you have built a test set for prompts before, three things are different now.

• You are often testing a product, not a model. An assistant with connectors, memory and an agent mode can fail in its search, its tool use or its final answer. Score the end result, but note where it went wrong.
• Agents act, so test actions. For an agent, a case should state what it is allowed to do and what it must not do. "Didn't email the client" can be part of a pass.
• Results vary from run to run. The same input can give different outputs, so run important cases more than once, especially for agents.

Build the test in four steps

1. Choose 30 real tasks. Pull them from last month's work, not from your imagination. Both Anthropic and OpenAI stress matching the real mix of tasks, so include the dull typical ones, a few hard ones and a few odd ones: typos, missing information, a request the tool should refuse.

2. Write the pass criteria before you run anything. For each task, list two to four things a good result must contain or avoid. Make them checkable: "states the refund deadline as 30 days", not "is accurate".

3. Decide how each case is graded. Some can be checked exactly (a number, a date, a category). Others need a rubric. A model can grade against the rubric, which is faster, but OpenAI notes that model graders favor longer answers and whichever answer comes first, so check a sample of its grades against your own before relying on it.

4. Run, grade blind, and record. If you compare two tools, hide which output came from which. Note pass or fail per case, plus a word on why.

Here is a grading prompt you can reuse:

You are grading an AI tool's output against a checklist. You will see the task, the checklist and the output.
For each checklist item, answer PASS or FAIL with one short reason quoting the output.
Then give an overall PASS only if every item passes. Do not reward length or polish; judge only the checklist.
Task: [paste]
Checklist: [paste]
Output: [paste]

Reading the results honestly

A score of 24 out of 30 looks like "80%", but small tests are blurry. Anthropic's research on evaluation recommends reporting an error margin with every score. For 30 cases, a rough 95% range around 80% runs from about 66% to 94%. The tool is probably decent; you can't say much more.

Comparisons are sharper when both tools face the same cases. Anthropic points out that looking at question-by-question differences gives a more precise answer than comparing two overall scores. In practice: count the cases where A passed and B failed, and the reverse. If it's 9 to 2, that's a real difference. If it's 5 to 4, it isn't.

Look at the failures, too, not only the count. One failure that sends wrong information to a client may matter more than five clumsy phrasings.

How many examples do you need for a useful test?

It depends on the decision. To rule out a tool that is clearly unfit, 20 to 30 well-chosen cases are enough, because big problems show up quickly. To choose between two tools that both seem good, you need more: roughly 100 cases narrows the range around a score to about plus or minus 8 points, and a paired comparison on the same cases helps more than adding cases does.

Anthropic's documentation favors more cases with simple automated checks over a few lovingly hand-graded ones, and that's good advice once your set is stable. Start with 30, grow it by adding every real failure you meet, and rerun the whole set whenever the tool, the model behind it or your instructions change. A test set that keeps growing is worth more than a perfect one you ran once.