# How do you use one model to grade another model's answers?

Date: 2024-11-16 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/use-model-grade-another-models-answers/

**Answer:** You give a second model, the judge, the question, the answer to check and a short written rubric, and ask it to reason against each criterion before giving a score in a fixed format. It works well enough to test hundreds of answers quickly, as long as a person checks a sample of its grades and you guard against its known biases.

When you build something with a language model, such as a support assistant or a summarizer, you soon need to know whether a change to your prompt made the answers better or worse. Reading 200 answers by hand after every change is slow. For questions with one correct answer, a simple program can compare the output to the expected text. For open-ended answers, like an email reply or an explanation, there is no single right string to match.

That is where a "model as judge" comes in. A second model reads each answer and grades it against criteria you write down. Researchers at UC Berkeley and their colleagues tested this idea in 2023 with a paper titled *Judging LLM-as-a-Judge*. They found that GPT-4, acting as a judge, agreed with human preferences more than 80% of the time, about as often as two humans agree with each other. That makes a judge a useful first filter, not a final word.

## Three ways to set up a judge

The same paper describes three set-ups, and each suits a different job.

| Set-up | What the judge sees | Good for |
|---|---|---|
| Single answer grading | One question and one answer; returns a score | Tracking quality over time on a fixed test set |
| Pairwise comparison | One question and two answers; picks the better one or calls a tie | Choosing between two prompts or two models |
| Reference-guided grading | The answer plus a correct reference solution | Math, facts, anything with a checkable result |

Single scores are easy to run at scale but can miss small differences. Pairwise comparison catches those differences, but the number of pairs grows quickly when you compare many versions.
## Write the rubric first

A rubric is a short list of criteria, each with a clear meaning for every score. A vague instruction like "rate this answer from 1 to 10" invites vague grades. A rubric with three or four criteria and a small scale (1 to 3, or pass and fail) gives more consistent results, because the judge has less room to guess what you meant.

Here is an illustrative rubric for a bakery's customer-service assistant:

| Criterion | 1 | 2 | 3 |
|---|---|---|---|
| Correct | States something wrong about hours, prices or policy | Partly right, or vague | Fully matches the facts provided |
| Helpful | Doesn't address the question | Addresses it, but misses the next step | Answers and tells the customer what to do next |
| Tone | Cold or overly casual | Polite but stiff | Warm and professional |

## A worked judge prompt

The judge prompt combines the facts, the rubric, the question and the answer. It asks the judge to explain its reasoning for each criterion before it scores, which tends to make grading more careful, and it asks for the result in JSON, a simple data format your code can read.

```prompt
You are grading replies written by a customer-service assistant for a bakery.

Facts the reply must respect:
- Open Tuesday to Sunday, 7:00 to 17:00. Closed Mondays.
- Custom cakes need 72 hours' notice and a 20% deposit.

Rubric (score each criterion 1, 2 or 3):
- correct: 1 = wrong about hours, prices or policy; 2 = partly right or vague; 3 = fully matches the facts.
- helpful: 1 = doesn't answer; 2 = answers but misses the next step; 3 = answers and gives the next step.
- tone: 1 = cold or too casual; 2 = polite but stiff; 3 = warm and professional.

Customer question:
<question>
Can I order a birthday cake for this Saturday? Today is Thursday.
</question>

Reply to grade:
<reply>
Of course! Just come by Friday and we'll have it ready for Saturday.
</reply>

First, write one sentence of reasoning for each criterion. Then output only this JSON on the last line:
{"correct": n, "helpful": n, "tone": n}
```

An illustrative result: the judge notes that Thursday to Saturday is less than 72 hours and the reply never mentions the deposit, so it scores correct 1, helpful 1 and tone 3. A friendly tone had hidden a wrong promise.

In code, you loop over your test questions, send each answer through this prompt, read the JSON, and average the scores per criterion. Set the temperature (the setting that controls randomness) to 0 so the same answer gets a more consistent grade from one run to the next.

> **Try it:** Write five customer questions and two versions of your assistant's prompt. Grade both sets of replies with the judge prompt above, then read the three replies with the lowest "correct" score yourself and decide whether the judge was right.

## How do you keep the judge honest?

Start by knowing its biases. The Berkeley study names three. Position bias: when comparing two answers, judges tend to favor one position, often the first. Verbosity bias: judges can prefer a longer answer even when it adds nothing. Self-enhancement bias: a judge may favor answers written by its own model, though the study could not prove this. Judges also stumble when grading math, sometimes misled by a wrong answer they could have solved correctly on their own.

Each bias has a practical counter. For pairwise grading, run every comparison twice with the answers swapped, and count a win only when both orders agree; otherwise, call it a tie. To counter verbosity, add a line to the rubric such as "Length is not a criterion; do not reward repetition." Where possible, use a judge from a different model family than the one being graded. For math or facts, give the judge a reference answer. In the study, having the judge first solve the problem and then use that solution as a reference cut its math grading failures from 70% to 15%.

Then check the judge against people. OpenAI's open-source Evals framework recommends exactly this for model-graded tests: collect human labels for a set of examples and measure how often the judge's choice matches them. You can do the same in a spreadsheet. Pick 20 to 30 answers at random, grade them yourself without looking at the judge's scores, then compare. If you agree on most of them, keep the judge and repeat the spot check whenever you change the rubric or the judge model. If you disagree often, read the disagreements: they usually point to a criterion that needs a clearer definition.

## Sources

- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena: https://arxiv.org/abs/2306.05685v4
- Building an eval (openai/evals docs): https://github.com/openai/evals/blob/bd3b4d0afa7785f0374c46c32a32dd4c55105c28/docs/build-eval.md