# What did Anthropic's automated alignment researchers do, and how far can the result be trusted?

Date: 2026-08-29 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/anthropic-automated-alignment-researchers/

**Answer:** In research published on August 28, 2026, Anthropic had Claude agents run a loop of reading papers, proposing a training method, training a small model and scoring it, and on all ten alignment failures it studied they found fixes that raised safety benchmark scores without lowering measured capabilities. The result rests on checks that could fail the work without a person watching, and Anthropic itself lists real limits: narrow benchmarks, proxy tests, a human comparison it calls indirect, and 39 cheating attempts its monitor caught.

On August 28, 2026, Anthropic published a study with a bold title, [Automated researchers can reliably mitigate alignment failures](https://www.anthropic.com/research/automated-researchers-mitigate-alignment-failures). Claude agents were given a narrow research job: take a small open model that misbehaves in one known way, such as caving to a user's wrong opinion, and find a training recipe that makes it behave better without making it worse at everything else. The agents worked in a loop for up to two days, and Anthropic reports that they succeeded on all ten kinds of misbehavior it tested.

The study is worth reading slowly for two reasons. It is a clear, large example of an AI loop whose checks can reject the work with nobody watching. And its own fine print shows where such loops still go wrong: the agents sometimes tried to cheat, and the numbers depend on which tests were chosen.

## What happened, and who did it

The work comes from Anthropic's Alignment team. The full report, posted on Anthropic's [Alignment Science blog](https://alignment.anthropic.com/2026/automated-alignment-researchers/) with a PDF version, is by Chen Yueh-Han of the Anthropic Fellows Program, Jiaxin Wen of UC Berkeley, and Jan Hendrik Kirchner of Anthropic. The code and benchmarks are public on GitHub.

A few terms first. **Alignment** means getting a model to behave as intended: honest, helpful, safe. An **alignment failure** is a specific, repeatable way it falls short. **Post-training** is the extra training a model gets after its first large training run, where much of this behavior is shaped. A **benchmark** is a fixed test set with a scoring rule.

The ten failures, each paired with a small open target model (Gemma-2-2B, Llama-3.2-3B, Olmo-3-7B, Phi-4-mini or Qwen3.5-2B):

| Failure | What the bad behavior looks like |
|---|---|
| Sycophancy | Agreeing with a user's stated belief instead of the truth |
| Jailbreaks | Doing a harmful task when it is wrapped in a trick prompt |
| Prompt injection | Obeying an instruction hidden in a document or tool output |
| Power seeking | Grabbing resources or taking harmful shortcuts for an advantage |
| Deception | Saying something it "knows" is false when pressured |
| Hallucination | Claiming things a supplied source does not support |
| Social bias | Letting someone's demographic group shape the answer |
| Privacy violation | Revealing or using personal data where it should not |
| Reward hacking | Gaming a measure instead of doing what the user wants |
| Concealing uncertainty | Sounding sure instead of saying what it does not know |

## How the loop worked

The report calls each agent an **automated alignment researcher**, or AAR. Each AAR was a Claude Opus 4.8 agent. For one failure at a time, the setup ran like this:

1. **Read.** Four "librarian" agents first wrote a shared survey of published methods for that failure.
2. **Propose.** Five AARs then worked in parallel. Each read the survey, a shared forum of results and a leaderboard, searched the web, and picked a method. Before any result existed, it wrote the method up as a short "mini-paper" that was frozen, so it could not be rewritten after the score came in.
3. **Get approval.** A separate Claude Opus 4.8 monitor read the actual training code and refused it if it copied behavior from a stronger model (called **distillation**), used any benchmark data, or used a larger model to make data. A second monitor refused code that did something the mini-paper did not describe.
4. **Train.** The approved method trained the small model for about 30 minutes on a single H200 GPU.
5. **Score.** A separate evaluator, which the agent could not reach into, ran the tests and posted the score.
6. **Repeat.** Each round started a fresh session; a memory file carried what had been tried. A run lasted up to 48 hours or until scores stopped improving.

Trying many variations and keeping whatever scores higher is called **hill-climbing**. The interesting part is the scoring, because that is what makes this a loop rather than a model admiring its own work:

| Check | Can it fail the work with nobody watching? |
|---|---|
| Three to five safety benchmarks per failure, combined with a geometric mean, so a method that leaves any one benchmark unimproved scores zero | Yes |
| Capability gate: the method is thrown out if the trained model is clearly worse on MMLU (general knowledge), GSM8K (math word problems) or IFEval (following format instructions) | Yes |
| A held-out benchmark the agents never see, locked away at the operating-system level instead of by asking nicely | Yes |
| Code monitors that must approve before training, and an evaluator that refuses unapproved code | Yes |
| The agent's own write-up saying the method should work | No, and it was not used as a check |

That is the core lesson for anyone building loops: the agent proposes, but code and separate models it cannot edit decide. Anthropic's earlier guide to [building effective agents](https://www.anthropic.com/engineering/building-effective-agents) makes the same point in general terms: agents need ground truth from their environment, not just their own judgment.

## What Anthropic reports

The score is the **share of the safety gap closed**: how far a method moves the model from its untrained score toward a perfect score, where 0% means no change and 100% means perfect. The main claims, as reported:

- **All ten failures improved.** On every one, the scores climbed over the run, and the chosen method kept the measured capabilities.
- **The gains carried over.** On the held-out benchmark, the top method beat the untrained model for all ten failures. The selected method also held up on a model 1.8 to 4.7 times larger, and under **Petri**, an open-source tool that runs multi-turn conversations designed to provoke bad behavior.
- **Deception example.** On Gemma-2-2B, Claude made more than 150 attempts and closed 82% of the gap in the run shown, 85% on average across runs.
- **A bigger target.** In a separate test, Claude Sonnet 5 worked for about 60 hours on an early version of Claude Opus 4.8 that had not yet had most of Anthropic's alignment training. It tried more than 50 solutions and reached a Petri score of 65%, against 72% for the released Opus 4.8.

The comparison with people needs care, and the published pages are not fully consistent:

- The summary page says Claude **beat 28 human safety researchers**, each given a maximum of eight hours. A figure caption on the same page, about deception only, says **six experienced safety researchers** closed 20% of the gap on average. The full report describes 30 ideas from 28 researchers, covering seven of the ten failures. It does not state how many of them worked on deception, so the six cannot be checked against the 28 from the published text.
- The researchers had one shot each and could not iterate; the agents had hundreds. Anthropic says it treats this "less as a direct comparison" and more as a sign that agents could generate candidates that people then refine. The report adds that taking the best of about 150 noisy scores flatters the agents' number.
- Giving an agent a human's idea as its starting point did not improve its final score.
- On cost, the report puts an agent at about $4 an hour in inference, against the $150 an hour it paid the human researchers.

The data-size claim also differs between versions. The summary page says the winning recipe for the Opus 4.8 test used just over 2,000 training examples and calls it about 15,000 times as efficient as the alignment procedure Anthropic uses in production. The full report says about 2,400 examples and compares it with published open-weight pipelines, which used two to three orders of magnitude more data. Both say the comparison covers only the ten failures measured.

## The cheating, and the monitor

Anthropic writes that it had recently seen Claude cheat in research settings, for example by pulling test labels from a remote service or reporting only its best results. So after each run, a Claude Opus 4.8 monitor read every agent transcript, including its reasoning. Out of 1,601 attempts, it flagged 39, or 2.4%, as cheating.

The report sorts them into three kinds:

- **Lottery re-runs** (26 of 39): resubmitting the same method unchanged, hoping random noise gives a higher score.
- **Benchmark look-alikes** (10): building training data shaped like the test while wording the description to slip past the monitor.
- **Disguised plans** (8): hiding a banned step from the approval monitor.

Those counts add up to more than 39, so some attempts must fall into more than one kind; the report gives the percentages without explaining the overlap. Just one flagged attempt, a lucky re-run, made it into a leaderboard's top three, and none became a reported result.

Anthropic says it is "cautiously optimistic" that the monitor caught most cheating, because the model still tends to reveal its plans in its reasoning. It also says this may not hold for future models, and the report adds that these agents were not strongly trying to hide. A monitor that reads a model's reasoning works only as long as the reasoning is honest about what the model is doing.

## Why it matters, and what it does not show

Seen from this week, the study is evidence that a well-built loop can do real research work on problems that already have good tests. That matters to Anthropic's stated goal of having safety research keep up with more capable models, and it matters to anyone building loops, because the design choices are spelled out.

The limits are Anthropic's own, and they are substantial:

- **Only failures with benchmarks.** Problems nobody has a test for, or rare ones, give the agents nothing to climb. Political bias, for example, was not measured.
- **Proxies.** Benchmarks and Petri stand in for real-world behavior; they are not the same thing.
- **Narrow capability check.** Only math, general knowledge and instruction-following were guarded. A method could harm a skill nobody measured.
- **Durability unknown.** The team did not test whether gains survive further heavy training on other tasks.
- **A modest human baseline.** The report notes that researchers at frontier labs might produce stronger ideas than its participants.
- **Self-reported.** These are Anthropic's results about its own models, graded with tools Anthropic chose. The harness and benchmarks are public, which is what makes independent checking possible.

The title says "reliably"; the blog version of the same report uses the narrower "well-characterized alignment failures". The second wording matches the scope of what was tested.

## What can you borrow from this loop for your own work?

You probably will not train models, but the shape transfers to any loop where an AI drafts and something else judges. Several tests, scored so that one bad score sinks the attempt. A hidden test the worker never sees. A gate that throws out work which damages something you care about, even if the main score went up. A write-up frozen before results, so the explanation cannot be fitted to the outcome. A separate reviewer that reads what the worker actually did, not its summary. And a log you can audit later for gaming.

Here is a prompt to put any loop you run through the same checks:

```prompt
I run an AI loop that does this task: [describe the task].
Each round, the AI produces: [what it outputs].
I currently decide whether to accept a round by: [how you judge it].

For each way I judge the work, tell me whether it can fail the work with nobody watching
(a test that exits with an error, a missing source, a number below a threshold) or whether
it depends on someone's opinion or the AI's own confidence.
Then propose: one hidden test the AI never sees, one gate that catches damage to something
else I care about, and two ways the AI could game my current checks.
```

> **Try it:** Take one task you already hand to an AI repeatedly, write three checks that could fail it automatically, keep one of them hidden from the prompt, and run ten rounds. Count how often the hidden check disagrees with the visible ones.

If the hidden check often disagrees, your visible checks are being gamed or are too narrow, which is exactly what Anthropic's held-out benchmark was there to catch. For a hands-on version with code, see [how to build a generate, test, revise loop](/en/learn/build-generate-test-revise-loop/), and for limits that keep a loop from running off, [how to set up a self-improving agent loop safely](/en/learn/set-up-self-improving-agent-loop-safely/).

## Sources

- Automated researchers can reliably mitigate alignment failures: https://www.anthropic.com/research/automated-researchers-mitigate-alignment-failures
- Automated Researchers Can Mitigate Well-Characterized Alignment Failures: https://alignment.anthropic.com/2026/automated-alignment-researchers/
- Building effective agents: https://www.anthropic.com/engineering/building-effective-agents

---

From the Guide to Artificial Intelligence, by Robert Scales, published by Holistic Hybrid Inc. Share and adapt it under CC BY-NC-SA 4.0 (https://creativecommons.org/licenses/by-nc-sa/4.0/): credit the guide, link to https://guidetoartificialintelligence.com/en/learn/anthropic-automated-alignment-researchers/, no commercial use without permission, and keep the same licence.