Short answer
Claude 2 is Anthropic's new AI model, released on July 11, 2023, with better scores in coding, math and reasoning than Claude 1.3, room for about 75,000 words in a single prompt, and longer answers. For the first time, anyone in the US or UK can chat with it for free in the public beta at claude.ai, while businesses use it through Anthropic's API at an unchanged price.
Follow-up question How do you get started with Claude 2 on claude.ai?
Yesterday, July 11, 2023, Anthropic announced Claude 2, the second generation of its AI assistant. Until now, most people met Claude through business partners or a waitlist. With this release, Anthropic opened a public website, claude.ai, where anyone in the United States or the United Kingdom can create an account and start chatting.
Anthropic describes Claude as something like a friendly, enthusiastic colleague you can instruct in plain language. Claude 2 keeps that personality and adds better coding, math and reasoning, longer answers, and the very large reading capacity Claude gained in May.
What Anthropic released
The release has two parts. The model, Claude 2, is available to businesses through Anthropic's API, the service programmers use to connect Claude to their own apps, at the same price as the previous version, Claude 1.3. TechCrunch puts that at roughly $0.0465 for every 1,000 words Claude generates, and notes that API access is still limited. The website, claude.ai, is a free open beta, a test version open to the public, running on Claude 2.
Anthropic shared several test results comparing Claude 2 with Claude 1.3:
| Test | What it measures | Claude 1.3 | Claude 2 |
|---|---|---|---|
| Bar exam, multiple-choice section | US legal knowledge | 73.0% | 76.5% |
| Codex HumanEval | Writing working Python code | 56.0% | 71.2% |
| GSM8k | Grade-school math word problems | 85.2% | 88.0% |
| GRE reading and writing | Graduate-school admission test | not given | Above the 90th percentile of applicants |
On the GRE's quantitative section, Anthropic says Claude 2 scores about the same as the median applicant. TechCrunch adds that Claude 2 can pass the multiple-choice part of the US Medical Licensing Exam.
How Claude 2 is different
More reading and writing room. A prompt can hold up to 100,000 tokens. A token is a piece of text, often part of a word, and 100,000 tokens is about 75,000 words, roughly the length of The Great Gatsby. That means you can hand Claude hundreds of pages of documentation, a long report or a book and ask questions about it. Claude 2 can also write longer pieces in one go, up to a few thousand tokens. TechCrunch gives the output ceiling as about 4,000 tokens, or roughly 3,000 words, and reports that the model could technically handle a 200,000-token window, which Anthropic is not offering at launch.
Newer training data. According to TechCrunch, Claude 2 learned from more recent material than Claude 1.3: web pages, licensed datasets and data users chose to share, up to early 2023, with about a tenth of it in languages other than English. Anthropic's Sandy Banerjee told the outlet that Claude 2 is a refined version of the earlier model rather than an entirely new design.
Tidier structured output. The model is better at producing correctly formatted data such as JSON, XML, YAML and Markdown, which matters to programmers who feed Claude's answers into other software.
What it does not do. Claude 2 cannot search the web, so its knowledge stops where its training data stops.
Safety: what improved and what didn't
Anthropic trains Claude with a method it calls constitutional AI: the model is given a written set of principles and uses them to judge the text it produces, for example to be helpful and avoid harmful content. On Anthropic's internal test of harmful prompts, Claude 2 gave harmless answers twice as well as Claude 1.3.
Anthropic is also clear about limits. No model is immune from jailbreaks, cleverly worded prompts designed to get around safety rules. TechCrunch, reading the technical report Anthropic released the same morning, notes that one jailbreak among 328 test prompts still produced a harmful answer. And Anthropic advises against relying on Claude in situations that affect a person's physical or mental health.
Why it matters this week
Claude 2 gives the public a second widely available, free chat assistant built by a major lab, with a different strength: reading very long material. Business partners are already using it. Jasper, a writing platform, and Sourcegraph, whose coding assistant Cody answers questions about large codebases, both describe the long context as a key advantage.
For readers, the practical news is choice. You can now compare answers from different assistants on the same task and pick the one that suits the job.
Try it: Take a long document you know well, such as a user manual, a lease or a report. Ask Claude 2 for a one-paragraph summary and three questions the document leaves unanswered. Then check each point against the text yourself.
How do you get started with Claude 2 on claude.ai?
If you live in the United States or the United Kingdom, go to claude.ai and create an account. Anthropic says it plans to bring Claude to more countries in the coming months. Once you are in, type a request in plain language, or attach a document and ask about it. Start with a clear instruction: who the answer is for, how long it should be, and what format you want.
Anthropic admits that getting good results from an assistant often takes a few attempts, and it publishes tips to help. A good habit is to ask Claude to quote the part of your document it used, so you can check it. Treat the beta as a capable helper for summarizing, organizing and drafting, keep sensitive personal information out of it, and verify anything that matters before you rely on it.
Sources
- Claude 2 — Anthropic, July 11, 2023
- Anthropic releases Claude 2, its second-gen AI chatbot — TechCrunch, July 11, 2023