DataLearner logo
CL

Claude 3.5 Sonnet New

Chat modelSonnetClaude 3.5

Claude 3.5 Sonnet New

Release date: 2024-10-22Updated: 2024-11-27Views: 2,023
Live demoGitHubHugging FaceCompare
Parameters
No data
Context length
200K
Multilingual
Supported
Reasoning ability
No data

Claude 3.5 Sonnet New is an AI model published by Anthropic, released on 2024-10-22, for Chat model, and 200K context length, with a 93.70 score on HumanEval.

Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology

Claude 3.5 Sonnet New

Model basics

Reasoning traces
No data
Thinking modes
Thinking modes not supported
Context length
200K tokens
Max output length
No data
Model type
Chat model
Modality (in / out)
No data
Release date
2024-10-22
Model file size
No data
MoE architecture
No
Total params / Active params
No data / Not applicable
Knowledge cutoff
No data
Claude 3.5 Sonnet New

Open source & experience

Code license
Proprietary
Weights license
Proprietary
GitHub repo
N/A
Hugging Face
N/A
Live demo
N/A
Claude 3.5 Sonnet New

Official resources

Paper
DataLearnerAI blog
Claude 3.5 Sonnet New

API details

API speed
No data
💡Default unit: $/1M tokens. If vendors use other units, follow their published pricing.
Standard
TypeConditionInputOutput
Text-$3.00/ 1M$15.00/ 1M
Image-$3.00/ 1M

“—” means the modality is not billed in that direction, or the vendor has not published a price for it.

Claude 3.5 Sonnet New

Benchmark Results

Claude 3.5 Sonnet New currently shows benchmark results led by HumanEval (5 / 140, score 93.70), BBH (2 / 21, score 92.60), MMLU (19 / 124, score 88.30). This page also consolidates core specs, context limits, and API pricing so you can evaluate the model from benchmark results and deployment constraints together.

Thinking
Tool usage

General Knowledge

3 evaluations
Benchmark / mode
Score
Rank/total
BBH
Standard Mode
92.60
2 / 21
MMLU
Standard Mode
88.30
19 / 124
MMLU Pro
Standard Mode
78
73 / 134

Coding and Software Engineer

5 evaluations
Benchmark / mode
Score
Rank/total
HumanEval
Standard Mode
93.70
5 / 140
SWE-bench Verified
Standard Mode
49
99 / 116
WeirdML v2
Standard ModeTools
39.97
47 / 52
LiveCodeBench
Standard Mode
38.70
110 / 128
GSO
Standard ModeTools
4.60
17 / 21

Math and Reasoning

5 evaluations
Benchmark / mode
Score
Rank/total
MATH
Standard Mode
78.30
12 / 42
MATH-500
Standard Mode
78
43 / 45
AIME 2024
Standard Mode
16
59 / 62
FrontierMath
Standard Mode
2.10
47 / 60
0
72 / 80

General Evaluation

1 evaluations
Benchmark / mode
Score
Rank/total
GPQA Diamond
Standard Mode
65
217 / 274

Common Sense

1 evaluations
Benchmark / mode
Score
Rank/total
SimpleQA
Standard Mode
28.40
26 / 47

Common Sense Reasoning

1 evaluations
Benchmark / mode
Score
Rank/total
SimpleBench
Standard Mode
41.40
62 / 92

Agent Level Benchmark

3 evaluations
Benchmark / mode
Score
Rank/total
Aider-Polyglot
Standard Mode
51.60
32 / 59
BALROG
Standard ModeTools
32.60
10 / 12
METR Time Horizons v1.1
Standard ModeTools
29.58
20 / 22

Multimodal Understanding

2 evaluations
Benchmark / mode
Score
Rank/total
GeoBench ACW
Standard Mode
62
18 / 20
VPCT
Standard Mode
33
24 / 24
Claude 3.5 Sonnet New

Publisher

Claude 3.5 Sonnet New

Model Overview

Claude 3.5 Sonnet New is an AI model published by Anthropic, released on 2024-10-22, for Chat model, and 200K context length, with a 93.70 score on HumanEval.

DataLearner on WeChat

Follow DataLearner on WeChat for AI model updates and research notes.

DataLearner WeChat QR code