Claude 3.5 Haiku Benchmark Details

Claude 3.5 Haiku currently shows benchmark results led by MBPP (6 / 69, score 85.60), HumanEval (17 / 101, score 88.10), MMLU (61 / 124, score 77.60).

Benchmark Results

Claude 3.5 Haiku

Benchmark Results

Knowledge Exams

2 evaluations

Benchmark / mode

Score

Rank/total

MMLU

Standard Mode

77.60

61 / 124

MMLU-Pro

Standard Mode

114 / 175

Code Generation & Editing

2 evaluations

Benchmark / mode

Score

Rank/total

HumanEval

Standard Mode

88.10

17 / 101

MBPP

Standard Mode

85.60

6 / 69

Mathematics

2 evaluations

Benchmark / mode

Score

Rank/total

MATH

Standard Mode

69.20

23 / 42

FrontierMath

Standard Mode

0.30

57 / 60

Scientific Reasoning

2 evaluations

Benchmark / mode

Score

Rank/total

GPQA Diamond

Standard Mode

41.60

239 / 253

GPQA

Standard Mode

37.50

15 / 17

Honesty & Factuality

1 evaluations

Benchmark / mode

Score

Rank/total

SimpleQA

Standard Mode

8.02

44 / 46

Writing

1 evaluations

Benchmark / mode

Score

Rank/total

Creative Writing

Standard Mode

1145.70

86 / 110

Agentic Development

2 evaluations

Benchmark / mode

Score

Rank/total

Aider-Polyglot

Standard Mode

44 / 59

Terminal Bench Hard

Standard ModeTools

2.30

229 / 244

Visual Understanding

1 evaluations

Benchmark / mode

Score

Rank/total

MMMU-Pro

Standard Mode

45.60

212 / 229

Capability Indices

1 evaluations

Benchmark / mode

Score

Rank/total

ECI

unknown

127.15

121 / 167

Compare with other models