DataLearner logo

OpenAI o1 Benchmark Details

OpenAI o1 currently shows benchmark results led by MMLU Pro (1 / 132, score 91.04), MATH (2 / 42, score 96.40), MMLU (4 / 66, score 91.80). 1 source link is attached for reference.

Benchmark Results

OpenAI o1

Benchmark Results

Thinking

General Knowledge

4 evaluations
Benchmark / mode
Score
Rank/total
MMLU
91.80
4 / 66
MMLU Pro
91.04
1 / 132
GPQA Diamond
77.30
92 / 187
HLE
9.10
148 / 170

Math and Reasoning

4 evaluations
Benchmark / mode
Score
Rank/total
MATH
96.40
2 / 42
MATH-500
96.40
17 / 44
AIME 2024
79.20
31 / 62
FrontierMath
9.30
30 / 60

Common Sense

1 evaluations
Benchmark / mode
Score
Rank/total
SimpleQA
42.60
19 / 47

Coding and Software Engineer

3 evaluations
Benchmark / mode
Score
Rank/total
LiveCodeBench
71
48 / 123
SWE-bench Verified
48.90
97 / 111
SWE-bench Verified
Thinking Level · High
41
100 / 111

Common Sense Reasoning

2 evaluations
Benchmark / mode
Score
Rank/total
Simple Bench
Thinking Level · Medium
36.70
43 / 63
Simple Bench
Thinking Level · High
40.10
39 / 63

Agent Level Benchmark

1 evaluations
Benchmark / mode
Score
Rank/total
Aider-Polyglot
Thinking Level · High
61.70
19 / 59
Compare with other models

Sources

openai.comopenai.com