OpenAI o1 Benchmark Details

OpenAI o1 currently shows benchmark results led by MMLU Pro (1 / 126, score 91.04), MATH (2 / 42, score 96.40), MMLU (4 / 65, score 91.80). 1 source link is attached for reference.

Benchmark Results

OpenAI o1

Benchmark Results

综合评估

4 evaluations

Benchmark / mode

Score

Rank/total

MMLU

Standard Mode

91.80

4 / 65

MMLU Pro

Standard Mode

91.04

1 / 126

GPQA Diamond

Standard Mode

77.30

85 / 177

HLE

Standard Mode

9.10

134 / 156

数学推理

4 evaluations

Benchmark / mode

Score

Rank/total

MATH

Standard Mode

96.40

2 / 42

MATH-500

Standard Mode

96.40

17 / 44

AIME 2024

Standard Mode

79.20

31 / 62

FrontierMath

Thinking Level · High

9.30

30 / 60

常识问答

1 evaluations

Benchmark / mode

Score

Rank/total

SimpleQA

Standard Mode

42.60

17 / 45

编程与软件工程

3 evaluations

Benchmark / mode

Score

Rank/total

LiveCodeBench

Standard Mode

47 / 120

SWE-bench Verified

Standard Mode

48.90

92 / 106

SWE-bench Verified

Thinking Level · High

95 / 106

常识推理

1 evaluations

Benchmark / mode

Score

Rank/total

Simple Bench

Thinking Level · High

40.10

18 / 27

Compare with other models

Sources

openai.comopenai.com