DataLearner logo

Gemma 3 - 27B (IT) Benchmark Details

Gemma 3 - 27B (IT) currently shows benchmark results led by MATH (6 / 42, score 89), GSM8K (5 / 26, score 95.90), BBH (9 / 20, score 87.60).

Benchmark Results

Gemma 3 - 27B (IT)

Benchmark Results

Thinking

General Knowledge

5 evaluations
Benchmark / mode
Score
Rank/total
BBH
Standard Mode
87.60
9 / 20
MMLU
Standard Mode
78.60
52 / 65
MMLU Pro
Standard Mode
67.50
96 / 126
GPQA Diamond
Standard Mode
42.40
162 / 179
GPQA
36.83
13 / 14

Math and Reasoning

3 evaluations
Benchmark / mode
Score
Rank/total
GSM8K
Standard Mode
95.90
5 / 26
MATH
Standard Mode
89
6 / 42
AIME 2024
25.30
57 / 62

Coding and Software Engineer

3 evaluations
Benchmark / mode
Score
Rank/total
HumanEval
Standard Mode
87.80
18 / 39
MBPP
Standard Mode
74.40
16 / 28
LiveCodeBench
Standard Mode
29.70
116 / 120

常识推理

1 evaluations
Benchmark / mode
Score
Rank/total
HellaSwag
Standard Mode
85.60
3 / 3

阅读理解

1 evaluations
Benchmark / mode
Score
Rank/total
DROP
Standard Mode
77.20
8 / 9

Common Sense

1 evaluations
Benchmark / mode
Score
Rank/total
SimpleQA
Standard Mode
10
40 / 45

Multimodal Understanding

1 evaluations
Benchmark / mode
Score
Rank/total
MMMU
Standard Mode
64.90
27 / 28

Agent Level Benchmark

1 evaluations
Benchmark / mode
Score
Rank/total
Aider-Polyglot
Standard Mode
4.90
58 / 59
Compare with other models