Gemma 3 - 27B (IT) Benchmark Details

Gemma 3 - 27B (IT) currently shows benchmark results led by GSM8K (6 / 68, score 95.90), MATH (6 / 42, score 89), HumanEval (32 / 140, score 87.80).

Benchmark Results

Gemma 3 - 27B (IT)

Benchmark Results

Knowledge Exams

2 evaluations

Benchmark / mode

Score

Rank/total

MMLU

Standard Mode

78.60

58 / 124

MMLU-Pro

Standard Mode

67.50

108 / 175

Mathematics

3 evaluations

Benchmark / mode

Score

Rank/total

GSM8K

Standard Mode

95.90

6 / 68

MATH

Standard Mode

6 / 42

AIME 2024

Standard Mode

25.30

57 / 61

Code Generation & Editing

2 evaluations

Benchmark / mode

Score

Rank/total

HumanEval

Standard Mode

87.80

32 / 140

MBPP

Standard Mode

74.40

30 / 95

Commonsense

1 evaluations

Benchmark / mode

Score

Rank/total

HellaSwag

Standard Mode

85.60

3 / 3

Language Understanding

1 evaluations

Benchmark / mode

Score

Rank/total

DROP

Standard Mode

77.20

8 / 9

General Reasoning

1 evaluations

Benchmark / mode

Score

Rank/total

BBH

Standard Mode

87.60

9 / 19

Scientific Reasoning

2 evaluations

Benchmark / mode

Score

Rank/total

GPQA Diamond

Standard Mode

42.40

238 / 253

GPQA

Standard Mode

36.83

16 / 17

Honesty & Factuality

1 evaluations

Benchmark / mode

Score

Rank/total

SimpleQA

Standard Mode

42 / 46

Algorithmic Coding

1 evaluations

Benchmark / mode

Score

Rank/total

LiveCodeBench

Standard Mode

29.70

121 / 125

Writing

1 evaluations

Benchmark / mode

Score

Rank/total

Creative Writing

Standard Mode

1262.90

76 / 106

Visual Understanding

2 evaluations

Benchmark / mode

Score

Rank/total

MMMU

Standard Mode

64.90

55 / 74

MMMU

unknown

64.90

55 / 74

Agentic Development

1 evaluations

Benchmark / mode

Score

Rank/total

Aider-Polyglot

Standard Mode

4.90

58 / 59

Compare with other models