Gemma2-27B Benchmark Details

Gemma2-27B currently shows benchmark results led by MBPP (24 / 70, score 62.60), GSM8K (28 / 70, score 74), HumanEval (49 / 101, score 51.80).

Benchmark Results

Gemma2-27B

Benchmark Results

General Knowledge

3 evaluations

Benchmark / mode

Score

Rank/total

MMLU

Standard Mode

75.20

66 / 124

BBH

Standard Mode

74.90

12 / 21

MMLU Pro

Standard Mode

56.54

116 / 133

Math and Reasoning

2 evaluations

Benchmark / mode

Score

Rank/total

GSM8K

Standard Mode

28 / 70

MATH

Standard Mode

42.30

38 / 42

Coding and Software Engineer

2 evaluations

Benchmark / mode

Score

Rank/total

MBPP

Standard Mode

62.60

24 / 70

HumanEval

Standard Mode

51.80

49 / 101

Common Sense Reasoning

1 evaluations

Benchmark / mode

Score

Rank/total

HellaSwag

Standard Mode

86.40

2 / 3

Reading Comprehension

1 evaluations

Benchmark / mode

Score

Rank/total

DROP

Standard Mode

74.20

9 / 9

Compare with other models