Gemma2-27B Benchmark Details

Gemma2-27B currently shows benchmark results led by BBH (12 / 21, score 74.90), HellaSwag (2 / 3, score 86.40), GSM8K (19 / 26, score 74).

Benchmark Results

Gemma2-27B

Benchmark Results

General Knowledge

3 evaluations

Benchmark / mode

Score

Rank/total

MMLU

Standard Mode

75.20

55 / 66

BBH

Standard Mode

74.90

12 / 21

MMLU Pro

56.54

115 / 132

Math and Reasoning

2 evaluations

Benchmark / mode

Score

Rank/total

GSM8K

Standard Mode

19 / 26

MATH

Standard Mode

42.30

38 / 42

Coding and Software Engineer

2 evaluations

Benchmark / mode

Score

Rank/total

MBPP

Standard Mode

62.60

22 / 28

HumanEval

Standard Mode

51.80

32 / 39

Common Sense Reasoning

1 evaluations

Benchmark / mode

Score

Rank/total

HellaSwag

Standard Mode

86.40

2 / 3

Reading Comprehension

1 evaluations

Benchmark / mode

Score

Rank/total

DROP

Standard Mode

74.20

9 / 9

Compare with other models