HumanEval pass@100 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Browse the latest scores, model modes, release dates, and parameter sizes for HumanEval pass@100.
Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology
| Rank | Model | License | |||
|---|---|---|---|---|---|
 CodeLLaMA-Python-34B Standard Mode | 94.70 | 2023-08-24 | 34B | Free Commercial | |
CodeLLaMA-Python-13B Standard Mode | 94.10 | 2023-08-24 | 13B | Free Commercial | |
CodeLLaMA-Instruct-34B Standard Mode | 93.50 | 2023-08-24 | 34B | Free Commercial | |
4 | CodeLLaMA-34B Standard Mode | 93.00 | 2023-08-24 | 34B | Free Commercial |
5 |  PanGu-Coder2 Standard Mode | 91.76 | 2023-07-27 | 15B | Closed |
6 | CodeLLaMA-Instruct-13B Standard Mode | 91.60 | 2023-08-24 | 13B | Free Commercial |
7 | CodeLLaMA-Python-7B Standard Mode | 90.60 | 2023-08-24 | 7B | Free Commercial |
8 |  WizardCoder-15B-V1.0 Standard Mode | 90.46 | 2023-06-14 | 15B | Free Commercial |
9 | CodeLLaMA-13B Standard Mode | 89.80 | 2023-08-24 | 13B | Free Commercial |
10 |  PaLM Standard Mode | 88.40 | 2022-04-05 | 540B | Closed |
11 | PaLM-Coder Standard Mode | 88.40 | 2022-04-05 | 540B | Closed |
12 | PaLM2-S Standard Mode | 88.40 | 2023-05-10 | Unknown | Closed |
13 |  CodeGeeX2-6B Standard Mode | 88.30 | 2023-07-25 | 6B | Commercial |
14 | CodeLLaMA-Instruct-7B Standard Mode | 88.10 | 2023-08-24 | 7B | Free Commercial |
15 | LLaMA2 70B Standard Mode | 87.00 | 2023-07-18 | 70B | Free Commercial |
16 | CodeLLaMA-7B Standard Mode | 85.90 | 2023-08-24 | 7B | Free Commercial |
17 | LLaMA2 34B Standard Mode | 79.50 | 2023-07-18 | 34B | Free Commercial |
18 |  Codex Standard Mode | 72.31 | 2021-08-10 | 175B | Closed |
19 | LLaMA2 13B Standard Mode | 61.20 | 2023-07-18 | 13B | Free Commercial |
20 | CodeGeeX Standard Mode | 60.92 | 2022-09-30 | 13B | Closed |
21 | LLaMA2 7B Standard Mode | 44.40 | 2023-07-18 | 7B | Free Commercial |
