A benchmark of 164 hand-written programming problems used to evaluate functional code generation.
Browse the latest scores, model modes, release dates, and parameter sizes for HumanEval.
Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology
| Rank | Model | License | |||
|---|---|---|---|---|---|
 OpenAI o3-mini (high) Standard Mode | 97.60 | 2025-01-31 | Unknown | Closed | |
 Claude Opus 4.6 Extended Thinking | 95.00 | 2026-02-05 | Unknown | Closed | |
Claude 3.5 Sonnet New Standard Mode | 93.70 | 2024-10-22 | Unknown | Closed | |
4 | OpenAI o1-mini Standard Mode | 92.40 | 2024-09-12 | Unknown | Closed |
5 | Claude 3.5 Sonnet Standard Mode | 92.00 | 2024-06-21 | Unknown | Closed |
6 |  Hunyuan-TurboS Standard Mode | 91.00 | 2025-03-10 | Unknown | Closed |
7 | GPT-4o(2024-11-20) Standard Mode | 90.20 | 2024-11-20 | Unknown | Closed |
8 | GPT-4o Standard Mode | 90.00 | 2024-05-13 | Unknown | Closed |
9 |  Gemini 1.5 Pro Standard Mode | 89.00 | 2024-02-15 | Unknown | Closed |
10 |  Llama3.1-405B Instruct Standard Mode | 89.00 | 2024-07-23 | 405B | Free Commercial |
11 |  Amazon Nova Pro Standard Mode | 89.00 | 2024-12-03 | Unknown | Closed |
12 |  DeepSeek-V3 Standard Mode | 89.00 | 2024-12-26 | 681B | Free Commercial |
13 |  Mistral-Small-3.1-24B-Instruct-2503 Standard Mode | 88.41 | 2025-03-17 | 24B | Free Commercial |
14 |  Qwen2.5-32B Standard Mode | 88.40 | 2024-09-18 | 32B | Free Commercial |
15 | Llama3.3-70B-Instruct Standard Mode | 88.40 | 2024-12-06 | 70B | Free Commercial |
16 |  Grok 2 Standard Mode | 88.40 | 2024-08-13 | 269B | Free Commercial |
17 | Claude 3.5 Haiku Standard Mode | 88.10 | 2024-10-22 | Unknown | Closed |
18 | Gemma 3 - 27B (IT) Standard Mode | 87.80 | 2025-03-12 | 27B | Free Commercial |
19 | GPT-4o mini Standard Mode | 87.20 | 2024-07-18 | Unknown | Closed |
20 | Codestral 25.01 Standard Mode | 86.60 | 2025-01-13 | Unknown | Closed |
21 | Claude3-Opus Standard Mode | 84.90 | 2024-03-04 | Unknown | Closed |
22 | Codestral Standard Mode | 81.10 | 2024-05-29 | 22B | Non-Commercial |
23 | Llama3.1-70B-Instruct Standard Mode | 80.50 | 2024-07-23 | 70B | Free Commercial |
24 |  Phi-4-mini-instruct (3.8B) Standard Mode | 74.40 | 2025-02-27 | 3.8B | Free Commercial |
25 | Grok-1.5 Standard Mode | 74.10 | 2024-03-29 | Unknown | Closed |
26 | Qwen2.5-Max Standard Mode | 73.20 | 2025-01-28 | Unknown | Closed |
27 | GPT-4 Standard Mode | 67.00 | 2023-03-14 | 175B | Closed |
28 | Llama3.1-8B-Instruct Standard Mode | 66.50 | 2024-07-23 | 8B | Free Commercial |
29 |  C4AI Aya Vision 32B Standard Mode | 62.20 | 2025-03-04 | 32B | Non-Commercial |
30 | Qwen2.5-72B Standard Mode | 59.10 | 2024-09-18 | 72.7B | Free Commercial |
