A benchmark of 974 short Python programming problems used to evaluate code generation.
Browse the latest scores, model modes, release dates, and parameter sizes for MBPP.
Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology
| Rank | Model | License | |||
|---|---|---|---|---|---|
 Llama3.1-405B Instruct Standard Mode | 88.60 | 2024-07-23 | 405B | Free Commercial | |
 Gemini 1.5 Pro Standard Mode | 87.80 | 2024-02-15 | Unknown | Closed | |
Llama3.3-70B-Instruct Standard Mode | 87.60 | 2024-12-06 | 70B | Free Commercial | |
4 |  GPT-4o mini Standard Mode | 87.20 | 2024-07-18 | Unknown | Closed |
5 | Llama3.1-70B-Instruct Standard Mode | 86.00 | 2024-07-23 | 70B | Free Commercial |
6 |  Claude 3.5 Haiku Standard Mode | 85.60 | 2024-10-22 | Unknown | Closed |
7 |  Qwen2.5-72B Standard Mode | 84.70 | 2024-09-18 | 72.7B | Free Commercial |
8 | Qwen2.5-32B Standard Mode | 84.00 | 2024-09-18 | 32B | Free Commercial |
9 |  Hunyuan-A13B-Instruct Standard Mode | 83.86 | 2025-06-27 | 80B | Free Commercial |
10 | Qwen2.5-Max Standard Mode | 80.60 | 2025-01-28 | Unknown | Closed |
11 |  Codestral 25.01 Standard Mode | 80.20 | 2025-01-13 | Unknown | Closed |
12 | Codestral Standard Mode | 78.20 | 2024-05-29 | 22B | Non-Commercial |
13 | Llama 4 Maverick Standard Mode | 77.60 | 2025-04-05 | 400B | Free Commercial |
14 | Qwen2.5-7B Standard Mode | 74.90 | 2024-09-18 | 7B | Free Commercial |
15 | Mistral-Small-3.1-24B-Instruct-2503 Standard Mode | 74.71 | 2025-03-17 | 24B | Free Commercial |
16 | Gemma 3 - 27B (IT) Standard Mode | 74.40 | 2025-03-12 | 27B | Free Commercial |
17 |  C4AI Aya Vision 32B Standard Mode | 70.43 | 2025-03-04 | 32B | Non-Commercial |
18 | Llama3.1-8B-Instruct Standard Mode | 69.40 | 2024-07-23 | 8B | Free Commercial |
19 | Llama 4 Scout Standard Mode | 67.80 | 2025-04-05 | 109B | Free Commercial |
20 |  Phi-4-mini-instruct (3.8B) Standard Mode | 65.30 | 2025-02-27 | 3.8B | Free Commercial |
21 |  Moonlight-16B-A3B-Instruct Standard Mode | 63.80 | 2025-02-23 | 16B | Free Commercial |
22 | Gemma2-27B Standard Mode | 62.60 | 2024-05-14 | 27B | Free Commercial |
23 |  Gemma 2 - 9B Standard Mode | 62.20 | 2024-06-27 | 9B | Free Commercial |
24 | Qwen2.5-3B Standard Mode | 57.10 | 2024-09-18 | 3B | Free Commercial |
25 | Llama3.1-8B Standard Mode | 53.90 | 2024-07-23 | 8B | Free Commercial |
26 | Mistral-7B-Instruct-v0.3 Standard Mode | 51.10 | 2024-05-22 | 7B | Free Commercial |
27 | Llama-3.2-3B Standard Mode | 48.70 | 2024-09-18 | 3.2B | Free Commercial |
28 |  Amazon Nova Pro Standard Mode | 0.00 | 2024-12-03 | Unknown | Closed |
