加载中...
加载中...
本页面提供当前主流大模型在代码能力上的评测结果，包括HumanEval和MBPP等基准数据集。
Data source: 论文或GitHub评测结果
| Model | Parameters | HumanEval Pass@1 | MBPP Pass@1 | Organization | License |
|---|---|---|---|---|---|
| Qwen2.5-Omni-7B | 70.0 | 84.80 | 79.20 | 阿里巴巴 | / |
| CodeQwen1.5-7B-Chat | 70.0 | 83.50 | 77.70 | 阿里巴巴 | / |
| Llama3.1-8B-Instruct | 80.0 | 72.60 | 72.80 | Facebook AI研究实验室 |
Data is for reference only. Official sources are authoritative. Click model names to view DataLearner model profiles.
| GLM-4-9B-Chat | 90.0 | 71.80 | / | 智谱AI | / |
| GLM-4-9B | 90.0 | 70.10 | / | 智谱AI | / |
| DeepSeek Coder-6.7B Instruct | 67.0 | 66.10 | 65.40 | DeepSeek-AI | / |
| Llama3-8B | 80.0 | 62.20 | / | Facebook AI研究实验室 | / |
| Llama3-8B-Instruct | 80.0 | 62.20 | / | Facebook AI研究实验室 | / |
| Phi-3-small 7B | 70.0 | 59.10 | 71.40 | Microsoft Azure | / |
| Qwen2.5-7B | 70.0 | 57.90 | 74.90 | 阿里巴巴 | / |
| CodeGemma-7B-IT | 70.0 | 56.10 | 54.20 | Google Research | / |
| CodeQwen1.5-7B | 70.0 | 51.80 | 72.20 | 阿里巴巴 | / |
| Qwen2-7B | 70.0 | 51.20 | 65.90 | 阿里巴巴 | / |
| CodeGemma-7B | 70.0 | 44.50 | 56.20 | Google Research | / |
| Gemma 2 - 9B | 90.0 | 40.20 | 52.40 | Google Research | / |
| CodeLLaMA-Python-7B | 70.0 | 38.40 | 47.60 | Facebook AI研究实验室 | / |
| PaLM2-S | 0.0 | 37.60 | 50 | Google Research | / |
| CodeGeeX2-6B | 60.0 | 35.90 | / | 智谱AI | / |
| CodeLLaMA-Instruct-7B | 70.0 | 34.80 | 44.40 | Facebook AI研究实验室 | / |
| WizardCoder-3B-V1.0 | 30.0 | 34.80 | 37.40 | WizardLM Team | / |
| CodeLLaMA-7B | 70.0 | 33.50 | 41.40 | Facebook AI研究实验室 | / |
| Gemma 7B | 70.0 | 32.30 | 44.40 | Google Research | / |
| Mistral 7B | 73.0 | 30.50 | 47.50 | MistralAI | / |
| Qwen-7B | 70.0 | 29.90 | 31.60 | 阿里巴巴 | / |
| AquilaCode-7B-py | 70.0 | 28.80 | / | 北京智源人工智能研究院 | / |
| WizardCoder-1B-V1.0 | 10.0 | 23.80 | 28.60 | WizardLM Team | / |
| AquilaCode-7B-multi | 70.0 | 22 | / | 北京智源人工智能研究院 | / |
| Baichuan2-7B-Base | 70.0 | 18.29 | 24.20 | 百川智能 | / |
| LLaMA2 7B | 70.0 | 12.20 | 20.80 | Facebook AI研究实验室 | / |
| Baichuan 7B | 70.0 | 9.20 | 6.60 | 百川智能 | / |