DeepSeek-V3 vs GPT-4o(2024-11-20) benchmark comparison

Verdict

DeepSeek-V3 leads on the shared-benchmark average

Ahead on 4 of 7 shared benchmarks, averaging 5.2 points higher

7
Comparable benchmarks: 4
Wins: 3
Losses: 0
Ties

Summarised only from the 7 percentage-scale benchmarks scored by every selected model; details are below.

“Best available” takes each model’s best recorded non-parallel result in the metric’s stated direction per benchmark, so it may combine modes into a virtual configuration that does not exist. Read it with the mode breakdown.

DeepSeek-V3

DeepSeek-AI

Release: 2024-12-26
Context length: 128K
Total parameters: 681B

Model profile

GPT-4o(2024-11-20)

OpenAI

Performance benchmarks

Benchmark-by-benchmark comparison. Changing the thinking mode or tool filters updates the chart and table below.

“Best available” picks the best non-parallel result in the metric’s stated direction separately for each benchmark. The resulting series can combine several reasoning levels and is not one reproducible runtime configuration. Choose a mode filter to compare like-for-like runs.

Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology

Average score by domain

Each axis is the mean percentage score of one benchmark domain. It is an average, not a capability rating.

View: Non-parallel mode average·5 domains

DeepSeek-V3

Relative edge: Agent能力评测 +30.2 / Relative gap: 常识问答 -13.9

GPT-4o(2024-11-20)

Relative edge: 常识问答 +13.9 / Relative gap: Agent能力评测 -30.2

Method: for each model and benchmark, all scores in the current mode scope are averaged (not the best score), then those benchmark scores are averaged within each domain. Only benchmarks scored on a 0-100 scale by at least two of the selected models count — Elo and rating-scale benchmarks such as Codeforces or Arena are excluded, because averaging a 1500 rating with an 85% accuracy produces a meaningless number. Missing values are not counted as zero, and the averages are unweighted, so domains with harder benchmarks read lower.

Filter: Best Available·2 models / series · 7 Benchmark

Loading chart…

Benchmark score table

Every model and runtime mode, benchmark by benchmark. Values are comparable along a row, not between different benchmarks.

7 benchmarks with comparable scores. Each model shows its best score; mode label is displayed below.

Benchmark	DeepSeek-V3	GPT-4o(2024-11-20)
MMLU Accuracy 综合评估	88.50Standard Mode	85.70Standard Mode
MMLU Pro Accuracy 综合评估	75.90Standard Mode	77.90Standard Mode
HumanEval Accuracy 编程与软件工程	89.00Standard Mode	90.20Standard Mode
FrontierMath Accuracy 数学推理	1.70Standard Mode	0.30Standard Mode
MATH Accuracy 数学推理	87.80Standard Mode	68.50Standard Mode
SimpleQA Accuracy 常识问答	24.90Standard Mode	38.80Standard Mode
Aider-Polyglot Accuracy Agent能力评测	48.40Standard Mode	18.20Standard Mode

API price comparison

Official list prices per model API, split by input and output. Unit: USD per 1M tokens.

Specs & licensing

Architecture, licensing and API modalities. "Not provided" means the field is missing from our database.

Features & specs	DeepSeek-V3DeepSeek-AI	GPT-4o(2024-11-20)OpenAI
Core specsRelease	2024-12-26	2024-11-20
Context length	128K	128K
Total parameters	681B	—
Architecture	Dense	Undisclosed
Availability & licensingCode availability	Available · MIT License	Not public
Weight availability	Available · DEEPSEEK LICENSE AGREEMENT	Not public
Use & commercial terms	免费商用授权	Official service only; subject to provider terms
Local deploymentWeight size	687.9 GB	Not provided
VRAM for weights	≈ 688 GB (weights only, excludes KV cache)	Not provided
Weights	Hugging Face	Not provided
Source repo	GitHub	Not provided
ResourcesPaper / report	Introducing DeepSeek-V3	Not provided
DataLearner blog	开源大模型的新里程碑：DeepSeek AI开源6510亿参数的DeepSeek V3模型，评测结果显著好于4050亿参数的Llama3.1 405B，比肩Sonnet 3.5的开源模型	Not provided