HumanEval
View DetailsA benchmark of 164 hand-written programming problems used to evaluate functional code generation.
LanguageEnglishDifficultyEasyCoding and Software EngineeringInstitutionOpenAI
Benchmark Library
This page aggregates mainstream LLM evaluation benchmarks including AIME 2025, SWE Bench Verified, MMLU, GSM8K, HumanEval, and more. We provide a comprehensive reference platform for researchers and developers to understand model performance across various evaluation datasets.
All Benchmarks
207
All categories
19
All languages
6
Detailed evaluation results on benchmark leaderboards:View Benchmark Leaderboards
207 benchmarks matched. Recommended flow: review leaderboards, then compare models.
A benchmark of 164 hand-written programming problems used to evaluate functional code generation.
Mostly Basic Python Problems
A benchmark of 974 short Python programming problems used to evaluate code generation.
Software Engineering Bench
A software-engineering benchmark built from real Python repositories and GitHub issues to test whether models can resolve practical coding tasks.
Software Engineering Bench - Verified
A human-validated subset of SWE-bench curated to provide more accurate and representative software-engineering tasks.
Individual Contributor SWE-Lancer(Diamond)
A benchmark that evaluates whether models can complete realistic software-engineering tasks performed by individual contributors.
SWE Manager-Lancer(Diamond)
A benchmark that evaluates model performance on realistic software-engineering management tasks.
A continuously updated benchmark that uses difficult problems from competitive-programming platforms to evaluate coding ability while reducing data contamination.
A benchmark that uses competitive-programming problems to evaluate code generation and algorithmic reasoning.
Software Engineering Bench Pro - Public Dataset
A public benchmark for evaluating whether models can solve realistic, complex software-engineering tasks.
Software Engineering Bench Pro - Commercial Dataset
A commercial benchmark dataset for evaluating whether models can solve realistic, complex software-engineering tasks.
A multilingual software-engineering benchmark covering realistic repository tasks across nine programming languages.
DeepSWE: Long-Horizon Software Engineering Benchmark
A long-horizon software engineering benchmark from Datacurve with 113 original tasks across 91 active open-source repositories and five programming languages. It emphasizes multi-file changes, behavioral correctness, and reduced contamination risk.
CodeClash: Benchmarking Goal-Oriented Software Engineering
An open benchmark for goal-oriented software engineering. Models maintain and improve codebases through multi-round tournaments, with win rate and Elo measuring strategic planning, log analysis, code evolution, and long-term maintenance.
AA Coding Agent Index (historical, pre-v1.4) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?
SWE-Marathon is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Program Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
FrontierSWE is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
PostTrain Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
MLS Bench Lite
MLS Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Kimi Code Bench 2.0 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
SciCode is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
NL2Repo-Bench evaluates whether coding agents can build a complete, installable Python repository from a natural-language specification and an empty workspace. The 104 tasks are graded against upstream pytest suites.
GSO: Challenging Software Optimization Tasks for Evaluating SWE-Agents
GSO evaluates SWE agents on 102 real software optimization tasks spanning 10 codebases and five programming languages. OPT@1 is the share of tasks where one attempt passes correctness tests and reaches at least 95% of the human expert speedup.
WeirdML v2 evaluates end-to-end machine-learning problem solving on 17 unusual tasks. Models iteratively write code, inspect results, and improve predictions; the headline metric macro-averages each task's best test accuracy.
LMArena Text Arena - Coding
Text Arena (Coding), formerly WebDev Arena, ranks web-application generations using anonymous pairwise user preferences. Arena Scores are dynamic and should be read with their observation date and confidence interval.
CursorBench 3.2 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
FrontierCode 1.1 Extended is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
AI Productivity Index for Software Engineering
APEX-SWE is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Code Arena WebDev Leaderboard
Code Arena WebDev is a live human-preference leaderboard for front-end web development. Users compare generated web applications in battle mode, and the aggregated votes produce Elo-style model ratings for practical multi-step web development.
DSBench-Hard is DeepSeek's internal hard benchmark for complex coding-agent capabilities. DeepSeek has published model scores but has not disclosed the task count or full methodology.
HumanEval pass@10 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
HumanEval pass@100 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Mostly Basic Python Problems pass@10
MBPP pass@10 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Mostly Basic Python Problems pass@100
MBPP pass@100 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
FrontierCode 1.1 Main is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Internal Database Migration is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
SRE-Bench (single-attempt) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
SEC-Bench Pro is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
OpenAI research harness; not the Artificial Analysis Codex component.
Success within four attempts; not single-attempt success.
Codex; max; Coding Agent Index v1.4; public AA table rounded to integers
Codex max, AA Coding Agent Index v1.4 component/telemetry; snapshot 2026-09-05.
Codex max, AA Coding Agent Index v1.4 component/telemetry; snapshot 2026-09-05.
Codex max, AA Coding Agent Index v1.4 component/telemetry; snapshot 2026-09-05.
Codex max, AA Coding Agent Index v1.4 component/telemetry; snapshot 2026-09-05.
Codex max, AA Coding Agent Index v1.4 component/telemetry; snapshot 2026-09-05.
A benchmark for evaluating the agentic ability of models to complete practical tasks by using tools in a terminal environment.
Terminal Bench Hard 2.0
An upgraded terminal-environment benchmark for evaluating the tool-use and task-completion abilities of language-model agents.
A benchmark that evaluates whether language models can use tools to complete complex tasks in realistic environments.
OS World Verified
A verified successor to OSWorld for evaluating the ability of language-model agents to operate computers in realistic environments.
A benchmark developed by Stanford University and the Laude Institute to evaluate AI agents in realistic terminal environments.
MCP-Atlas: A Large-Scale Benchmark for Tool-Use Competency with Real MCP Servers
A large-scale Scale AI benchmark for tool use with real MCP servers. It covers 36 servers, 220 tools, and 1,000 natural-language tasks that test tool discovery, cross-server orchestration, multi-step calls, and answer synthesis.
Vals CyberBench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
OSWorld 2.0 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Toolathlon-Verified is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
MLE-Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
MCPMark-Verified is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
SaaS-Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
CyberGym: Evaluating AI Agents' Real-World Cybersecurity Capabilities at Scale
CyberGym is a large-scale cybersecurity evaluation framework for assessing AI agents on real-world vulnerability analysis tasks in reproducible execution environments.
Terminal-Bench 3.0 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ExploitGym — 2-Hour Budget
ExploitGym evaluates whether AI agents can turn real-world vulnerabilities into working exploits across userspace programs, V8, and the Linux kernel. This entry records the number of solved instances under the 2-hour timeout budget used in the GLM-5.3 release evaluation; the current benchmark release contains 869 instances.
ExploitGym — 6-Hour Budget
ExploitGym evaluates whether AI agents can turn real-world vulnerabilities into working exploits across userspace programs, V8, and the Linux kernel. This entry records the number of solved instances under the 6-hour timeout budget used in the GLM-5.3 release evaluation; the current benchmark release contains 869 instances.
ExploitBench evaluates how far AI agents progress through a deterministic exploit-capability ladder on 41 real V8 vulnerabilities, from reaching vulnerable code through exploit primitives to arbitrary code execution. The GLM-5.3 release evaluation averages capability coverage over all tasks and three revisions with up to 300 agent-environment interaction rounds.
BioMysteryBench contains 99 expert-authored bioinformatics research questions over messy real-world datasets. Agents may use a Linux terminal, bioinformatics software, Python and R, plus restricted access to canonical data sources; answers are graded on objective biological conclusions rather than a prescribed analysis path.
LABBench2: An Improved Benchmark for AI Systems Performing Biology Research
LABBench2 is a public benchmark of nearly 1,900 tasks that measure practical biology research capabilities across literature understanding, biological database access, molecular biology, protocol troubleshooting and experiment planning in more realistic settings than the original LAB-Bench.
BioMysteryBench — Human-solvable Subset
This BioMysteryBench subset contains the 76 tasks that at least one of up to five human domain experts solved correctly. It preserves the benchmark's real-world data analysis, terminal tools, bioinformatics software and restricted external data access conditions.
BioMysteryBench — Human-difficult Subset
This BioMysteryBench subset contains the 23 tasks that no member of the human-expert panel solved correctly. Tasks still have controlled objective answers and use real-world bioinformatics data, tools and restricted external data access.
ClawEval-MM is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
RecreationBench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
AndroidWorld is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Terminal-Bench 4.0 evaluates complete model-and-agent systems on 66 containerized professional computer tasks. The release gives every agent run an eight-hour timeout, fixes 19 tasks, and removes eight tasks affected by saturation, refusals, public solutions, or unresolved quality and platform issues. Each task is run five times, and the leaderboard reports resolution rate with a 95% confidence interval. Because version 4.0 changes both the task set and resource rules, its scores are not directly comparable with Terminal-Bench 3.0 or 2.x.
Terminal-Bench-Science 0.1 evaluates scientific research agents on 70 expert-authored terminal tasks spanning life, physical, Earth, mathematical, and engineering sciences. Each model-agent system is run three times per task. Scores therefore measure the named model and agent harness together, not an agent-independent model capability.
Berkeley Function Calling Leaderboard V4
BFCL-V4 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use
MCPMark is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ExploitGym (unlimited) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ExploitBench (Jun–Aug 2026) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Impossible ExploitGym is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Official release configuration; best across efforts. Separate from earlier xhigh reports.
v2026.08.08 offline subset, partial score; not online/full-set or strict success.
OpenAI footnote: fewer turn-limit hits; do not compare with 300-turn-limited run.
Massive Multitask Language Understanding
A multiple-choice benchmark spanning 57 subjects that evaluates the knowledge and reasoning capabilities of large language models.
Massive Multitask Language Understanding Professional Level
A more challenging, professional-level variant of MMLU designed to evaluate model understanding and reasoning in specialized domains.
Beyond the Imitation Game Benchmark
A broad benchmark with more than 200 tasks covering reasoning, language understanding, knowledge, and other model capabilities.
Chinese Evaluation Benchmark
A Chinese multiple-choice benchmark spanning humanities, social sciences, and STEM subjects to evaluate knowledge and reasoning in Chinese.
Big Bench Hard
A difficult subset of BIG-bench containing especially challenging tasks that test the limits of model capabilities.
Abstraction and Reasoning Corpus (ARC-AGI-1)
Introduced by François Chollet in 2019, ARC-AGI-1 uses few-shot grid transformation tasks to measure how efficiently a system can infer novel rules and generalize from minimal examples. The public corpus contains 400 training tasks and 400 public evaluation tasks, with additional semi-private and private evaluation sets.
Humanity's Last Exam
An exceptionally difficult multidisciplinary benchmark with questions at graduate level and beyond.
The highest-quality subset of GPQA, designed to evaluate expert-level knowledge and reasoning across difficult scientific domains.
A Graduate-Level Google-Proof Q&A Benchmark
A graduate-level, Google-proof question-answering benchmark designed to evaluate expert knowledge and rigorous reasoning.
ARC-AGI-2 is the second-generation abstract-reasoning benchmark for frontier reasoning systems. It emphasizes symbolic interpretation, compositional reasoning, contextual rule application, and computational efficiency. Its public data includes 1,000 training tasks and 120 public evaluation tasks, plus calibrated semi-private and private evaluation sets of 120 tasks each.
A contamination-resistant benchmark designed to provide frequently updated and objective evaluation of large language models.
ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
ARC-AGI-3 is the first interactive reasoning benchmark designed for human-comparable agent evaluation. Without natural-language instructions, agents must explore unfamiliar environments, infer goals on the fly, build and update world models, plan over long horizons, and learn from sparse feedback. A 100% score means completing every game as efficiently as humans.
AA Intelligence Index (historical versions) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Vals AI Index
Vals Index is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
CritPt is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam
HLE-Verified is a reliability-focused revision of Humanity's Last Exam. Its scored set contains 1,811 expert-level items: 668 verified original items and 1,143 revised-and-verified items, while uncertain original items are excluded from the main accuracy calculation.
AGIEval is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
GeneBench Pro is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
MedChemBench Internal is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
LifeSciBench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
HealthBench Professional (length-adjusted) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
OpenAI release measurement; not Epoch or Artificial Analysis measurements.
AA v4.2; max; mixed task-specific tools. Not comparable to v4.1.1.
ARC Prize Semi-Private; Provider Adapter with persisted reasoning and compaction; high; 99.95 is displayed precision.
With tools; OpenAI release table does not report Sol under the same conditions.
Massive Multi-discipline Multimodal Understanding and Reasoning
A college-level multimodal benchmark that evaluates cross-disciplinary reasoning and the application of expert knowledge to text-and-image problems.
Simple Visual Question Answering
A visual question-answering benchmark focused on the factual accuracy of short natural-language answers from multimodal models.
Document Visual Question Answering
A visual question-answering benchmark built around document images and document-understanding tasks.
MMMU-Pro is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
CharXiv (RQ)
CharXiv RQ is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
MathVision is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
BabyVision is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
ZeroBench is a difficult visual-reasoning benchmark with 100 manually curated main questions and 334 subquestions, spanning natural and synthetic images and both single-image and multi-image settings. Main-set results commonly report pass@1, pass@5, and pass^5. Vendor-reported release scores and benchmark-team official leaderboard runs should be interpreted separately with their evaluation settings.
WorldVQA ForceAnswer is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
OmniDocBench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
PerceptionBench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Video-MME with Subtitles
Video-MME (w. sub) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
MMVU is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Visual Physics Comprehension Test
VPCT uses 100 simulator-generated image problems to measure whether vision-language models can predict which bucket a ball will enter from the initial physical setup.
GeoBench - A Community World Country Accuracy
GeoBench evaluates image geolocation in GeoGuessr. This DataLearner entry fixes the benchmark identity to country accuracy on the 100-image A Community World set, keeping it separate from other maps and distance-based metrics.
GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents
GDP.pdf evaluates grounded multimodal reasoning over 100 real professional PDF workflows spanning ten domains. Tasks require locating evidence, interpreting tables, charts, forms and diagrams, cross-referencing documents, and avoiding unsupported claims; strict pass rate requires satisfying all task rubric criteria.
LVBench: An Extreme Long Video Understanding Benchmark
LVBench evaluates long-video understanding across extended videos such as television, sports and surveillance footage. It measures information extraction, temporal grounding, reasoning and summarization over videos whose duration can span hours.
Chartography is a vision-dependent agent evaluation reported in the DeepSeek-V4-Flash-Vision-Exp release. DeepSeek did not link a standalone benchmark page or disclose its task count or creating institution in that release.
Vision2Web is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ERQA is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
RealWorldQA is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ScreenSpot-Pro (no tools) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Grade School Math 8K
A benchmark of 8,500 grade-school mathematics word problems used to evaluate mathematical reasoning.
Mathematics Problem Solving
A challenging mathematics dataset covering algebra, calculus, geometry, probability, and other fields.
A representative set of 500 problems selected from the MATH dataset to evaluate advanced mathematical reasoning.
The 2024 American Invitational Mathematics Examination, consisting of 15 challenging problems completed in a three-hour session.
Problems from the 2025 American Invitational Mathematics Examination used to evaluate advanced mathematical reasoning.
International Olympiad in Informatics 2024
A 2024 International Olympiad in Informatics benchmark used to evaluate advanced algorithmic problem solving and code generation.
International Olympiad in Informatics 2025
A 2025 International Olympiad in Informatics benchmark used to evaluate advanced algorithmic problem solving and code generation.
FrontierMath (Tier 1-3)
A benchmark of hundreds of novel and exceptionally difficult problems written by mathematicians to measure the frontier of advanced mathematical reasoning.
The most difficult FrontierMath tier, featuring research-level mathematical problems intended to test frontier reasoning systems.
International Mathematics Olympiad-ProofBench
A benchmark for evaluating whether large language models can generate complete, logically rigorous proofs for Olympiad-level mathematics problems.
International Mathematics Olympiad-ProofBench Advanced
An advanced benchmark for evaluating complete and logically rigorous proofs generated for difficult Olympiad-level mathematics problems.
International Mathematical Olympiad-AnswerBench
A benchmark that tests answers to difficult International Mathematical Olympiad-level mathematics problems.
American Invitational Mathematics Examination 2026
A benchmark based on the 2026 American Invitational Mathematics Examination for evaluating advanced mathematical reasoning.
FrontierMath Tiers 1-3 (v2)
The FrontierMath v2 private Tiers 1-3 set contains 295 expert-written problems spanning advanced undergraduate mathematics through early-career research. Released on June 12, 2026, v2 corrected 123 Tiers 1-3 problems and removed five.
FrontierMath Tier 4 (v2)
FrontierMath Tier 4 v2 is a private expansion set of 43 exceptionally difficult research-level mathematics problems. Released on June 12, 2026, v2 corrected 12 Tier 4 problems and removed seven.
Harvard-MIT Mathematics Tournament - February 2026
HMMT Feb 2026 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
China College Entrance Examination (Gaokao) Mathematics 2026
Gaokao 2026 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
OpenAI release measurement, rounded to one decimal; independent Epoch measurements remain separate.
Aider-Polyglot Benchmark
A multilingual benchmark that evaluates whether models can follow instructions to make practical, correct edits to existing code.
τ²-Bench - Overall
An open benchmark for evaluating task execution and communication by language-model agents in dual-control human-agent environments.
A telecom-domain benchmark that evaluates task execution by language-model agents in human-agent collaboration scenarios.
An Artificial Analysis benchmark variant focused on difficult tool-use tasks in terminal environments.
GAIA: A Benchmark for General AI Assistants
A benchmark for general AI assistants introduced by researchers from Meta, Hugging Face, and partner institutions. Its 466 real-world questions require reasoning, multimodal understanding, web browsing, and tool use.
Agents' Last Exam evaluates AI agents on long-horizon, economically valuable professional workflows with verifiable outcomes across a broad set of industries.
JobBench
Job Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
APEX-Agents is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
τ³-Benchmark - Banking
τ³-Banking is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games
BALROG evaluates long-horizon planning, exploration, and spatial reasoning across BabyAI, Crafter, TextWorld, Baba Is AI, MiniHack, and NetHack. The headline metric averages environment completion progress.
METR Task-Completion Time Horizons v1.1
METR Time Horizons v1.1 estimates the human-expert task duration at which an AI agent succeeds 50% of the time on software, ML, and cybersecurity tasks. DataLearner stores minutes; this is not agent runtime, and METR cautions that estimates above 16 hours are unreliable with the current suite.
ApexBench is an agent evaluation reported in the DeepSeek-V4-Flash-Vision-Exp release and includes multimodal elements. DeepSeek reports Pass@1 but did not link a standalone benchmark page or disclose its task count or creating institution in that release.
CoWorkBench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
τ³-Bench - Overall
τ³-Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies
Workspace-Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions
VitaBench 2.0 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Gross Domestic Product Value - Artificial Analysis
A benchmark that measures performance on knowledge-work tasks drawn from industries that contribute heavily to United States GDP.
GDPval-AA v2 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
AutomationBench, created by Zapier, evaluates agents on realistic cross-application business workflows across sales, marketing, operations, support, finance, and HR. The public score covers 600 tasks across 47 simulated SaaS tools and uses assertion-based final-state grading. Zapier's verified leaderboard uses a separate held-out private set, so public and private scores must be distinguished.
Artificial Analysis Briefcase
AA-Briefcase is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
OfficeQA Pro
Office QA Pro is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
SpreadsheetBench 2 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
DECK-Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ResearchRubrics is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Harvey Lab - Artificial Analysis
Harvey Lab-AA is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Corporate Finance v2
CorpFin v2 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Finance Agent v2 is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Legal Research Bench is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Artificial Analysis AnalystAgent
AA-AnalystAgent is a private research-agent evaluation from Artificial Analysis with 80 questions across 14 domains. Each task is attempted five times, and the headline pass^5 score requires all five attempts to be correct. The Stirrup harness provides code execution, web fetch, image viewing, and answer-submission tools, so results include the named reasoning setting and internet-enabled tool environment.
BenchCAD is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Internal Design Tasks is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Internal Data Science Tasks is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Artificial Analysis - Long Context Reasoning
An Artificial Analysis benchmark that simulates knowledge workers such as analysts, researchers, and lawyers reasoning over large document collections.
LongBench v2: Deeper Understanding and Reasoning on Realistic Long-Context Multitasks
A realistic long-context understanding and reasoning benchmark with 503 multiple-choice questions spanning contexts from 8,000 to 2 million words. Tasks include single- and multi-document QA, long conversations, code repositories, and structured data.
Google DeepMind MRCR v2 8-needle - 128K Average
GDM-MRCR v2 (8-needle, 128K) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Google DeepMind MRCR v2 8-needle - 1M Pointwise
GDM-MRCR v2 (8-needle, 1M) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Fiction.liveBench tests long-context comprehension of fiction while varying the amount of intervening context. DataLearner uses the 16k-token score as the canonical comparison metric.
OpenAI MRCR v2 (8-needle, 256K–512K) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
OpenAI MRCR v2 (8-needle, 512K–1M) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
A benchmark of 817 questions that measures whether models produce truthful answers instead of repeating common misconceptions.
Computer-use safety is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Computer-use safety (with AutoReview) is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Circumvention is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
ExploitGym honeypot is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Hallucination rate is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Hella Swag
A benchmark of roughly 70,000 multiple-choice questions designed to evaluate commonsense reasoning.
AI2 Reasoning Challenge
A benchmark of 7,787 multiple-choice science questions used to evaluate scientific and commonsense reasoning.
SimpleBench is a 213-question English multiple-choice commonsense reasoning benchmark spanning spatio-temporal reasoning, social intelligence, and linguistic adversarial robustness. Each question is run five times and the reported score is average accuracy. Runs normally use temperature 0.7 and top-p 0.95, except for o-series models. A small official sample of nine participants produced the 83.7 non-specialist human baseline.
A benchmark that evaluates the ability of AI agents to locate difficult-to-find information through web research.
DeepSearchQA is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
Instruction Following Bench
An instruction-following benchmark with 58 verifiable single-turn tasks that test generalization to novel and complex constraints.
Instruction-Following Eval
IFEval is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.
A benchmark that evaluates the ability of a language model to serve as the control center for OpenClaw.
A benchmark that evaluates language models across five common categories of OpenClaw tasks.
Massive Text Embedding Benchmark
A comprehensive benchmark suite for evaluating text-embedding models across retrieval, classification, clustering, and related tasks.
A benchmark that uses multi-turn conversations to evaluate long-context understanding and reasoning.
Creative Writing v3
A benchmark that uses strong model judges to evaluate the quality of creative writing produced by language models.
OpenScore String Quartets is an AI benchmark used to evaluate model capabilities. Review its overview, metrics, official resources, and model leaderboard results on DataLearnerAI.