The 51 most challenging tasks within the 642-task SWE-Bench Pro V2 public split. It targets complex multi-file dependencies, system bugs and long-horizon repository engineering.
Browse the latest scores, model modes, release dates, and parameter sizes for SWE-Bench Pro V2 Hard.
Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology
| Rank | Model | License | |||
|---|---|---|---|---|---|
— |  Claude Opus 5 Thinking Level · Extra HighTools | 98.00 | 2026-07-24 | Unknown | Closed |
— | Claude Fable 5.1 Thinking Level · HighTools | 92.20 | 2026-09-01 | Unknown | Closed |
— |  GPT-6 Astra Thinking Level · HighTools | 90.20 | 2026-09-03 | Unknown | Closed |
— | Claude Sonnet 5 Thinking Level · Extra HighTools | 88.20 | 2026-06-30 | Unknown | Closed |
— |  Kimi K3 Thinking Level · MaxTools | 88.20 | 2026-07-16 | 2800B | Conditional |
— | GPT-5.6 Terra Thinking Level · Extra HighTools | 86.30 | 2026-06-26 | Unknown | Closed |
— |  GLM-5.3 Thinking Level · MaxTools | 84.30 | 2026-08-14 | 744B | Conditional |
— | GPT-5.6 Sol Thinking Level · Extra HighTools | 82.40 | 2026-06-26 | Unknown | Closed |
— |  Gemini 3.8 Flash Thinking Level · HighTools | 58.80 | 2026-09-02 | Unknown | Closed |
— | Inkling Thinking Level · Extra HighTools | 56.90 | 2026-07-15 | 975B | Free Commercial |
— | Haiku 4.5 Thinking Level · Extra HighTools | 25.50 | 2025-10-15 | Unknown | Closed |
Official Scale Labs results, September 22, 2026. Scores use pass@1 resolve rate under a network-locked agent protocol. Agent harnesses differ; public task exposure may affect generalization claims.
