AutomationBench, created by Zapier, evaluates agents on realistic cross-application business workflows across sales, marketing, operations, support, finance, and HR. The public score covers 600 tasks across 47 simulated SaaS tools and uses assertion-based final-state grading. Zapier's verified leaderboard uses a separate held-out private set, so public and private scores must be distinguished.
Browse the latest scores, model modes, release dates, and parameter sizes for AutomationBench.
Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology
| Rank | Model | License | |||
|---|---|---|---|---|---|
— |  DeepSeek-V4.1-Flash Thinking Level · MaxTools | 54.80 | 2026-09-10 | 552B | Free Commercial |
— |  Qwen3.8-Max-0902 Thinking Level · Extra HighTools | 50.80 | 2026-09-02 | 2400B | Closed |
— |  Claude Opus 5 Thinking Level · MaxTools | 50.30 | 2026-07-24 | Unknown | Closed |
— |  Muse Spark 1.3 Thinking Level · MaxTools | 49.40 | 2026-09-02 | Unknown | Closed |
— |  GLM-5.3 Thinking Level · MaxTools | 48.80 | 2026-08-14 | 744B | Conditional |
— | GLM-5.3-Flash Thinking Level · MaxTools | 48.80 | 2026-08-26 | 320B | Free Commercial |
— |  Kimi K3 Thinking Level · MaxTools | 46.70 | 2026-07-16 | 2800B | Conditional |
— |  GPT-5.6 Sol Thinking Level · MaxTools | 45.80 | 2026-06-26 | Unknown | Closed |
— | GPT-6 Astra Thinking Level · MaxTools | 41.40 | 2026-09-03 | Unknown | Closed |
— | DeepSeek-V4-Flash Thinking Level · MaxTools | 37.70 | 2026-04-24 | 284B | Free Commercial |
— |  Hy4 preview Thinking Level · HighTools | 32.10 | 2026-08-28 | 770B | Free Commercial |
— | DeepSeek-V4-Pro Thinking Level · Extra HighTools | 31.80 | 2026-08-13 | 1600B | Free Commercial |
— | Claude Fable 5.1 Thinking Level · MaxTools | 31.40 | 2026-09-01 | Unknown | Closed |
— |  Gemini 3.7 Flash Thinking EnabledTools | 30.40 | 2026-08-13 | Unknown | Closed |
— | Qwen3.8-Max Thinking Level · Extra HighTools | 27.30 | 2026-08-03 | 2400B | Conditional |
— | DeepSeek-V4-Flash-Vision-Exp Thinking Level · MaxTools | 25.70 | 2026-08-21 | 305B | Free Commercial |
— | GPT-5.6 Sol unknown | 18.10 | 2026-06-26 | Unknown | Closed |
DeepSeek reported 25.7 on AutomationBench (Public) for DeepSeek-V4-Flash-Vision-Exp on August 21, 2026.
