NL2Repo-Bench evaluates whether coding agents can build a complete, installable Python repository from a natural-language specification and an empty workspace. The 104 tasks are graded against upstream pytest suites.
Browse the latest scores, model modes, release dates, and parameter sizes for NL2Repo-Bench.
Data sourced primarily from official releases (GitHub, Hugging Face, papers), then benchmark leaderboards, then third-party evaluators. Learn about our data methodology
| Rank | Model | License | |||
|---|---|---|---|---|---|
 DeepSeek-V4-Flash Thinking Level · MaxTools | 54.20 | 2026-04-24 | 284B | Free Commercial |
