Qwen3.5-397B-A17B Benchmark Details

Qwen3.5-397B-A17B currently shows benchmark results led by MMLU Pro (10 / 126, score 87.80), Pinch Bench (3 / 37, score 89.10), IF Bench (3 / 29, score 76.50). 1 source link is attached for reference.

Benchmark Results

Qwen3.5-397B-A17B

Benchmark Results

General Knowledge

5 evaluations

Benchmark / mode

Score

Rank/total

C-Eval

Thinking Mode

3 / 9

GPQA Diamond

Thinking Mode

88.40

28 / 180

MMLU Pro

Thinking Mode

87.80

10 / 126

HLE

Thinking Mode

28.70

87 / 164

HLE

Thinking ModeToolsInternet

48.30

32 / 164

Coding and Software Engineer

4 evaluations

Benchmark / mode

Score

Rank/total

LiveCodeBench

Thinking Mode

83.60

20 / 120

SWE-bench Verified

Thinking ModeTools

76.40

30 / 109

SWE-bench Multilingual

Thinking Mode

69.30

18 / 21

SWE-Bench Pro - Public

Thinking Mode

50.90

33 / 47

Multimodal Understanding

1 evaluations

Benchmark / mode

Score

Rank/total

MMMU

Thinking Mode

4 / 28

Agent Level Benchmark

1 evaluations

Benchmark / mode

Score

Rank/total

τ²-Bench

Thinking ModeTools

86.70

7 / 40

Instruction Following

1 evaluations

Benchmark / mode

Score

Rank/total

IF Bench

Thinking Mode

76.50

3 / 29

AI Agent - Information Search

2 evaluations

Benchmark / mode

Score

Rank/total

BrowseComp

Thinking ModeToolsInternet

78.60

17 / 49

BrowseComp

Thinking ModeTools

25 / 49

AI Agent - Tool Usage

3 evaluations

Benchmark / mode

Score

Rank/total

OSWorld-Verified

Thinking ModeTools

62.20

15 / 20

Terminal Bench 2.0

Thinking ModeTools

52.50

29 / 46

Tool Decathlon

Thinking ModeTools

38.30

7 / 9

Math and Reasoning

2 evaluations

Benchmark / mode

Score

Rank/total

AIME 2026

Thinking Mode

91.30

12 / 15

IMO-AnswerBench

Thinking Mode

80.90

17 / 21

Long Context

2 evaluations

Benchmark / mode

Score

Rank/total

AA-LCR

Thinking Mode

68.70

6 / 14

LongBench v2

Standard Mode

63.20

2 / 11

Claw-style Agent Evaluation

1 evaluations

Benchmark / mode

Score

Rank/total

Pinch Bench

Thinking ModeTools

89.10

3 / 37

Compare with other models

Sources

qwen.aiqwen.ai