Qwen3.5-397B-A17B Benchmark Details

Qwen3.5-397B-A17B currently shows benchmark results led by C-Eval (3 / 48, score 93), MMLU Pro (10 / 133, score 87.80), Pinch Bench (4 / 38, score 89.10). 1 source link is attached for reference.

Benchmark Results

Qwen3.5-397B-A17B

Benchmark Results

General Knowledge

4 evaluations

Benchmark / mode

Score

Rank/total

C-Eval

Thinking Mode

3 / 48

MMLU Pro

Thinking Mode

87.80

10 / 133

HLE

Thinking Mode

28.70

100 / 181

HLE

Thinking ModeToolsInternet

48.30

37 / 181

Other

1 evaluations

Benchmark / mode

Score

Rank/total

GPQA Diamond

Thinking Mode

88.40

52 / 224

Coding and Software Engineer

4 evaluations

Benchmark / mode

Score

Rank/total

LiveCodeBench

Thinking Mode

83.60

22 / 126

SWE-bench Verified

Thinking ModeTools

76.40

33 / 114

SWE-bench Multilingual

Thinking Mode

69.30

21 / 25

SWE-Bench Pro - Public

Thinking Mode

50.90

42 / 57

Writing and Creative Capabilities

1 evaluations

Benchmark / mode

Score

Rank/total

Creative Writing

Standard Mode

1475.80

50 / 99

Multimodal Understanding

1 evaluations

Benchmark / mode

Score

Rank/total

MMMU

Thinking Mode

4 / 28

Agent Level Benchmark

1 evaluations

Benchmark / mode

Score

Rank/total

τ²-Bench

Thinking ModeTools

86.70

7 / 43

Instruction Following

1 evaluations

Benchmark / mode

Score

Rank/total

IF Bench

Thinking Mode

76.50

6 / 33

AI Agent - Information Search

2 evaluations

Benchmark / mode

Score

Rank/total

BrowseComp

Thinking ModeToolsInternet

78.60

19 / 54

BrowseComp

Thinking ModeTools

29 / 54

AI Agent - Tool Usage

3 evaluations

Benchmark / mode

Score

Rank/total

OSWorld-Verified

Thinking ModeTools

62.20

21 / 26

Terminal Bench 2.0

Thinking ModeTools

52.50

31 / 48

Tool Decathlon

Thinking ModeTools

38.30

8 / 10

Math and Reasoning

2 evaluations

Benchmark / mode

Score

Rank/total

AIME 2026

Thinking Mode

91.30

14 / 19

IMO-AnswerBench

Thinking Mode

80.90

19 / 23

Long Context

2 evaluations

Benchmark / mode

Score

Rank/total

AA-LCR

Thinking Mode

68.70

8 / 18

LongBench v2

Standard Mode

63.20

3 / 13

Claw-style Agent Evaluation

1 evaluations

Benchmark / mode

Score

Rank/total

Pinch Bench

Thinking ModeTools

89.10

4 / 38

Compare with other models

Sources

qwen.aiqwen.ai