Skip to content

Qwen4 Max Scores 9 Points Above Qwen3.8 Max on Intelligence Index

Qwen · XRelayed· @Alibaba_Qwen·· 21 hours agoPicks

10.6K views · 142 likes · 3 reposts (at collection)

Original
Who
Artificial Analysis
What
published an evaluation Qwen4 Max's Intelligence Index results and cost per task
Quote
+9 points over Qwen3.8-Max with lower costs.

What happened

Qwen says Qwen4 Max scores 43 on the Artificial Analysis Intelligence Index with xhigh reasoning effort, 9 points above Qwen3.8 Max and 3 points below Kimi K3.1 Pro. Artificial Analysis reports a cost of $3.27 per index task for Qwen4 Max, about three times less than Qwen3.8 Max and about 11% less than Kimi K3.1 Pro. The model is available through Alibaba's first-party APIs.

Why it matters

Qwen4 Max scores 9 points above Qwen3.8 Max on the Intelligence Index while costing about three times less per task, giving API users a lower-cost option at that benchmark level.

Read next

See the full storyQwen4 Max Scores 43 on the Artificial Analysis Intelligence Index
Article

+9 points over Qwen3.8-Max with lower costs. Give it a try! @ArtificialAnlys

Quoted postArtificial Analysis@ArtificialAnlys

Alibaba has launched Qwen4 Max, gaining 9 points over Qwen3.8 Max on the Artificial Analysis Intelligence Index and landing 3 points behind Kimi K3.1 Pro at ~11% lower Cost per Task

Qwen4 Max, using xhigh reasoning effort, scores 43 on the Intelligence Index, 9 points ahead of Qwen3.8 Max (xhigh) and 3 points behind Kimi K3.1 Pro (max, 46)

Key takeaways:

  • Qwen4 Max improves on every evaluation in the Index vs Qwen3.8 Max (xhigh, 34): HLE-Diamond +21 points (24% to 45%), AA-Omniscience +18 points, AutomationBench-AA +17 points, CritPt v2 +13 points, Terminal-Bench-Science +11 points, Terminal-Bench 4.0 +9 points, GDP.pdf +3 points, FrontierCode 2.0 +3 points and AA-Briefcase +82 Elo.
  • Qwen4 Max costs $3.27 per Intelligence Index task, ~3x less than Qwen3.8 Max ($10.02) and ~11% less than Kimi K3.1 Pro ($3.68): This is driven by 25% lower per-token pricing than Qwen3.8 Max, 70% cheaper cache hits and lower token usage. Tencent Hy4 costs ~3x less per task ($1.03) than Qwen4 Max but scores 8 points lower on the Index.
  • Qwen4 Max uses 39% fewer output tokens per task than Qwen3.8 Max: ~105k output tokens per Intelligence Index task vs ~171k for Qwen3.8 Max, ~72k for Kimi K3.1 Pro and ~78k for Tencent Hy4. Qwen4 Max uses fewer tokens, while increasing 9 points on the Intelligence Index
  • Improvements in AA-Briefcase (knowledge work): On AA-Briefcase, our evaluation of agentic work across real business files, Qwen4 Max reaches 1686 Elo vs 1674 for Kimi K3.1 Pro and 1446 for Tencent Hy4.
  • Qwen4 Max trails Kimi K3.1 Pro on Terminal-Bench 4.0 and Tencent Hy4 on FrontierCode 2.0: Qwen4 Max scores 48% on Terminal-Bench 4.0 vs 67% for Kimi K3.1 Pro and 37% for Tencent Hy4. On FrontierCode 2.0, @cognition’s private evaluation of real-world coding tasks, it scores 26%, level with Kimi K3.1 Pro (26%) and behind Tencent Hy4 (31%).

Key model details:

  • Context window: 1M tokens
  • Multimodal: Text and image input with text output
  • Pricing: $1.50/$4.50 per 1M input/output tokens on Alibaba's first-party APIs, with a $0.075 cache hit price. This is lower than Qwen3.8 Max across the board ($2.00/$6.00, with a $0.25 cache hit price)
  • Availability: @Alibaba_Qwen 's first-party APIs
View the quoted post on X

The story so far

Qwen says Qwen4 Max scored 43 on the Artificial Analysis Intelligence Index with xhigh reasoning effort, 9 points above Qwen3.8 Max and 3 points below Kimi K3.1 Pro. The model is available through Alibaba’s first-party APIs. Artificial Analysis reports that Qwen4 Max costs $3.27 per index task, about one-third the cost of Qwen3.8 Max and about 11% less than Kimi K3.1 Pro.

This story is in 11/17 AI DailyRead the whole issue