833 views · 4 likes · 0 reposts (at collection)
Kimi K3.1 Pro leads openly available models on Artificial Analysis Cyber Index
- Who
- Artificial Analysis
- What
- published benchmark results Kimi K3.1 Pro's performance on the Artificial Analysis Cyber Index
- When
- Quote
Kimi K3.1 Pro is the new #1 openly-available model for agentic cyber defense, behind only trusted-access model GPT-6 Sol (Daybreak Blue) on the Artificial Analysis Cyber Index
What happened
Artificial Analysis reports that Kimi K3.1 Pro scores 64% on its Cyber Index, leading openly available models and trailing trusted-access GPT-6 Sol (Daybreak Blue, max, 69%). The index measures agentic cyber defense across three benchmarks, primarily through public APIs, where safety refusals can reduce scores. K3.1 Pro encounters no safety refusals, but its $2.81 Cost per Task is 15% higher than Kimi K3 despite 33% lower per-token prices.
Read next
See the full storyKimi K3.1 Pro launches as revised Artificial Analysis index places it ninthKimi K3.1 Pro is the new #1 openly-available model for agentic cyber defense, behind only trusted-access model GPT-6 Sol (Daybreak Blue) on the Artificial Analysis Cyber Index
@Kimi_Moonshot's Kimi K3.1 Pro scores 64% on the Artificial Analysis Cyber Index, 7 points ahead of the previous leading openly-available model, DeepSeek V4.1 Flash 1017 (57%), and 5 points behind GPT-6 Sol (Daybreak Blue, max, 69%), which is only available through OpenAI's trusted-access program. Once the weights for K3.1 Pro are released, K3.1 Pro will also be the leading open weights model for cyber defense.
The Artificial Analysis Cyber Index measures capability on agentic cyber defense tasks and is composed of three benchmarks: CWE-Bench-AA, DeepsecBench-AA and CyberGym-E2E-AA. The Index primarily evaluates models through their publicly accessible APIs, reflecting the cyber capabilities available to developers.
Proprietary models from OpenAI, Anthropic and Google may have stronger cyber capabilities but lose points to safety refusals: GPT-6 Sol (Daybreak Blue, max), available through OpenAI's trusted-access program, scores 32 points above its public counterpart. Kimi K3.1 Pro encounters no safety refusals.
Key results:
➤ #1 openly-available model and #2 overall: Kimi K3.1 Pro scores 64%, up 23 points from Kimi K3 (41%) and ahead of DeepSeek V4.1 Flash 1017 (57%). Only the trusted-access GPT-6 Sol (Daybreak Blue, max, 69%) scores higher
➤ Highest CyberGym-E2E-AA score of any model: K3.1 Pro scores 95% on CyberGym-E2E-AA, placing it ahead of GPT-6 Sol (Daybreak Blue, 93%) and the next openly-available model, Mistral Large 4 (82%). On DeepsecBench-AA, K3.1 Pro scores 37%, behind GPT-6 Sol (Daybreak Blue, 48%) and GPT-6 Sol (46%). On CWE-Bench-AA it scores 61%, 4 points behind GPT-6 Sol (Daybreak Blue, 65%), with DeepSeek V4.1 Flash 1017 leading at 69%
➤ 15% increase in Cost per Task despite lower token prices: K3.1 Pro has a Cost per Task of $2.81, higher than GPT-6 Sol (Daybreak Blue, max, $1.77) and DeepSeek V4.1 Flash 1017 ($0.42). Kimi K3.1 Pro is 33% cheaper per token than Kimi K3 ($2/$10 vs. $3/$15 per 1M input/output tokens), but uses ~2x more input tokens, resulting in a 15% higher Cost per Task ($2.81 vs. $2.45)
The story so far
Moonshot AI released Kimi K3.1 Pro through its apps and API, but its weights remain unreleased, and Artificial Analysis’s revised Intelligence Index now places it ninth, behind eight closed models. The Decoder reported that version 5 assigns 60% of the index to private test sets to counter training on known questions. K3.1 Pro scored 46, trailing the leader by 12 points; under version 4.3, it ranked fourth with 55, three points behind Claude Opus 5.5. Ethan Mollick urged caution in interpreting the revised findings. Simon Willison reported Moonshot’s promise to release the weights by November 4, 2026. On Artificial Analysis’s separate Cyber Index, K3.1 Pro scored 64%, leading openly available models for agentic cyber defense while trailing trusted-access GPT-6 Sol (Daybreak Blue, max) at 69%. Artificial Analysis said K3.1 Pro encountered no safety refusals across the index’s three benchmarks. Its cost per task was $2.81, 15% higher than Kimi K3 despite 33% lower per-token prices. Willison reported standard API prices of $2 per million input tokens and $10 per million output tokens. Moonshot says the Highspeed variant runs two to three times faster; Willison reported that it costs twice as much, except for input token cache hits, which retain the standard price.
Keep up with AI, one email a day
Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-10-30