- 谁
- Artificial Analysis
- 引文
SubQ Snap 1.2 scores 73 on the Artificial Analysis Decision Index v2.1, up from 68 for SubQ Snap 1.1: Drex Max v1.1 (73) and GLiDE v1.2 (73) are within 0.1 points.
发生了什么
Artificial Analysis 报告称，SubQ Snap 1.2 在其 Decision Index v2.1 中得分 73，高于 1.1 版的 68 分，与 Drex Max v1.1 和 GLiDE v1.2 的分差在 0.1 分以内。它在 AA-Paperclip 上的全通过率从 46% 升至 62%。在 AA-Omniscience-Decision 和 HLE-MultiChoice 上，非幻觉率有所提高，而准确率分别维持在 21% 和 9%；该帖子将这些提升归因于弃答。
接下来读什么
查看这件事的完整进展Artificial Analysis Adds SubQ Snap 1.2 Results to Decision IndexSubquadratic’s SubQ Snap 1.2 gains 5 points over SubQ Snap 1.1 on the Artificial Analysis Decision Index v2.1, and is tied with Drex Max v1.1 and GLiDE v1.2 as the leading decision model
SubQ Snap 1.2 is the latest decision model from @subquadratic. Pricing is unchanged at $0.04 per 1M input tokens, and it supports a 384K token context window.
Decision Index v2.1 updates AA-Symmetry to v1.1. AA-Symmetry measures whether a model keeps the same decision and confidence when a question changes in ways that should not matter. v1.1 moves the answer options into the state and rewords them with the same meaning.
Key takeaways:
- SubQ Snap 1.2 scores 73 on the Artificial Analysis Decision Index v2.1, up from 68 for SubQ Snap 1.1: Drex Max v1.1 (73) and GLiDE v1.2 (73) are within 0.1 points. The largest gains are on AA-Paperclip (+8 points) and AA-Symmetry (+8 points).
- The AA-Paperclip gain comes from more documents answered fully correctly: the all-pass rate rises from 46% to 62%, level with Drex Max v1.1 (63%) and GLiDE v1.2 (62%).
- The gains on AA-Omniscience-Decision and HLE-MultiChoice come from abstention, not accuracy: non-hallucination rates rise from 85% to 89% and from 46% to 55%. Accuracy is flat at 21% and 9%. On AA-Omniscience-Decision, SubQ Snap 1.2 attempts 29% of questions, down from 33%.
这件事到目前为止
Artificial Analysis reports that SubQ Snap 1.2 scored 73 on Decision Index v2.1, up from 68 for version 1.1 and within 0.1 points of Drex Max v1.1 and GLiDE v1.2. Its all-pass rate on AA-Paperclip rose from 46% to 62%. On AA-Omniscience-Decision and HLE-MultiChoice, non-hallucination rates improved while accuracy stayed at 21% and 9%, respectively; Artificial Analysis attributes those gains to abstention. The results build on Decision Index v2, which Artificial Analysis released with private datasets accounting for 45% of the index, up from 25% in v1. The updated Decision Score rewards both correct answers and calibrated confidence.