Skip to content
K = 0.731Subscribe
Original
IndustryArtificial Analysis · X· @ArtificialAnlys·· 3 hours ago

4.8K views · 45 likes · 3 reposts (at collection)

Artificial Analysis to Make the CritPt v2 Grading Server Available via Its API

Who
Artificial Analysis
What
plans to make available the CritPt v2 grading server via its API
Quote
We’ve worked with the CritPt authors to finalize the CritPt v2 grading server and will be making this available to the community via Artificial Analysis’ API tomorrow.

What happened

Artificial Analysis says it finalized the CritPt v2 grading server with the CritPt authors and will make it available to the community via its API tomorrow. The authors’ robustness improvements to the grading logic were incorporated as v5.0.1 of the Artificial Analysis Intelligence Index; after regrading every model, most saw small score increases, while frontier model rankings stayed unchanged and scores moved by at most 1 point.

Read next

See the full storyArtificial Analysis Plans CritPt v2 Grading Server API Release
Article

We’ve worked with the CritPt authors to finalize the CritPt v2 grading server and will be making this available to the community via Artificial Analysis’ API tomorrow. As part of this process the authors made robustness improvements to the grading server logic, which we have incorporated as v5.0.1 of the Artificial Analysis Intelligence Index

We regraded every model and observe small score increases for most models. Intelligence Index rankings are unchanged among frontier models, with scores moving by at most 1 point.

Thanks to @MiniHui_zhu, @MinyangTian1 and the rest of the CritPt team for their ongoing collaboration

The story so far

Artificial Analysis says it finalized the CritPt v2 grading server with the CritPt authors and plans to make it available to the community through its API tomorrow. The authors’ robustness improvements to the grading logic were incorporated into version 5.0.1 of the Artificial Analysis Intelligence Index. After regrading every model, Artificial Analysis reported that most scores rose slightly, while frontier model rankings stayed the same and scores changed by no more than 1 point.

Open sourceProduct updatesBenchmarks#Product update#Benchmarks#Open-source ecosystem

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-11-04