Skip to content
K = 0.731
Benchmark sources

MCP Atlas

Scale AI / Professional work · Uses real MCP services to retrieve material and operate documents and databases, completing tasks across software.
Official benchmark
Across e/accUnder observation
Evidence budgetNot scored
Upstream data timeUnverified
Last successful syncCollection not started

What it measures and how

Fix the protocol revised in April 2026 and the 100-tool-call budget. Separate the full 1,000 tasks from the public 500. Use only task pass rate, without duplicate votes for answer-condition coverage. Judges, retries and tool versions must be locked to the score batch.

How this evidence is used

Under observation: could add evidence for tool execution across office software. Stable score exports, actual evaluation batches and display terms need confirmation. Only the source is listed; no automated collection or ranking yet.

Limitations and attribution

Older website methodology and live scores may be out of sync. Public task code is not a live score export. Other Scale benchmarks belong to the same organization and do not increase the independent organization count.

Data license: Official environment code is MIT. This does not automatically cover live website scores; leaderboard display terms remain unconfirmed.

Results are published by Scale AI. Raw scores and the e/acc consensus index use different scales and cannot be added directly.