Skip to content
K = 0.731
Benchmark sources

FACTS Parametric

Google DeepMind / Kaggle / Knowledge · Answers factual questions about the world without search tools to test the accuracy of a model’s own knowledge.
Official benchmark
Across e/accUnder observation
Evidence budgetNot scored
Upstream data timeUnverified
Last successful syncCollection not started

What it measures and how

Consider only Score from a fixed task version. Retain public-set, private-set and confidence-interval data for review; three correlated scores must not add duplicate votes. FACTS is not among the 10 underlying AA v4.3 benchmarks.

How this evidence is used

Under observation: official protocol and licensing are confirmed, but completed task counts or completeness flags are absent. Some rows have very wide intervals; aggregation of total, public-set and private-set scores also needs clarification. Awaiting the organization’s explanation; unusual rows are not deleted to force integration, and no scoring budget is used.

Limitations and attribution

Judges and the initiating organization are Google, so other organizations’ evidence is needed. Exports do not state whether some results completed the same task set; partial results are not full evaluations. Sample sizes inferred from intervals are not officially confirmed facts.

Data license: Apache 2.0 · Explicitly declared on the official benchmark and Score task pages. Kaggle documentation provides public leaderboard JSON downloads without sign-in.

Results are published by Google DeepMind / Kaggle. Raw scores and the e/acc consensus index use different scales and cannot be added directly.