Skip to content
K = 0.731
Benchmark sources

BullshitBench v2

Peter Gostev / BullshitBench / Knowledge · Detects false premises in questions and checks whether models continue inventing answers from them.
Official benchmark
Across e/accUnder observation
Evidence budgetNot scored
Upstream data timeUnverified
Last successful syncCollection not started

What it measures and how

Use the official default explicit-detection rate across all attempts and retain the average score of answered responses. Refusal rates remain for audit. Neither raw metric enters current overall or category vote budgets.

How this evidence is used

Raw results for reference: automatically collected after aggregates, tasks, configurations and run dates are checked against one public data version. Anonymous models stay private. Rolling APIs without confirmed fixed versions do not represent models. Excluded from current overall and category scores.

Limitations and attribution

All questions have false premises, so results do not represent general knowledge breadth. Model judges and refusal treatment affect scores; rolling aliases and anonymous models require identity verification first.

Data license: MIT · Copyright (c) 2026 Peter Gostev. Official scores ship with the repository, with no separate data-license exception.

Results are published by Peter Gostev / BullshitBench. Raw scores and the e/acc consensus index use different scales and cannot be added directly.