3.7K views · 31 likes · 2 reposts (at collection)
Ethan Mollick says GDPx needs human baselines and flags grading errors
- Who
- Ethan Mollick
- What
- comments on the GDPx benchmark, noting the need for human baselines and potential grading errors
- Quote
I also think the 0% scores hide economically meaningful differences in success rates.
What happened
Nice to see a successor to GDPval to evaluate AI work but really needs human baselines like GDPval. Funnily enough, GPT-6 Pro spotted some errors in the grading. I also think the 0% scores hide economically meaningful differences in success rates. But still a valuable benchmark!
Links in the post: 2Read next
See the full storyEthan Mollick Calls for Human Baselines and Questions GDPx GradingNice to see a successor to GDPval to evaluate AI work but really needs human baselines like GDPval.
Funnily enough, GPT-6 Pro spotted some errors in the grading. I also think the 0% scores hide economically meaningful differences in success rates. But still a valuable benchmark! https://t.co/mfKfsKzcpM https://t.co/DBqS9czZ8t
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵 https://t.co/RNnkcuUkihView the quoted post on X
The story so far
Ethan Mollick called for human baselines in the GDPx benchmark and flagged grading problems, while still describing it as a valuable way to evaluate AI work. He welcomed GDPx as a successor to GDPval but said it needs human baselines like those in GDPval. Mollick said GPT-6 Pro had spotted errors in the grading. He also argued that GDPx’s 0% scores hide economically meaningful differences in success rates.
Keep up with AI, one email a day
Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-11-02