Skip to content
K = 0.731Subscribe

Ethan Mollick Calls for Human Baselines and Questions GDPx Grading

1 reports 1 reporting sources Updated 1 hours ago

Understand the event

AI overview

Ethan Mollick called for human baselines in the GDPx benchmark and flagged grading problems, while still describing it as a valuable way to evaluate AI work. He welcomed GDPx as a successor to GDPval but said it needs human baselines like those in GDPval. Mollick said GPT-6 Pro had spotted errors in the grading. He also argued that GDPx’s 0% scores hide economically meaningful differences in success rates.

Generated by AI from reports · Updated 50 minutes ago

Latest development11/2 14:30 (UTC+8)
Ethan Mollick says GDPx needs human baselines and flags grading errorsEthan Mollick · X

Read next

More changes in today’s briefingTypeSafe AI and Treg release open-source people search tool powered by Jev

Report timeline

Follow the reports to explore different sides of the event.

11/2
  1. Ethan Mollick · X
    Ethan Mollick says GDPx needs human baselines and flags grading errors

    Nice to see a successor to GDPval to evaluate AI work but really needs human baselines like GDPval. Funnily enough, GPT-6 Pro spotted some errors in the grading. I also think the 0% scores hide economically meaningful differences in success rates. But still a valuable benchmark! https://t.co/mfKfsKzcpM https://t.co/DBqS9czZ8t

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.