This is a fantastic idea - a mass attempted replication of old findings of LLMs with new models. Much conventional wisdom is outdated. Recent models no longer invent citations and both hallucination rates & stereotyping has dropped. But they are still sycophantic & lean left.

Read next

• More changes in today’s briefingOpenAI’s updated Model Spec explained through its five core values

The story so far

Ethan Mollick called for a mass effort to replicate older findings about large language models using newer models, arguing that much conventional wisdom may be outdated. He said recent models no longer invent citations and have lower hallucination and stereotyping rates, while remaining sycophantic and leaning left.