Why Muse Wonder’s lava lamp failure complicates comparisons of coding agent benchmarks

What happened

Meta’s Muse Wonder generated a red cylinder in BridgeMind’s 3D lava lamp test, prompting accusations of benchmark overoptimization. The article explains that the test used standard Muse Wonder, while some highlighted scores came from Wonder Full, and Meta acknowledges weaknesses in visual and spatial reasoning. It also examines how execution frameworks and reasoning budgets affect agent benchmarks, arguing that one failed task cannot establish benchmark overfitting and that unfamiliar tasks require separate evaluation.

Read next