Skip to content
K = 0.731Subscribe

Matthew Honnibal Criticizes How the Astra Study Presents Its Experiments

1 reports 1 reporting sources Updated 3 hours ago

Understand the event

AI overview

Matthew Honnibal criticized the MIT study’s presentation of Astra’s experiments, saying it could make distinct methods look like one ability to read and modify model weights. The study tested small networks using raw weights without executing code, while a Qwen repair experiment let Astra use Python and up to 50 forward tests on an 800-million-parameter model. Honnibal argued that grouping the experiments could leave readers with the impression that the small-network approach scales directly to Qwen.

Generated by AI from reports · Updated 1 hours ago

Latest development11/5 15:10 (UTC+8)
Astra’s Qwen Results Draw Criticism Over How the MIT Study Frames Its MethodsQbitAI

Read next

More changes in today’s briefingOpenAI Releases Images 2.6 With SVG Export, GIFs, and Faster Generation

Report timeline

Follow the reports to explore different sides of the event.

11/5
  1. QbitAI
    Astra’s Qwen Results Draw Criticism Over How the MIT Study Frames Its Methods

    QbitAI reports that spaCy creator Matthew Honnibal challenged how an MIT study presents Astra’s experiments. He says tests on small networks used raw weights without code execution, while the Qwen repair experiment allowed Python and up to 50 forward tests on the 800-million-parameter model. Honnibal argues that grouping these distinct methods can give readers the impression that the small-model approach scales directly to Qwen.

Keep up with AI, one email a day

Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.