Microsoft says MAI is OpenRouter’s most accurate and popular transcription model
MAI is the most accurate & most popular transcription model on OpenRouter across all languages and use cases. Try it out for your agents now! 👇 https://t.co/rHOlsQWzro
All news · Today0 items
#Benchmarks
All collected and published items, ordered by time. Picks are a selected subset.
Opinion
Viewpoints, commentary, analysis, interviews, and discussion of trends.
Mustafa Suleyman · X30 François Chollet · XPicks81 Chollet compares Gemini 4 Argon’s ARC-AGI-3 performance with Astra
Argon has a similar performance profile on ARC-AGI-3 as Astra, François Chollet says, adding that Google is back at the frontier. https://t.co/a6GGEsMN4A The quoted ARC Prize post reports Gemini 4 Argon’s verified ARC-AGI-3 results of 97.5%, $4.8K with a provider adapter harness and 49.8%, $5.8K with the standard harness, plus 91.3%, $0.32/task on ARC-AGI-2 and 98.5%, $0.16/task on ARC-AGI-1. https://t.co/8siwgfBJcr
Why it matters: The reported ARC-AGI-3 scores distinguish provider adapter and standard harness results, providing context for Chollet’s comparison of Argon with Astra and his assessment of Google’s frontier position.
Alexandr Wang · X29 Alexandr Wang Says Muse Wonder Full May Surprise You on Hard Coding Tasks
Try Muse Wonder Full on your hardest coding tasks—it will surprise you! “By my vibe testing, It's better than Astra. Not quite as good as Claude Code Fable and 5.5 Opus.” https://t.co/8RTGZ434kG
Alexandr Wang · X26 Alexandr Wang Urges Muse Wonder to Invest in Its Tech Tree
@Hesamation is benchmarking models on Stronghold Crusader, a strategic game they grew up with. They said Muse Wonder playing the game on an old laptop is “ridiculously fun to watch,” and that they have tested Opus, Sol, GLM flash, Kimi K3, and more models. Alexandr Wang reacted: “cmon muse wonder i believe in you invest in your tech treeeeeeee” https://t.co/dSUEmGVxV7
The Verge · AI58 Ray-Ban Meta Audio glasses review: I don’t miss the camera
Victoria Song finds the camera-free Ray-Ban Meta Audio glasses comfortable and useful as open-ear headphones, while finding Muse Spark helpful mainly for simple requests and reminders. Her review unit lasted two full workdays without returning to its case, but audio suffered from sound leakage and loud surroundings. Removing the camera eased her discomfort wearing smart glasses in public, though she argues that it does not resolve broader privacy concerns and finds the glasses expensive for their current capabilities.
Alexandr Wang · X15 Alexandr Wang Reacts to @lillyguisnet’s Image-Pointing Benchmark
@lillyguisnet posted: “New sota for my image pointing.” Alexandr Wang: now this is the kind of benchmarking i can get behind
Alexandr Wang · X31 Alexandr Wang urges followers to “fill your soul with wonder”
fill your soul with wonder
Ethan Mollick · X15 People take the Artificial Analysis Index and its individual scores way too seriously.
People take the Artificial Analysis Index and its individual scores way too seriously.
Alexandr Wang · X25 Alexandr Wang Reacts to @theo’s S-Tier Model Ranking
@theo ranked Opus 5.5 and Muse Wonder as S tier. Alexandr Wang said, “wow, S tier from theo! not on my bingo card”
Alexandr Wang · X18 Alexandr Wang Says Muse Wonder Outperformed Astra in Physics Testing
“[Muse Wonder] definitely mogged Astra in my hardcore physics testing.” — the new beff times
Nathan Lambert · X19 If you announce a model's benchmarks without releasing it (weights, product, api, etc "later"), feels like almost a sure thing that someone releases a better model in your category before you get it out.
If you announce a model's benchmarks without releasing it (weights, product, api, etc "later"), feels like almost a sure thing that someone releases a better model in your category before you get it out.
Nathan Lambert · X33 Nathan Lambert Sees Investment in RL Data and Real-World Evaluations
Nathan Lambert says the data industry has grown, but low-quality output can amplify behaviors such as reward hacking. He is advising Mercor on research into sample-efficient data for RL and says the field needs stronger evaluation science and benchmarks tied to areas of clear economic value. Lambert expects major investment around frontier labs in open inference, open post-training and data, with opportunities to build on open models.
François Chollet · X58 ARC Prize reports an ARC-AGI-2 high score of 88.06% by @tufalabs
François Chollet writes, "We're reaching really good ARC-AGI-2 scores on Kaggle now!" In the quoted post, @arcprize reports a new ARC Prize 2026 ARC-AGI-2 high score of 88.06% by @tufalabs. The $150K Bonus Prize, on top of the guaranteed prizes, will be split among all teams scoring over 85%.
Microsoft Research22 Microsoft Research: why AI evaluations can fail unexpectedly outside traditional benchmarks
Jennifer Neville of Microsoft says models can fail unexpectedly when tested beyond traditional benchmarks. Evaluation is essential to pushing the performance boundaries of current AI systems and aligning them with user needs. She recommends paying close attention to data when using AI systems, since the data itself is often the first thing to examine when results differ from expectations.
Andrej Karpathy · X30 Karpathy shares a land-or-sea benchmark for LLMs
Cool eval. Just ask an LLM 'land or sea?' and give it a latitude and longitude as text. Ask 16,200 times and plot the results as an image. The model knows the answer. It's a compression of the internet.
Noam Brown · X59 Noam Brown advocates measuring model intelligence as a function of cost
Noam Brown welcomes how OpenAI presents model evaluations, writing that intelligence should be measured as a function of cost. In the quoted post, OpenAI describes GPT-6.1 Sol as offering near-Astra intelligence for a fifth of the price and calls it the most cost-efficient model for its performance available today.
Logan Kilpatrick · X22 Logan Kilpatrick: AI product teams should spend over 25% of their time on benchmarks
If you're building a product with AI, you should spend more than 25% of your time on benchmarks and work to get model labs to pay attention to them. It's the simplest way to accelerate a company's progress.
Import AI42 Import AI 468: 23 RSI policy recommendations, PostTrainBench+, and how trust and transparency shape the AI race
Import AI issue 468 summarizes 23 policy recommendations from the think tank IFP for managing risks from automated AI research and development. They span seven categories, including improving transparency in automated AI R&D, accelerating development of AI verification technology, and extending U.S. AI leadership.
Import AI52 Import AI 464: Fable writes GPU kernels, AI automation, and analog computing
Import AI 464 summarizes several research developments. Fable wrote a megakernel on KernelBench-Mega that achieved an 18.71× speedup with CUDA on an RTX PRO 6000 Blackwell, compared with 14.4× for Claude Opus 4.8, 11.14× for GLM-5.2, and 4.34× for GPT 5.5.