What happened
We're grateful for our research partner working with us to study looped transformers. Goodfire explored how layers pass information to future loops, and which interpretability tools allow us to understand the model's inner workings.
We're grateful for our research partner working with us to study looped transformers.
Goodfire explored how layers pass information to future loops, and which interpretability tools allow us to understand the model's inner workings.
View the quoted post on XLooped models are weird. A layer can write to itself or read from "future" layers.
@thinkymachines gave us early access to Inkling 2 Mini so we could study its inner mechanisms. We uncover how it coordinates information flow across layers, blocks, and loops.
Studying Inkling 2 Mini gave us cause for optimism: as transformers continue to evolve, we believe our ability to understand their internal mechanisms can evolve with them.
Read the full post: goodfire.com/research/interpr…
How the story developed · 2
View full event- 11/21 08:23 (UTC+8)
Thinking Machines Thanks Goodfire for Studying Looped Transformers This report
- 11/21 04:39 (UTC+8)Hugging Face and Thinking Machines Lab Release Inkling 2 MiniHugging Face Blog
