Skip to content

Thinking Machines Thanks Goodfire for Studying Looped Transformers

Thinking Machines · X· @thinkymachines·· 4 hours ago

4.7K views · 111 likes · 15 reposts (at collection)

Original

What happened

We're grateful for our research partner working with us to study looped transformers. Goodfire explored how layers pass information to future loops, and which interpretability tools allow us to understand the model's inner workings.

Article

We're grateful for our research partner working with us to study looped transformers.

Goodfire explored how layers pass information to future loops, and which interpretability tools allow us to understand the model's inner workings.

Quoted postGoodfire@GoodfireAI

Looped models are weird. A layer can write to itself or read from "future" layers.

@thinkymachines gave us early access to Inkling 2 Mini so we could study its inner mechanisms. We uncover how it coordinates information flow across layers, blocks, and loops.

Studying Inkling 2 Mini gave us cause for optimism: as transformers continue to evolve, we believe our ability to understand their internal mechanisms can evolve with them.

Read the full post: goodfire.com/research/interpr…

View the quoted post on X

How the story developed · 2

View full event
  1. 11/21 08:23 (UTC+8)

    Thinking Machines Thanks Goodfire for Studying Looped Transformers This report

  2. 11/21 04:39 (UTC+8)
    Hugging Face and Thinking Machines Lab Release Inkling 2 MiniHugging Face Blog

Read next

See the full storyThinking Machines Releases Inkling 2 Mini