跳到正文

Thinking Machines Thanks Goodfire for Studying Looped Transformers

Thinking Machines · X· @thinkymachines·· 5 小时前

4700 次浏览 · 111 赞 · 15 转发 (收录时)

原文

中文版本尚未就绪 · 显示原文

发生了什么

We're grateful for our research partner working with us to study looped transformers. Goodfire explored how layers pass information to future loops, and which interpretability tools allow us to understand the model's inner workings.

正文 · 中文版本尚未就绪(原文)

We're grateful for our research partner working with us to study looped transformers.

Goodfire explored how layers pass information to future loops, and which interpretability tools allow us to understand the model's inner workings.

引用Goodfire@GoodfireAI

Looped models are weird. A layer can write to itself or read from "future" layers.

@thinkymachines gave us early access to Inkling 2 Mini so we could study its inner mechanisms. We uncover how it coordinates information flow across layers, blocks, and loops.

Studying Inkling 2 Mini gave us cause for optimism: as transformers continue to evolve, we believe our ability to understand their internal mechanisms can evolve with them.

Read the full post: goodfire.com/research/interpr…

在 X 查看被引用的帖子

这件事的进展 · 2

查看完整事件
  1. 11月21日 08:23

    Thinking Machines Thanks Goodfire for Studying Looped Transformers 本条

  2. 11月21日 04:39
    Hugging Face and Thinking Machines Lab Release Inkling 2 MiniHugging Face Blog

接下来读什么

查看这件事的完整进展Thinking Machines Releases Inkling 2 Mini