Skip to content

Baoyu Explains How Main Sessions and Threads Can Reduce Token Use

Baoyu · X· @dotey·· 18 hours ago

3.8K views · 7 likes · 1 reposts (at collection)

Original
Who
dotey
What
explained model routing and prompt caching strategies for main sessions and threads
Quote
主会话 + Thread 的模式,优化好的话 Token 消耗其实还好。

What happened

Baoyu says a main session can use a less capable model for task routing, while threads handle tasks with stronger models. In Claude Project, Baoyu uses Sonnet 5.5 for the main session and Opus 5.5 for subtasks. Baoyu also recommends sending only recent messages when a main session’s cache expires and compressing thread context before a cache expires, such as at 50 minutes for a one-hour cache.

Read next

See the full storyBaoyu Explains Model Routing and Prompt Caching Across Main Sessions and Threads
Body · English edition pending (original content)

主会话 + Thread 的模式,优化好的话 Token 消耗其实还好。

首先主会话反而不需要特别强的模型,Sonnet 5.5 这个级别都够用了,因为它的主要职责是甄别和路由,或者说调度。

就跟公司前台一样,你不需要招一个专家,只要知道什么任务该找谁。

主会话里面很多任务都可以让 Thread 去完成,反倒是 Thread 应该用相对强一些的模型。当然更科学的办法是主会话在调度的时候根据任务难度去选择合适的模型。

比如我在用 Claude Project 的时候,主会话是 Sonnet 5.5,子任务是 Opus 5.5。

另外就是 Prompt Caching 的利用。

像主会话这种,特点是偶尔来个消息,很难利用好缓存,所以缓存如果过期了,不应该跟把所有历史会话和工具调用都发过去,而是只把最新的几条消息记录发过去,再配合 Session 的检索工具,可以很容易的收集齐上下文。

而对于 Thread,则是像传统的 Session 会话,会集中时间有很多对话,通常 Cache 利用率很高,但是也可能会出现缓存过期后又来新的消息,所以最佳策略是在缓存快过期的时候,比如 50 分钟(如果缓存一个小时的话)的时候,压缩一下上下文。

Quoted postGary Zhang@GaryZhang124

问题是,Grok bot这个如果走单主会话的交互太耗额度了。x.com/dotey/status/2121434782…

View the quoted post on X

The story so far

Baoyu outlined how AI workflows can pair a lightweight main session for task routing with stronger models in separate threads. In Claude Project, he uses Sonnet 5.5 for the main session and Opus 5.5 for subtasks. In a GrokBot math research setup, Baoyu said the main session acts like a front desk that receives incoming messages and opens a new thread for each research request, forwarding the prompt as written. A stronger model such as Fable plans, schedules, and reviews the work, while Opus handles execution. To reduce token consumption, Baoyu recommends sending only recent messages after a main session's prompt cache expires and compressing thread context before its cache expires, such as at 50 minutes for a one-hour cache.