What happened
Aravind Srinivas highlights KV-cache-compatible, decision-model-based routing on the Perplexity Orchestrator API. In a quoted post, Denis Yarats says the API offers 45+ open-weight and proprietary models, automatic provider failover, and unified billing. Yarats gives an example routing benign security and biology tasks, and anything after a refusal, from Fable 5.2 to GPT-6.1 Sol; on Terminal-Bench 4.1, the result rises from 38 to 41 solved tasks out of 63.
Why it matters
Task-specific routing preserves KV cache compatibility while sending refusals and selected security or biology tasks to GPT-6.1 Sol; the cited Terminal-Bench result rises from 38 to 41 solved tasks out of 63.
KV-cache compatible decision-model based routing on Perplexity Orchestrator API
View the quoted post on Xwe are launching the Orchestrator API: 45+ open-weight and proprietary models in one api, with automatic provider failover and unified billing.
general model routing doesn't really work in practice: it breaks the kv cache and is usually overfit to a few benchmarks. so we designed our routing api differently: it's kv cache friendly, and you decide which models to use for which tasks.
one example: fable 5.2 often refuses benign security and biology tasks, so we route those (and anything after a refusal) to gpt-6.1 sol. on terminal-bench 4.1 this goes from 38 to 41 solved tasks out of 63.
The story so far
Perplexity launched its Orchestrator API, providing developers with unified billing and access to over 45 open-weight and proprietary AI models. The platform automatically selects the optimal model provider for each task by balancing factors such as capability, capacity, latency, and availability. Aravind Srinivas, CEO of Perplexity, stated that the service features KV-cache-compatible, decision-model-based routing. According to Denis Yarats, this system includes automatic provider failover and allows for routing based on specific task types, such as handling security or biology queries. In an example provided by Yarats, routing tasks from Fable 5.2 to GPT-6.1 Sol resulted in an increase from 38 to 41 solved tasks out of 63 on Terminal-Bench 4.1.
