Skip to content
K = 0.731

METASTONE Releases TokenFleet Distributed Inference Scheduler

1 reports 1 reporting sources 22 hours agoNew

Understand the event

AI overview

METASTONE released TokenFleet, a distributed inference scheduling framework, and reported lower time to first token in company-provided GLM5.3 tuning results. Under the tested conditions, P95 time to first token fell 32.4% and P50 fell 71.3%. TokenFleet combines cache-aware request routing, capacity coordination across data centers, and mixed-workload management to improve the effective output of existing computing resources. A separate illustrative calculation estimated about 29.4% higher output throughput meeting service requirements, compared with about 1.5% higher total output throughput; those estimates were distinct from the reported GLM5.3 tuning results.

Generated by AI from reports · Updated 54 minutes ago

Latest development10/20 16:01 (UTC+8)
METASTONE launches TokenFleet inference scheduler, reporting 32.4% lower P95 time to first token量子位

Models in this story

  • GLM-5.3#16 overall$0.079 input / $1.4 output per million tokens
API prices of all models

Report timeline

Follow the reports to explore different sides of the event.

10/20
  1. 量子位
    METASTONE launches TokenFleet inference scheduler, reporting 32.4% lower P95 time to first token

    METASTONE released TokenFleet, a distributed inference scheduling framework combining cache-aware request routing, capacity coordination across data centers, and mixed-workload management. Company-provided GLM5.3 tuning results report 32.4% lower P95 and 71.3% lower P50 time to first token under tested conditions. A separate illustrative calculation estimates about 29.4% higher output throughput meeting service requirements, versus about 1.5% higher total output throughput.

Tomorrow at 08:00, a briefing like this arrives in your inbox.

The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.

Keep reading

Latest daily · Issue 28Mistral AI releases Voxtral 3 speech models and a voice agent SDK

Trending now

  1. 1Mistral Releases Voxtral 3 Speech Models, Voice Agents API and LiveKit-Based SDK
  2. 2DeepSeek Open-Sources DeepInfer and API Components, With Model and Hardware Limits
  3. 3OpenAI Acquires Orchestra Bio to Expand AI Tools for Drug Development
  4. 4Google Rolls Out Gemini Personal Intelligence to Android Auto, With Google Built-In Cars to Follow