1K views Β· 21 likes Β· 6 reposts (at collection)
vLLM Announces Speculators v0.9.0 With Faster, Lighter Speculative Decoding Training
- Who
- Red Hat AI
- What
- released Speculators v0.9.0 with a fused loss kernel and an FP8 hidden-state backend
What happened
Speculators v0.9.0 is out β faster, lighter speculative decoding training π· Up to 2x faster forward training passes with a fused loss kernel, plus an FP8 hidden-state backend that cuts storage 50% with no quality loss. Train better speculators, serve them in vLLM. Great work
Links in the post: 1The story so far
Red Hat AI released Speculators v0.9.0 with a fused loss kernel and an FP8 hidden-state backend for speculative decoding training. vLLM said the fused loss kernel makes forward training passes up to 2x faster, while the FP8 hidden-state backend cuts storage by 50% with no quality loss. The trained speculators can be served in vLLM.
Keep reading
Latest daily Β· Issue 26VAST open-sources EdenLive world model with over 24FPS generation at 720PTomorrow at 08:00, a briefing like this arrives in your inbox.
The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue Β· 2026-10-19