Skip to content
K = 0.731

Speculators v0.9.0 Adds Fused Loss Kernel and FP8 Hidden-State Backend

1 reports 1 reporting sources 2 hours agoNew

Understand the event

AI overview

Red Hat AI released Speculators v0.9.0 with a fused loss kernel and an FP8 hidden-state backend for speculative decoding training. vLLM said the fused loss kernel makes forward training passes up to 2x faster, while the FP8 hidden-state backend cuts storage by 50% with no quality loss. The trained speculators can be served in vLLM.

Generated by AI from reports · Updated 2 hours ago

Latest development10/20 00:23 (UTC+8)
vLLM Announces Speculators v0.9.0 With Faster, Lighter Speculative Decoding TrainingvLLM · X

Report timeline

Follow the reports to explore different sides of the event.

10/20
  1. vLLM · X
    vLLM Announces Speculators v0.9.0 With Faster, Lighter Speculative Decoding Training

    Speculators v0.9.0 is out — faster, lighter speculative decoding training 📷 Up to 2x faster forward training passes with a fused loss kernel, plus an FP8 hidden-state backend that cuts storage 50% with no quality loss. Train better speculators, serve them in vLLM. Great work https://t.co/FO7I7N75dC

Tomorrow at 08:00, a briefing like this arrives in your inbox.

The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.

Keep reading

Latest daily · Issue 26VAST open-sources EdenLive world model with over 24FPS generation at 720P