vLLM Releases v0.32.0 With Structured Decisions and Low-Precision KV Updates

1 reports 1 reporting sources 1 hours agoNew

Understand the event

AI overview

vLLM released v0.32.0, adding a /v1/systemone structured-decision endpoint, a generic hf tool/reasoning parser, and low-precision KV updates. The release adds NVFP4 KV support on SM8x/90/12x, makes FP8 KV the default for GLM-5.x, and adds 4-bit KV on ROCm. It also includes Model Runner V2.

Generated by AI from reports · Updated 1 hours ago

Latest development10/18 21:40 (UTC+8)

vLLM v0.32.0 adds a structured-decision endpoint and low-precision KV updates vLLM · X

Report timeline

Follow the reports to explore different sides of the event.

10/18

21:40
vLLM · XPicks
vLLM v0.32.0 adds a structured-decision endpoint and low-precision KV updates
🚀 vLLM v0.32.0 is out! 814 commits · 363 contributors · 121 first-timers 🧭 New /v1/systemone structured-decision endpoint + generic hf tool/reasoning parser 🗜️ Low-precision KV everywhere: NVFP4 on SM8x/90/12x, FP8 KV default for GLM-5.x, 4-bit KV on ROCm 🛠️ Model Runner V2: https://t.co/xxeIWPXr2w

The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.

Keep reading

Latest daily · Issue 25DeepSeek releases DeepSeek-V4.1-Flash-1017 on its website, app, and API