vLLM and Physis-AI Introduce PhysisServe for Autoregressive Denoising Serving

What happened

vLLM and Physis-AI introduce PhysisServe, combining fine-grained stage pipelining, time-sharing, and multi-stream joint scheduling to optimize single-stream output latency and multi-stream system efficiency; its core techniques are coming to vLLM-Omni. Physis-AI reports that MiniMax H3 33B produces a chunk every 4.083 ms on 200 GPUs, measuring DiT denoising only, excluding VAE decoding and end-to-end response latency. https://physis-ai.com/blog/physisserve/en https://github.com/Physis-AI/PhysisServe

The story so far

Physis-AI and vLLM have introduced PhysisServe, an inference system for autoregressive denoising, with its core techniques planned for integration into vLLM-Omni. The system combines fine-grained stage pipelining, time-sharing, and joint scheduling across multiple streams to target lower single-stream output latency and greater multi-stream system efficiency. Physis-AI reports that MiniMax H3 33B produces a chunk every 4.083 ms on 200 GPUs using PhysisServe. That measurement covers only DiT denoising; it excludes VAE decoding and does not represent end-to-end response latency.

Keep reading