跳到正文

vLLM-Omni 为 AURA 2.0 增加首日支持

vLLM · X当事人· @vllm_project·· 15 小时前精选

1062 次浏览 · 11 赞 · 2 转发 (收录时)

原文

发生了什么

vLLM-Omni 为 AURA 2.0 提供首日支持。AURA 2.0 是一个 9B 流式视频模型,能够发起对话、根据实时场景和对话调用工具,并用帧对齐框指出讨论的对象。在配置两个 ~140 GB 加速器的单会话配置中,该流水线实现了 344 毫秒的首次回复音频数据包中位数时间。AURA 2.0 在所示的四项综合基准测试对比中处于领先地位:IPIBench、StreamArena、ProactiveVideoQA 和 PhoStream。

推荐理由

AURA 2.0 的 344 毫秒首次回复音频数据包中位数时间显示了 vLLM-Omni 在配置两个 ~140 GB 加速器的单会话配置中所达到的延迟;该结果仅限于该设置。

正文 · 中文版本尚未就绪(原文)

🎉 Day-0 support for AURA 2.0 in vLLM-Omni!

A 9B streaming video model that watches the scene, starts relevant conversations, and acts on what it sees.

💬 Model-Initiated Conversation: decide when to speak and what to say, without waiting for a question.
🔧 Stream-Aware Tool Calling: use the live scene and conversation to call tools and bring results back into the dialogue.
🎯 Interactive Visual Grounding: point to the objects being discussed with frame-aligned boxes.

vLLM-Omni powers the pipeline from speech recognition through video-language inference to speech generation. Session and prefix reuse, a fast path for silent turns, and sentence-level speech handoff deliver 344 ms median time to the first reply audio packet in the single-session setup with two ~140 GB accelerators.

AURA 2.0 leads the four aggregate benchmark comparisons shown below: IPIBench, StreamArena, ProactiveVideoQA and PhoStream.

Thanks to the AURA team at @Huawei Research, CUHK MMLab and HKUST!

Project & demos: aurateam2026.github.io/Aura-2…
Weights on @huggingface: huggingface.co/aurateam/AURA_2

这件事到目前为止

vLLM-Omni has introduced day-one support for AURA 2.0, a 9B streaming video model capable of initiating conversations, executing tool calls based on live video and audio input, and identifying objects with frame-aligned bounding boxes. In a single-session configuration utilizing two 140 GB accelerators, the system achieved a median time to first reply audio packet of 344 ms. According to the release, AURA 2.0 outperforms other models across four primary benchmarks: IPIBench, StreamArena, ProactiveVideoQA, and PhoStream.

接下来读什么

查看这件事的完整进展vLLM-Omni Adds Day-One Support for AURA 2.0 Streaming Video Model
这条收录于 11月18日 AI 日报看这一期日报的全部内容