NVIDIA 的全栈协同设计如何应对 AI 的内存挑战
More Performance From Every Byte: How Full-Stack Codesign Solves AI’s Memory Challenge
- 谁
- NVIDIA
- 引文
NVIDIA’s full-stack codesign makes every byte of HBM count by relieving pressure from model weights, KV cache and RAG data.
发生了什么
NVIDIA 认为，随着模型规模扩大、智能体上下文变长，GPU 内存承受的压力也随之增加，而全栈协同设计可以提高 AI 工厂内存层级中每字节所完成的有效工作量。NVIDIA 介绍了如何利用 NVFP4 缩减模型权重和 KV 缓存的占用，将不活跃的上下文迁移到成本较低的层级，并将 RAG 索引保留在 HBM 之外。针对 SemiAnalysis AgentX，NVIDIA 报告称，在每位用户每秒生成 100 个 token 的条件下，GB300 NVL72 的每美元吞吐量最高可达 H200 的 10 倍；在每位用户每秒生成 125 个 token 的条件下，Vera Rubin NVL72 的每美元吞吐量最高可达 GB300 的 30 倍。
接下来读什么
查看这件事的完整进展NVIDIA Outlines Five Ways to Ease AI Inference Memory Pressure这件事到目前为止
NVIDIA outlined five ways its full-stack approach can ease AI inference memory pressure, from shrinking model weights to reusing context across servers. It says NVFP4 reduces model-weight and KV-cache footprints, inactive context can move to lower-cost memory tiers, and retrieval-augmented generation indexes can stay outside high-bandwidth memory (HBM). For SemiAnalysis AgentX, NVIDIA reported up to 10x higher throughput per dollar for GB300 NVL72 than H200 at 100 tokens per second per user, and up to 30x for Vera Rubin NVL72 than GB300 at 125 tokens per second per user.
每天一封，跟上 AI 的重要变化
模型发布、价格变化、算力进展，附原始来源。免费，确认后开始发送，随时一键退订。 看一份往期日报 · 2026-11-05