Amazon SageMaker Inference 为端点新增多实例 GPU 分区功能
SageMaker Inference now supports Multi-Instance GPU to maximize GPU utilization for model endpoints
- 谁
- Amazon SageMaker
- 引文
Today, Amazon SageMaker Inference supports GPU partitioning, which uses NVIDIA Multi-Instance GPU (MIG) to divide a physical GPU into independent partitions.
发生了什么
Amazon SageMaker Inference 现已在推理组件端点上支持 NVIDIA Multi-Instance GPU (MIG) GPU 分区，使团队能够在相互隔离的 GPU 分区中部署模型。AWS 在七个区域的 P4d、P5、G7 和 G7e 实例系列上支持此功能。根据 AWS 的对比，在测试的高并发、长输入工作负载中，一个 H100 mig-3g.40gb 分区的吞吐量高于完整的 A100-40GB，且每百万输出令牌的成本更低；对于短输入或低并发工作负载，A100 的成本仍然更低。
推荐理由
这篇文章对比了一个 H100 分区与完整的 A100 在五种流量模式下的表现，展示了新款 GPU 分区在哪些情况下速度更快、成本更低，以及 A100 在哪些情况下成本仍然更低。
接下来读什么
查看这件事的完整进展Amazon SageMaker Adds NVIDIA MIG Partitioning for Inference Endpoints这件事到目前为止
Amazon SageMaker Inference now supports NVIDIA Multi-Instance GPU partitioning on inference component endpoints, allowing teams to assign models isolated portions of a physical GPU. The feature is available on P4d, P5, G7, and G7e instances in seven AWS Regions, including Northern Virginia, Oregon, Singapore, Tokyo, Frankfurt, and Ireland. Single-model and serverless endpoints, as well as heterogeneous instance pools, cannot use GPU partitioning. AWS reported that in its tests, one H100 MIG 3g.40gb partition delivered higher throughput and lower cost per million output tokens than a full A100-40GB for high-concurrency workloads with long inputs. For short inputs or low concurrency, AWS found the A100 remained cheaper.
每天一封，跟上 AI 的重要变化
模型发布、价格变化、算力进展，附原始来源。免费，确认后开始发送，随时一键退订。 看一份往期日报 · 2026-11-06