Deploy NVIDIA Nemotron Models on Amazon EKS and SageMaker HyperPod EKS
Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS
- Who
- AWS
- What
- explains architecture, validation, and performance evaluation patterns for deploying NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS
- Quote
This post explores its architecture and use cases, then shows how to deploy it on Amazon Elastic Kubernetes Service (Amazon EKS) , Amazon EKS Auto Mode , and Amazon SageMaker HyperPod EKS .
What happened
AWS explains how to deploy NVIDIA Nemotron 3.5 Super VL on Amazon EKS, EKS Auto Mode, and SageMaker HyperPod EKS using NVIDIA Dynamo and vLLM. The guide covers the model’s 120B total parameters and 12B active parameters per token, deployment architecture, and validation on B200 and H200 instances. It also shows how to use aiperf concurrency sweeps to compare aggregated and disaggregated deployments and plan capacity against latency targets.
Why it matters
The concurrency sweep method connects latency budgets to per-replica throughput, offering a practical way to size deployments and identify whether prefill or decode capacity needs to increase.
Read next
See the full storyAWS Details Deploying NVIDIA Nemotron Models on EKS and SageMaker HyperPod EKSThe story so far
AWS published a guide for deploying NVIDIA Nemotron 3.5 Super VL on Amazon EKS, EKS Auto Mode, and SageMaker HyperPod EKS using NVIDIA Dynamo and vLLM, with deployment architecture and validation on B200 and H200 instances. The model has 120B total parameters and 12B active parameters per token. The guide also shows how to run aiperf concurrency sweeps to compare aggregated and disaggregated deployments and plan capacity against latency targets.
Keep up with AI, one email a day
Model releases, price changes and compute developments, with original sources. Free, confirmed by email, and easy to unsubscribe. Read a past issue · 2026-10-29