
- Who
- AWS
- What
- introduces SageMaker HyperPod Global Inference Gateway for routing inference requests across clusters and Regions
- Quote
Today, SageMaker HyperPod introduces the Global Inference Gateway.
What happened
AWS introduces the Amazon SageMaker HyperPod Global Inference Gateway, a SageMaker-managed endpoint that routes requests across registered HyperPod clusters in one or multiple Regions. It selects clusters using KV cache headroom, queue depth, Region proximity, and error rate, and automatically shifts traffic from failed clusters. The service is available in 13 AWS Regions and uses SigV4, IAM, and AWS PrivateLink.
Why it matters
The gateway lets teams route inference across HyperPod clusters in multiple Regions through one endpoint, with automatic failover and private connectivity handled by SageMaker. Teams can tune routing weights across capacity, queue depth, proximity, and errors.
