Stream low-latency speech from AI agents using Kokoro TTS on Amazon SageMaker AI

What happened

AWS demonstrates how to stream completed sentences from an AI agent to Kokoro-82M on Amazon SageMaker AI so speech synthesis begins before the full response is generated. Its tests reduced mean time to first audio byte from 329.3 ms to 156.7 ms in isolation and measured 3.5 times faster first audio through the deployed AgentCore runtime. The guide covers persistent bidirectional sessions, binary WebSocket frames, and audio transport tradeoffs involving SSE and base64.

Read next

• More changes in today’s briefingMicrosoft Research and Collaborator Resolve the KLS Conjecture With Human-Directed AI

The story so far

AWS demonstrated a method for streaming AI agent responses sentence by sentence to Kokoro-82M on Amazon SageMaker AI, reporting that speech synthesis began sooner and reduced the wait for the first audio. Instead of waiting for an agent’s complete response, the method sends each completed sentence to the hosted text-to-speech model as soon as it is available. AWS said its isolated tests reduced mean time to first audio byte from 329.3 ms to 156.7 ms, while tests through the deployed AgentCore runtime produced first audio 3.5 times faster. The guide covers persistent bidirectional sessions, binary WebSocket frames, and audio transport tradeoffs involving SSE and base64. Its sample container pins an older PyTorch version with known advisory PYSEC-2026-2286, which is fixed in torch 2.10.0.