Training and serving i14’s Laplace 3B on AWS Trainium2 with native PyTorch
Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
- Who
- AWS Machine Learning Blog
- What
- explained training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
- Quote
This post explains the model design, the changes i14 made for Trainium2, and the training and inference results.
What happened
i14 and AWS explain how they adapted Laplace 3B’s linear attention architecture for training and serving on AWS Trainium2 using native PyTorch through TorchNeuron Native. The model maintains a fixed 29.5 MB recurrent state per sequence and reached roughly 120,000 training tokens per second, with results reported through about 44 billion tokens. Inference reached 11,544 output tokens per second across 1,280 concurrent sequences on one trn2.48xlarge instance.
Why it matters
The engineering walkthrough shows how communication patterns, operator dispatch, and changing batch widths can dominate performance, offering a transferable profiling approach for deploying custom architectures on accelerators.
The story so far
i14 and AWS adapted i14’s Laplace 3B linear attention model to train and serve on AWS Trainium2 using native PyTorch through TorchNeuron Native, reporting roughly 120,000 training tokens per second and inference throughput of 11,544 output tokens per second. AWS said its GenAIIC team trained the model on one trn2.48xlarge instance using all 64 logical NeuronCores, with training results reported through about 44 billion tokens. The model maintains a fixed 29.5 MB recurrent state per sequence. For inference, the team used an unmodified vLLM engine on one trn2.48xlarge instance; the reported throughput was measured across 1,280 concurrent sequences.
Keep reading
Latest daily · Issue 27TorchNeuron brings native PyTorch support for Hugging Face models on AWS TrainiumTrending now
- 1Mistral Releases Voxtral 3 Speech Models, Voice Agents API and LiveKit-Based SDK
- 2DeepSeek Open-Sources DeepInfer and API Components, With Model and Hardware Limits
- 3OpenAI Acquires Orchestra Bio to Expand AI Tools for Drug Development
- 4Google Rolls Out Gemini Personal Intelligence to Android Auto, With Google Built-In Cars to Follow
Tomorrow at 08:00, a briefing like this arrives in your inbox.
The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue · 2026-10-20