Skip to content
K = 0.731

i14 and AWS Adapt Laplace 3B for Training and Serving on Trainium2

1 reports 1 reporting sources 2 hours agoNew

Understand the event

AI overview

i14 and AWS adapted i14’s Laplace 3B linear attention model to train and serve on AWS Trainium2 using native PyTorch through TorchNeuron Native, reporting roughly 120,000 training tokens per second and inference throughput of 11,544 output tokens per second. AWS said its GenAIIC team trained the model on one trn2.48xlarge instance using all 64 logical NeuronCores, with training results reported through about 44 billion tokens. The model maintains a fixed 29.5 MB recurrent state per sequence. For inference, the team used an unmodified vLLM engine on one trn2.48xlarge instance; the reported throughput was measured across 1,280 concurrent sequences.

Generated by AI from reports · Updated 2 hours ago

Latest development10/21 05:14 (UTC+8)
Training and serving i14’s Laplace 3B on AWS Trainium2 with native PyTorchAWS Machine Learning Blog

Report timeline

Follow the reports to explore different sides of the event.

10/21
  1. AWS Machine Learning BlogPicks
    Training and serving i14’s Laplace 3B on AWS Trainium2 with native PyTorch

    i14 and AWS explain how they adapted Laplace 3B’s linear attention architecture for training and serving on AWS Trainium2 using native PyTorch through TorchNeuron Native. The model maintains a fixed 29.5 MB recurrent state per sequence and reached roughly 120,000 training tokens per second, with results reported through about 44 billion tokens. Inference reached 11,544 output tokens per second across 1,280 concurrent sequences on one trn2.48xlarge instance.

Tomorrow at 08:00, a briefing like this arrives in your inbox.

The same kind of briefing as the one above: a few stories worth reading. Free. Mail starts after you confirm, and every letter has a one-click unsubscribe. Read a past issue

Event heat trend

There is not enough continuous observation data to plot a trend yet.

Keep reading

Latest daily · Issue 27TorchNeuron brings native PyTorch support for Hugging Face models on AWS Trainium

Trending now

  1. 1Mistral Releases Voxtral 3 Speech Models, Voice Agents API and LiveKit-Based SDK
  2. 2DeepSeek Open-Sources DeepInfer and API Components, With Model and Hardware Limits
  3. 3OpenAI Acquires Orchestra Bio to Expand AI Tools for Drug Development
  4. 4Google Rolls Out Gemini Personal Intelligence to Android Auto, With Google Built-In Cars to Follow