PyTorch 2.15 brings FP8 and faster MPS decoding to Macs

What happened

Hugging Face’s PyTorch team details MPS improvements in the PyTorch 2.15 release candidate, including FP8 support and faster LLM decoding on Macs. On an M3 Ultra with torch.compile, Qwen3.6-35B-A3B decoding rises from 9.4 to 61.3 tokens/s compared with 2.14. Moving constant padding to Metal kernels also reduces peak driver memory for Wan2.2’s tiled VAE decode from about 270 GiB to 8 GiB.

The story so far

PyTorch’s 2.15 release candidate adds FP8 support and faster Metal kernels to its MPS backend for Apple silicon Macs, with Hugging Face’s PyTorch team reporting faster LLM decoding and lower memory use in specific workloads. The release candidate requires macOS 15 or later and Python 3.11 or later. The new torch.float8_e4m3fn tensors on MPS use one byte per value, half the storage of FP16 or BF16. In the team’s benchmark on an M3 Ultra with torch.compile, Qwen3.6-35B-A3B decoding increased from 9.4 tokens/s with PyTorch 2.14 to 61.3 tokens/s with 2.15. The team also reported that moving constant padding to Metal kernels reduced peak driver memory for Wan2.2’s tiled VAE decode from about 270 GiB to 8 GiB.

Keep reading

Trending now

1. 1DeepSeek Open-Sources DeepInfer Inference Framework, With vLLM Planning to Adopt Its Ideas
2. 2MiniMax Releases M3.1 Flash With Higher Benchmark Score and Lower Task Cost Than M3
3. 3Anthropic Commits 5% of Its Shares to New Public Benefit Fund