PyTorch 2.15 Release Candidate Adds FP8 Support and Faster MPS Decoding on Macs

PyTorch’s 2.15 release candidate adds FP8 support and faster Metal kernels to its MPS backend for Apple silicon Macs, with Hugging Face’s PyTorch team reporting faster LLM decoding and lower memory use in specific workloads. The release candidate requires macOS 15 or later and Python 3.11 or later. The new torch.float8_e4m3fn tensors on MPS use one byte per value, half the storage of FP16 or BF16. In the team’s benchmark on an M3 Ultra with torch.compile, Qwen3.6-35B-A3B decoding increased from 9.4 tokens/s with PyTorch 2.14 to 61.3 tokens/s with 2.15. The team also reported that moving constant padding to Metal kernels reduced peak driver memory for Wan2.2’s tiled VAE decode from about 270 GiB to 8 GiB.

Generated by AI from reports · Updated 49 minutes ago

There is not enough continuous observation data to plot a trend yet.