Sourced from torch's releases.
PyTorch 2.2: FlashAttention-v2, AOTInductor
PyTorch 2.2 Release Notes
- Highlights
- Backwards Incompatible Changes
- Deprecations
- New Features
- Improvements
- Bug fixes
- Performance
- Documentation
Highlights
We are excited to announce the release of PyTorch® 2.2! PyTorch 2.2 offers ~2x performance improvements to
scaled_dot_product_attention
via FlashAttention-v2 integration, as well as AOTInductor, a new ahead-of-time compilation and deployment tool built for non-python server-side deployments.This release also includes improved torch.compile support for Optimizers, a number of new inductor optimizations, and a new logging mechanism called TORCH_LOGS.
Please note that we are deprecating macOS x86 support, and PyTorch 2.2.x will be the last version that supports macOS x64.
Along with 2.2, we are also releasing a series of updates to the PyTorch domain libraries. More details can be found in the library updates blog.
This release is composed of 3,628 commits and 521 contributors since PyTorch 2.1. We want to sincerely thank our dedicated community for your contributions. As always, we encourage you to try these out and report any issues as we improve 2.2. More information about how to get started with the PyTorch 2-series can be found at our Getting Started page.
Summary:
scaled_dot_product_attention
(SDPA) now supports FlashAttention-2, yielding around 2x speedups compared to previous versions.- PyTorch 2.2 introduces a new ahead-of-time extension of TorchInductor called AOTInductor, designed to compile and deploy PyTorch programs for non-python server-side.
torch.distributed
supports a new abstraction for initializing and representing ProcessGroups called device_mesh.- PyTorch 2.2 ships a standardized, configurable logging mechanism called TORCH_LOGS.
- A number of torch.compile improvements are included in PyTorch 2.2, including improved support for compiling Optimizers and improved TorchInductor fusion and layout optimizations.
- Please note that we are deprecating macOS x86 support, and PyTorch 2.2.x will be the last version that supports macOS x64.
torch.ao.quantization
now offers a prototypetorch.export
based flow
... (truncated)
8ac9b20
Run docker release build on final tag (#117131)
(#117182)2490352
Fix cuInit test on Windows (#117095)3a44bb7
[CI] Test that cuInit is not called during import (#117043)1c8ba38
[CI] Use jemalloc for CUDA builds (#116900)
(#116988)96d2ddb
Store user model to simplify
ONNXProgram.{adapt_torch_*,call} APIs (#1152...738b4a5
Update ONNX's IO Adapter to support FakeTensor with ExportedProgram (#114407)...4cf10bf
[Cherry-pick] [Quant] [PT2] Enable batchnorm in
_move_exported_model_to_eval ...7e97e4b
[AARCH64] Fall back to GEMM if mkldnn_matmul fails (#115936)
(#116666)1a3e3c7
[CUDA] baddmm should fall back to addmm for batch=1 (#114992)
(#116518)ab7505f
Fix broken PyYAML 6.0 on MacOS x86 (#115956)
(#116551)