Skip to content

Commit

Permalink
fixed a small grammatical error
Browse files Browse the repository at this point in the history
  • Loading branch information
kocchop committed Sep 3, 2024
1 parent 76f222a commit 5973d83
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion rosetta/docs/NATIVE_FP8.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ python -m paxml.main \
...
```

Please not that disabling the triton gemm and pipelined collectives are essential for enabling the FP8 functionality and performance.
Please not that disabling the triton gemm and pipelined collectives is essential for enabling the FP8 functionality and performance.

## Transformer Engine vs Native FP8 Support
Native XLA-FP8 specifically targets matrix multiplication operations. In contrast, the Transformer Engine focuses on enhancing the overall performance of the entire transformer layer. This encompasses not only the FP8 matrix multiplication but also attention mechanisms, layer normalizations, and other components.
Expand Down

0 comments on commit 5973d83

Please sign in to comment.