From 5973d83d28acacdef9523812b934b244191345a7 Mon Sep 17 00:00:00 2001 From: Md Fahim Faysal Khan Date: Tue, 3 Sep 2024 13:49:29 -0700 Subject: [PATCH] fixed a small grammatical error --- rosetta/docs/NATIVE_FP8.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rosetta/docs/NATIVE_FP8.md b/rosetta/docs/NATIVE_FP8.md index 127b3ca4c..069b06fdd 100644 --- a/rosetta/docs/NATIVE_FP8.md +++ b/rosetta/docs/NATIVE_FP8.md @@ -123,7 +123,7 @@ python -m paxml.main \ ... ``` -Please not that disabling the triton gemm and pipelined collectives are essential for enabling the FP8 functionality and performance. +Please not that disabling the triton gemm and pipelined collectives is essential for enabling the FP8 functionality and performance. ## Transformer Engine vs Native FP8 Support Native XLA-FP8 specifically targets matrix multiplication operations. In contrast, the Transformer Engine focuses on enhancing the overall performance of the entire transformer layer. This encompasses not only the FP8 matrix multiplication but also attention mechanisms, layer normalizations, and other components.