Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Skip layerNorm layer weight quantitation while conversion #85

Open
siddhawan opened this issue Aug 29, 2023 · 0 comments
Open

Skip layerNorm layer weight quantitation while conversion #85

siddhawan opened this issue Aug 29, 2023 · 0 comments

Comments

@siddhawan
Copy link

I am trying to convert my torchscript module to tensorrt using torch_tensorrt.compile is there any argument to skip the layers which shows warnings while converting.
This is the warning its giving.
WARNING: [Torch-TensorRT TorchScript Conversion Context] - Running layernorm after self-attention in FP16 may cause overflow. Exporting the model to the latest available ONNX opset (later than opset 17) to use the INormalizationLayer, or forcing layernorm layers to run in FP32 precision can help with preserving accuracy. Although I am not converting to onnx at my end as I am using torch_tensorrt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant