Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more quantization support for burn-jit #2275

Merged
merged 21 commits into from
Sep 17, 2024
Merged

Add more quantization support for burn-jit #2275

merged 21 commits into from
Sep 17, 2024

Conversation

laggui
Copy link
Member

@laggui laggui commented Sep 12, 2024

Checklist

  • Confirmed that run-checks all script has been executed.

Changes

Quantization support for burn-jit.

  • Completed QJitTensor implementation w/ contained qparams
    • Multiple quantized values are packed into a u32 tensor representation
  • Added quantize_per_tensor and dequantize_per_tensor cube kernels
    • Handle affine/symmetric quantization with different vectorization factors based on input tensor size
  • Added from_data and into_data conversions
  • Added unit tests

Testing

Unit tests.

Copy link

codecov bot commented Sep 12, 2024

Codecov Report

Attention: Patch coverage is 60.58520% with 229 lines in your changes missing coverage. Please review.

Project coverage is 85.79%. Comparing base (58ce502) to head (d46076e).
Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
...rates/burn-jit/src/kernel/quantization/quantize.rs 44.79% 106 Missing ⚠️
...tes/burn-jit/src/kernel/quantization/dequantize.rs 45.40% 101 Missing ⚠️
crates/burn-jit/src/ops/qtensor.rs 76.66% 21 Missing ⚠️
crates/burn-tensor/src/tensor/data.rs 83.33% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2275      +/-   ##
==========================================
- Coverage   85.92%   85.79%   -0.13%     
==========================================
  Files         750      754       +4     
  Lines       94328    95189     +861     
==========================================
+ Hits        81047    81671     +624     
- Misses      13281    13518     +237     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to change the equality assertion for quantized values since some tests produced very close values on macos (in floating point), but the assertion failed on the quantization parameters values.

Copy link
Member

@nathanielsimard nathanielsimard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@laggui laggui merged commit aa79e36 into main Sep 17, 2024
11 checks passed
@laggui laggui deleted the feat/jit/quantize branch September 17, 2024 14:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants