Skip to content

Commit

Permalink
Fix attention fusion for UNet onnx model export when using LoRA weigh…
Browse files Browse the repository at this point in the history
…ts (microsoft#17249)

### Description
Tested with stable diffusion unet models exported by both pytorch 2.1.0
(nightly) and pytorch 1.13.1, with and without LoRA weights.



### Motivation and Context
LoRA weights modifiy the unet model by adding matmul and scale
operations to every q/k/v/out tensors, which breaks the current MHA
pattern recognition.
  • Loading branch information
PatriceVignola authored Aug 29, 2023
1 parent 761c433 commit 4880f1d
Showing 1 changed file with 673 additions and 23 deletions.
Loading

0 comments on commit 4880f1d

Please sign in to comment.