We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Hi, there is an issue with broadcasting when using batch size > 1 with the line: https://github.com/XLabs-AI/x-flux/blob/main/train_flux_lora_deepspeed.py#L254 the shape of t should be (batch_size, 1, 1) to be broadcasted and multiplied with x_0 and x_t
You can try and fix it this way:
https://github.com/aravindhv10/x-flux/blob/aravind_prodigy_dataset/train_flux_lora_deepspeed.py#L253-L256
or perhaps there might be a better way? The above method worked for me.
The text was updated successfully, but these errors were encountered:
This link provides the details of broadcasting:
https://stackoverflow.com/questions/65121614/pytorch-how-to-multiply-via-broadcasting-of-two-tensors-with-different-shapes
https://pytorch.org/docs/stable/notes/broadcasting.html
Sorry, something went wrong.
Raised PR:
#104
No branches or pull requests
Hi, there is an issue with broadcasting when using batch size > 1 with the line:
https://github.com/XLabs-AI/x-flux/blob/main/train_flux_lora_deepspeed.py#L254
the shape of t should be (batch_size, 1, 1) to be broadcasted and multiplied with x_0 and x_t
You can try and fix it this way:
https://github.com/aravindhv10/x-flux/blob/aravind_prodigy_dataset/train_flux_lora_deepspeed.py#L253-L256
or perhaps there might be a better way? The above method worked for me.
The text was updated successfully, but these errors were encountered: