Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add loss_backward_retain_graph to __init__() #856

Closed

Conversation

stephenyan1231
Copy link
Contributor

Summary:
Mask2Former (M2F) Executorch QAT model has its 5 top-level submodules prepared separately (https://fburl.com/code/44qk8qu3).
This is because the model graph during a) QAT training b) QAT evaluation c) ET model export time are different.

  • We empirically find to train such ET QAT model, we need to turn on loss.backward(retain_graph=True) in train step. Otherwise, the training step will fail as in P1447579952.
  • Thus, we add a new loss_backward_retain_graph to AutoUnit.init() to allow the user to have control on retain_graph kwargs.
  • Note this change is back-compatible.

Differential Revision: D58901158

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D58901158

@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D58901158

stephenyan1231 added a commit to stephenyan1231/tnt that referenced this pull request Jun 26, 2024
Summary:
Pull Request resolved: pytorch#856

Expose **retain_graph** kwarg in **loss.backward()** by adding a new argument **loss_backward_retain_graph** to **AutoUnit.__init__()**

Differential Revision: D58901158
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D58901158

stephenyan1231 added a commit to stephenyan1231/tnt that referenced this pull request Jun 27, 2024
Summary:
Pull Request resolved: pytorch#856

Expose **retain_graph** kwarg in **loss.backward()** by adding a new argument **loss_backward_retain_graph** to **AutoUnit.__init__()**

Differential Revision: D58901158
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D58901158

stephenyan1231 added a commit to stephenyan1231/tnt that referenced this pull request Jun 27, 2024
Summary:
Pull Request resolved: pytorch#856

Expose **retain_graph** kwarg in **loss.backward()** by adding a new argument **loss_backward_retain_graph** to **AutoUnit.__init__()**

Differential Revision: D58901158
Summary:
Pull Request resolved: pytorch#856

Expose **retain_graph** kwarg in **loss.backward()** by adding a new argument **loss_backward_retain_graph** to **AutoUnit.__init__()**

Differential Revision: D58901158
@facebook-github-bot
Copy link
Contributor

This pull request was exported from Phabricator. Differential Revision: D58901158

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants