Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

clone so3 embedding object #781

Merged
merged 1 commit into from
Jul 30, 2024
Merged

Conversation

rayg1234
Copy link
Collaborator

@rayg1234 rayg1234 commented Jul 25, 2024

1 line change, this is needed for activation checkpointing to work if we want the interface to remain passing SO3_embedding objects to the transformer. In normal operation the code is not wrong but error prone because the function modifies the input inplace, which the user does not expect. See https://fairwandb.org/fairchem/fm_testing/reports/Training-Instability-of-Large-EQV2-Models--Vmlldzo0MzI5NQ for more details

The alternative solution is to change the transformer.forward signature to take tensors instead but this introduces more breaking changes in the code

@rayg1234 rayg1234 marked this pull request as ready for review July 25, 2024 00:15
@rayg1234 rayg1234 requested review from misko and yilunliao July 25, 2024 00:16
Copy link

codecov bot commented Jul 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Files Coverage Δ
...hem/core/models/equiformer_v2/transformer_block.py 80.95% <100.00%> (ø)

@rayg1234 rayg1234 enabled auto-merge July 29, 2024 23:33
@rayg1234 rayg1234 added this pull request to the merge queue Jul 30, 2024
Merged via the queue into main with commit 3c0fade Jul 30, 2024
7 checks passed
@rayg1234 rayg1234 deleted the rgao_clone_embedding_trans_block branch July 30, 2024 18:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants