Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comm reference fixes + Remove __del__ method and add weakref.finalizer #712

Merged
merged 15 commits into from
Jan 17, 2024

Conversation

JDBetteridge
Copy link
Collaborator

@JDBetteridge JDBetteridge commented Nov 14, 2023

This was intended to be just checking current status of PyOP2 but I have discovered issues with the way the @atexit finalizer is cleaning up comms.

@JDBetteridge JDBetteridge changed the title Turn on debugging Comm reference fixes Nov 15, 2023
@JDBetteridge
Copy link
Collaborator Author

JDBetteridge commented Nov 15, 2023

I think this is now the output I expect at the end of testing:

STATE0
PYOP2 Communicator reference counts:
| Communicator name                      | Count |
==================================================
| PYOP2_COMM_WORLD_DUP                   |     1 |
| PYOP2_COMM_WORLD_DUP_COMPILATION       |     2 |
| PYOP2_COMM_SELF_DUP                    |     1 |

Freeing PYOP2_COMM_WORLD
Deleting innercomm keyval on PYOP2_COMM_WORLD
Removing compilation comm on inner comm
DECREF PYOP2_COMM_WORLD_DUP_COMPILATION TO 1
Deleting compilationcomm keyval on PYOP2_COMM_WORLD_DUP
STATE1
PYOP2 Communicator reference counts:
| Communicator name                      | Count |
==================================================
| PYOP2_COMM_SELF_DUP                    |     1 |

Freeing PYOP2_COMM_SELF
Deleting innercomm keyval on PYOP2_COMM_SELF
STATE2
PYOP2 Communicator reference counts:
| Communicator name                      | Count |
==================================================

More generally at the end of any script STATE0 can have:

  • 1 reference to PYOP2_COMM_WORLD_DUP and up to 2 references to PYOP2_COMM_WORLD_DUP_COMPILATION.
  • Likewise, 1 reference to PYOP2_COMM_SELF_DUP and up to 2 references to PYOP2_COMM_SELF_DUP_COMPILATION.

In its current state it looks like we can get rid of the "ignore all potential sources of error if PYOP2_FINALIZED == True", which would be nice!

Copy link
Collaborator

@connorjward connorjward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is OK but I don't 100% understand what is going on/why this change was necessary.

@JDBetteridge JDBetteridge changed the title Comm reference fixes Comm reference fixes + Remove __del__ method and add weakref.finalizer Dec 12, 2023
connorjward
connorjward previously approved these changes Dec 13, 2023
Copy link
Collaborator

@connorjward connorjward left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks fine.

@dham dham merged commit ad0c430 into master Jan 17, 2024
4 checks passed
@dham dham deleted the JDBetteridge/check_something branch January 17, 2024 16:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants