Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resource leak in tests #35

Open
iasoon opened this issue Aug 13, 2022 · 3 comments
Open

Resource leak in tests #35

iasoon opened this issue Aug 13, 2022 · 3 comments
Labels
bug Something isn't working question Further information is requested

Comments

@iasoon
Copy link
Member

iasoon commented Aug 13, 2022

When running the test suite, the following message appears:

2022-08-12 20:12:41,753	WARNING worker.py:1404 -- WARNING: 46 PYTHON worker processes have been started on node: 711ff7ca6e04ebd43f741dc793cac210f317e2cb1a237efd6ed17789 with address: 10.1.0.30. This could be a result of using a large number of actors, or due to tasks blocked in ray.get() calls (see https://github.com/ray-project/ray/issues/3644 for some discussion of workarounds).

As the number of worker processes keeps increasing as the test suite progresses, it looks like we have a resource leak somewhere in our code.

@pabloem
Copy link
Collaborator

pabloem commented Aug 15, 2022

hmmm thanks for finding that. I'll try to debug this week...

@pabloem
Copy link
Collaborator

pabloem commented Aug 31, 2022

This should clean up all Ray resouces after every test:
#40

Note there's the IMPULSE element that we were creating in a 'global' context.

@pdames
Copy link
Member

pdames commented Jan 11, 2023

@iasoon @pabloem Do we still have known issues with resource leaks during test runs? Does this need further investigation or is it ready to be closed?

@pdames pdames added bug Something isn't working question Further information is requested labels Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants