Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SacessOptimizer: Use cloudpickle for passing data to workers #1467

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

dweindl
Copy link
Member

@dweindl dweindl commented Sep 16, 2024

Cloudpickle is able to handle more complex objects than pickle.

See #1465
Closes #1465

If the new process are forked, we could skip the pickling, but at this point, I don't think that's necessary.

Cloudpickle is able to handle more complex objects than pickle.

See ICB-DCM#1465
Closes ICB-DCM#1465

If the new process are forked, we could skip the pickling, but at this point, I don't think that's necessary.
@codecov-commenter
Copy link

codecov-commenter commented Sep 16, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.43%. Comparing base (b88356f) to head (a63f8ce).

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1467      +/-   ##
===========================================
+ Coverage    83.40%   83.43%   +0.03%     
===========================================
  Files          160      160              
  Lines        13502    13505       +3     
===========================================
+ Hits         11261    11268       +7     
+ Misses        2241     2237       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dweindl dweindl marked this pull request as ready for review September 16, 2024 19:00
Copy link
Collaborator

@PaulJonasJost PaulJonasJost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to perhaps test with specifically a model where it previously failed (e.g. Schwen_PONE2014 from #1465)?

@dweindl
Copy link
Member Author

dweindl commented Sep 17, 2024

Does it make sense to perhaps test with specifically a model where it previously failed (e.g. Schwen_PONE2014 from #1465)?

Generally, yes. I already have that lying around. But it will be another slow test due to model import, that's why I left it out so far.

@PaulJonasJost
Copy link
Collaborator

Does it make sense to perhaps test with specifically a model where it previously failed (e.g. Schwen_PONE2014 from #1465)?

Generally, yes. I already have that lying around. But it will be another slow test due to model import, that's why I left it out so far.

As we always have the problem to balance between "elaborate" test vs time. Would it make sense to have a set of tests that only run in the (I think weekly) master test?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants