Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Override constrain_resources() in individual steps #481

Merged
merged 9 commits into from
Dec 20, 2022

Conversation

xylar
Copy link
Collaborator

@xylar xylar commented Dec 18, 2022

This merge drops calls to Testcase.run() and instead override step.constrain_resources() in individual steps.

We have deprecated Testcase.run() because its use is not compatible with the task parallelism approach we (@altheaden and I) are developing. Instead, the individual steps in a test case that need to set the number of tasks or cpus per task should override constrain_resources(), set self.ntasks, self.min_tasks, etc., and then call super().constrian_resources().

In the near future (again related to task parallelism), it will also be useful to override step.runtime_setup() for tasks like creating graph partitions before running an MPAS component. In anticipation of this change and to be consistent with the deprecation warning about Testcase.run(), this merge adds a call to runtime_setup() when a step gets run. Currently, no steps override this function.

Checklist

  • Developer's Guide has been updated
  • API documentation in the Developer's Guide (api.rst) has any new or modified class, method and/or functions listed
  • Document (in a comment titled Testing in this PR) any testing that was used to verify the changes

@xylar xylar added ocean in progress This PR is not ready for review or merging framework labels Dec 18, 2022
@xylar xylar self-assigned this Dec 18, 2022
@xylar xylar changed the title Drop calls to Testcase.run() and instead override constrain_resources() in individual steps Override constrain_resources() in individual steps Dec 18, 2022
@xylar
Copy link
Collaborator Author

xylar commented Dec 18, 2022

Testing

I ran the pr test suite on Chrysalis with Intel and OpenMPI in optimized mode, using the current master as a baseline.

I have also run the remaining test groups that aren't part of the pr suite on their own.

Tests from test group run and are BFB (if there is a baseline comparison):

  • GlobalConvergence
  • GlobalOcean
  • Hurricane
  • IsomipPlus
  • PlanarConvergence
  • SphereTransport
  • Tides

Manually changing resources before calling compass run has the desired effect:

  • GlobalConvergence
  • GlobalOcean
  • Hurricane
  • IsomipPlus
  • PlanarConvergence
  • SphereTransport
  • Tides

@xylar
Copy link
Collaborator Author

xylar commented Dec 19, 2022

@vanroekel and @sbrus89, this should address the issue with the Tides test group discussed in #456 (comment)

@xylar xylar force-pushed the call_constrain_resources branch 2 times, most recently from 8d72783 to 960d0fc Compare December 19, 2022 11:26
@xylar xylar marked this pull request as ready for review December 19, 2022 11:26
@xylar xylar removed the request for review from cbegeman December 19, 2022 11:26
@xylar
Copy link
Collaborator Author

xylar commented Dec 19, 2022

@sbrus89 and @vanroekel, I'm not quite done testing Hurricane and Tides because they haven't run to completion but I did verify that I could change the resources for both init and forward at runtime.

I would appreciate having you each test as well, though, just to make sure the issue if fixed.

Copy link
Collaborator

@vanroekel vanroekel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested the hurricane and tides test case on chrysalis and confirmed a change to ntasks in the cfg file changed the number of cores used in the run. Thanks @xylar!

@xylar
Copy link
Collaborator Author

xylar commented Dec 19, 2022

Seeing #482 but otherwise everything is working.

@xylar
Copy link
Collaborator Author

xylar commented Dec 19, 2022

Thanks @vanroekel.

@xylar xylar removed the in progress This PR is not ready for review or merging label Dec 20, 2022
@xylar xylar removed the request for review from sbrus89 December 20, 2022 12:18
@xylar
Copy link
Collaborator Author

xylar commented Dec 20, 2022

@sbrus89, I think @vanroekel's testing is sufficient. I'm going to merge.

@xylar xylar merged commit 091224d into MPAS-Dev:master Dec 20, 2022
@xylar xylar deleted the call_constrain_resources branch December 20, 2022 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants