Open PRs to associated ACCESS-NRI/<model>-configs repo upon Pre/Release

[CodeGat]

It's a lot of work, opening PRs to update exe paths in a models <model>-configs repository - especially if you are doing Prereleases. We should make this somewhat automated.

Solution Space:

• For Prerelease model builds that do have an associated <model>-configs repo - allow a new comment command !configs config... that opens PRs into dev-<config> for each config. This will also set off the associated QA checks, but NOT repro checks. We might want to open PRs from dev-<config> into release-<config>, but that could be computationally expensive, and would require an admin PAT to bypass the branch -> dev -> release pipeline we have going. As it stands now, we might need a PAT that is able to open new PRs across repos, that will also trigger associated workflows.
• For Prerelease model builds that do NOT have an associated <model>-configs repo - maybe just leave a comment saying to link up a configs repo?
• For Release model builds that do have an associated <model>-configs repo - on merge, should we automatically open PRs to all associated configs? At that point, people might not have to do config PRs at all.
• For Release model builds that do NOT have an associated <model>-configs repo - do nothing. Leave it to the humans!

Things to note:

• How will we know what model corresponds to what <model>-configs repo? For example, access-esm-configs doesn't match perfectly with access-esm-1.5. This might have to be a model-specific config item.
• We might need to lock those comment commands down to administrators since they could be abused.
• We need logic to update exe paths in the associated PRs. This could be complex, and should almost certainly be a separated reusable workflow or action.
  • This might have links to Changing model versions is onerous and error prone model-config-tests#10

References ACCESS-NRI/ACCESS-OM2#60 (comment)

[aidanheerdegen]

• We need logic to update exe paths in the associated PRs. This could be complex, and should almost certainly be a separated reusable workflow or action.

  • This might have links to Changing model versions is onerous and error prone access-om2-configs#77

Yes that is very relevant. If/when we start using module load/model-version it will be trivial to update versions programmatically. So yet another reason to change to that approach.

[aidanheerdegen]

How will we know what model corresponds to what <model>-configs repo? For example, access-esm-configs doesn't match perfectly with access-esm-1.5. This might have to be a model-specific config item.

You mean there isn't a consistent naming scheme? The ESM1.5 configs repo is access-nri/access-esm1.5-configs. The model repo doesn't exist yet, but will be access-nri/access-esm1.5.

I think that is consistent with access-nri/access-om2-configs and access-nri/access-om2.

We might need to lock those comment commands down to administrators since they could be abused.

I suppose. Or a team, which we could then add or remove people from depending on the requirements of their role.

[aidanheerdegen]

For Prerelease model builds that do have an associated <model>-configs repo - allow a new comment command !configs config... that opens PRs into dev-<config> for each config.

I'd like something that was a bit more explicit about the purpose of the "command". e.g.

!test-config
!update-config
!pr-config

are possibilities. I'm not sold of any of them TBH, so suggestions welcome.

We could have an associated repo defined, either in a config file, or in an environment var? In some ways that might be desirable, to limit the possible annoyance of setting of PRs to unrelated repos, either maliciously or accidentally.

I imagine something like this for the actual command:

!update-config dev-1deg_jra55_ryf

This will also set off the associated QA checks, but NOT repro checks. We might want to open PRs from dev-<config> into release-<config>, but that could be computationally expensive, and would require an admin PAT to bypass the branch -> dev -> release pipeline we have going. As it stands now, we might need a PAT that is able to open new PRs across repos, that will also trigger associated workflows.

It's a good point, and potentially a pitfall of the architecture we've chosen with separate dev and release branches.

I think it's reasonable to push directly to dev and then make a PR from there to release.

Another option is to be able to invoke the repro tests directly from any branch using a comment command. This might be a desirable feature in any case. For the test I was about to do for the update software stack, I'd like to test that before merging into dev.

[aidanheerdegen]

For Release model builds that do have an associated <model>-configs repo - on merge, should we automatically open PRs to all associated configs? At that point, people might not have to do config PRs at all.

Yes automating much of this drudge work should be a goal. I think on merge it is reasonable to expect the configurations would be updated at that point.

If we did have some configuration file(s) for this then the "associated configs" could be defined there, either an explicit and/or regex to match branches against.

[aidanheerdegen]

I had a thought: it might make sense to have the workflow to update a configuration callable from the configuration repository. It would make it easy then to update a configuration to the latest model version in any branch just with a comment command.

Knowing which model repo is required can be determined by inspection of the config.yaml (the module load step) or (probably better) the model parameter in the metadata.yaml.

Then automatically updating configurations based on a new release would involve creating a branch for each config and adding the update comment command.