Move automation default vars into seperate scenario files #375

raukadah · 2024-08-27T13:43:22Z

#374 adds the trigger job which will run different Baremetal VA jobs downstream on different architecture file changes.

Currently automation/vars/default.yaml contains different multiple scenarios, stored in a single file. Trigger job may run unwanted jobs downstream without testing the proper architecture changes.

By moving automations vars into different scenario files allow us to run selective trigger job and test the proper prs.

openstack-k8s-operators/architecture#375 splits the default vars into different scenario files. We need to fix the reproducer to adapt this change. This pull-request adds cifmw_arch_automation_file var to include different scenario files. Depends-On: openstack-k8s-operators/architecture#375 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

abays · 2024-08-29T08:43:48Z

This looks fine to me. Do we need to merge a change to CIFMW in parallel, @cjeanner ?

cjeanner · 2024-08-29T09:00:04Z

@abays we indeed need a change on our side. @raukadah has proposed one already (openstack-k8s-operators/ci-framework#2267), but we can make it better by ensuring the file name matches the scenario name.

BTW, we might want to get a new linter here, in architecture repo, to ensure files only have ONE scenario, and that file name matches the scenario. WDYT?

abays · 2024-08-29T09:01:43Z

@cjeanner A new linter as you suggest sounds like a solid idea 👍

cjeanner · 2024-08-29T09:07:31Z

@abays I think we can edit the YAML schema check to get only one entry in vas. Then, we should be able to add a new thing regarding the filename vs scenario name. I can have a look once this PR is in.

openstack-k8s-operators/architecture#375 splits the default vars into different scenario files. We need to fix the reproducer to adapt this change. This pull-request uses cifmw_architecture_scenario to set proper automation file. Depends-On: openstack-k8s-operators/architecture#375 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

tosky · 2024-08-29T11:06:09Z

Shouldn't this change be done in multiple steps?

add the split files
migrated the users
remove the default file

cjeanner · 2024-08-29T11:14:51Z

I have a linter that seems to do what we want.
For instance:

AssertionError: !! bgp.yaml does not match bgp_dt01.yaml

and yamele is also unhappy:

yamale -s .ci/automation-schema.yaml automation/vars
Finding yaml files...
Found 15 yaml files.
Validating...
Validation failed!
Error validating data '/home/cjeanner/work/github.com/openstack-k8s-operators/architecture/automation/vars/bgp.yaml' with schema '.ci/automation-schema.yaml'

So we're getting to a nice place.

This is in addition to a coming PR, #375. This change here is allowing to ensure we follow those practices: - only one scenario per file - filename matches "scenario_name.yaml" pattern

cjeanner · 2024-08-29T11:18:55Z

#379 FYI (it's independent from that PR)

abays · 2024-08-29T14:00:32Z

Shouldn't this change be done in multiple steps?
1. add the split files

2. migrated the users

3. remove the default file

That might be the smoother way to transition this. What do you think @raukadah and @cjeanner ?

openstack-k8s-operators#374 adds the trigger job which will run different Baremetal VA jobs downstream on different architecture file changes. Currently automation/vars/default.yaml contains different multiple scenarios, stored in a single file. Trigger job may run unwanted jobs downstream without testing the proper architecture changes. By moving automations vars into different scenario files allow us to run selective trigger job and test the proper prs. Let's keep the defaults.yaml file till we finish the migration. Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

This is in addition to a coming PR, #375. This change here is allowing to ensure we follow those practices: - only one scenario per file - filename matches "scenario_name.yaml" pattern

openstack-k8s-operators/architecture#375 splits the default vars into different scenario files. We need to fix the reproducer to adapt this change. This pull-request uses cifmw_architecture_scenario to set proper automation file. It drops cifmw_arch_automation_file params from scenario files. Depends-On: openstack-k8s-operators/architecture#375 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

cjeanner · 2024-08-30T06:28:58Z

@abays @raukadah proposal sounds good indeed, allowing to not have to hurry-merge the ci-framework side.
I think the current code state is doing that now, correct?

So we can go with it. My linter is ready, once we're good with the migration, we'll be able to merge it and make things consistent. It can also be used to test the cleaning PR, btw :).

raukadah · 2024-08-30T06:44:58Z

@abays @raukadah proposal sounds good indeed, allowing to not have to hurry-merge the ci-framework side. I think the current code state is doing that now, correct?

So we can go with it. My linter is ready, once we're good with the migration, we'll be able to merge it and make things consistent. It can also be used to test the cleaning PR, btw :).

sounds good to me.

This is in addition to a coming PR, #375. This change here is allowing to ensure we follow those practices: - only one scenario per file - filename matches "scenario_name.yaml" pattern

openstack-k8s-operators/architecture#375 splits the default vars into different scenario files. We need to fix the reproducer to adapt this change. This pull-request uses cifmw_architecture_scenario to set proper automation file. It drops cifmw_arch_automation_file params from scenario files. Depends-On: openstack-k8s-operators/architecture#375 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

Move bgp_dt01 into its own automation file #375 splits the default vars into different scenario files. We need to follow similar pattern for bgp_dt01 also. Depends-On: openstack-k8s-operators/ci-framework#2267 Reviewed-by: Marios Andreou

openstack-k8s-operators#375 splits the default vars into different scenario files. openstack-k8s-operators/ci-framework#2267 suggests to use Use cifmw_architecture_scenario to set proper automation file. So we are no longer needing default.yaml file. Depends-On: openstack-k8s-operators/ci-framework#2267 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

#375 splits the default vars into different scenario files. openstack-k8s-operators/ci-framework#2267 suggests to use Use cifmw_architecture_scenario to set proper automation file. So we are no longer needing default.yaml file. Depends-On: openstack-k8s-operators/ci-framework#2267 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

This is in addition to a coming PR, #375. This change here is allowing to ensure we follow those practices: - only one scenario per file - filename matches "scenario_name.yaml" pattern

Drop default.yaml file #375 splits the default vars into different scenario files. openstack-k8s-operators/ci-framework#2267 suggests to use Use cifmw_architecture_scenario to set proper automation file. So we are no longer needing default.yaml file. Depends-On: openstack-k8s-operators/ci-framework#2292 Reviewed-by: Cédric Jeanneret

This is in addition to a coming PR, #375. This change here is allowing to ensure we follow those practices: - only one scenario per file - filename matches "scenario_name.yaml" pattern

openstack-k8s-operators#375 splits the default vars into different scenario files. We need to follow similar pattern for bgp_dt01 also. Depends-On: openstack-k8s-operators/ci-framework#2267 Signed-off-by: Chandan Kumar (raukadah) <[email protected]> (cherry picked from commit 116ab90)

ciecierski · 2024-09-09T12:25:18Z

When are you planing to land this change in 18.0.0-proposed ?

fultonj · 2024-09-09T14:03:17Z

@raukadah do you want to cherry pick this to 18.0.0-proposed as described here?

https://github.com/openstack-k8s-operators/architecture/blob/main/docs/contributing/cherry-picking.md

Same for these two?

#383
#379

raukadah · 2024-09-10T02:25:39Z

@ciecierski @fultonj Hello, Since we merged #383 and #379. we saw breakage in downstream jobs due to mixing of upstream and downstream content. We fixed that.
I think it is safe to backport it. But I need acks from @cjeanner @eduolivares @eshulman2 before doing that to avoid breakage.

raukadah · 2024-09-10T07:40:01Z

After discussing with @cjeanner , we need to cherry-pick following prs

Use cifmw_architecture_scenario to set proper automation file ci-framework#2267 - Use cifmw_architecture_scenario to set proper automation file
Move automation default vars into seperate scenario files #375 - Move automation default vars into seperate scenario files
Drop default.yaml file #383 - Drop default.yaml file
Ensure we have one scenario per automation file #379 - Ensure we have one scenario per automation file
Move bgp_dt01 into its own automation file #384 - Move bgp_dt01 into its own automation file

It backports following patches: openstack-k8s-operators#375 - Move automation default vars into seperate scenario files openstack-k8s-operators#383 - Drop default.yaml file openstack-k8s-operators#379 - Ensure we have one scenario per automation file openstack-k8s-operators#384 - Move bgp_dt01 into its own automation file Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

openstack-k8s-operators/architecture#375 splits the default vars into different scenario files. We need to fix the reproducer to adapt this change. This pull-request uses cifmw_architecture_scenario to set proper automation file. It drops cifmw_arch_automation_file params from scenario files. Depends-On: openstack-k8s-operators/architecture#375 Note: it excludes zuul.d/architecture-jobs.yaml changes. Cherry-picked from 07a6146#diff-693e3f6ce03f7b6994b1195bcced3d58c89f1c286667356f827c3aecd4518ef6 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

raukadah · 2024-09-10T08:13:10Z

Cherry-pick patches

Feel free to test it before merging.

It backports following patches: openstack-k8s-operators#375 - Move automation default vars into seperate scenario files openstack-k8s-operators#383 - Drop default.yaml file openstack-k8s-operators#379 - Ensure we have one scenario per automation file openstack-k8s-operators#384 - Move bgp_dt01 into its own automation file openstack-k8s-operators#373 - NFV-HCI minor fixes Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

openstack-k8s-operators/architecture#375 splits the default vars into different scenario files. We need to fix the reproducer to adapt this change. This pull-request uses cifmw_architecture_scenario to set proper automation file. It drops cifmw_arch_automation_file params from scenario files. Depends-On: openstack-k8s-operators/architecture#375 Note: it excludes zuul.d/architecture-jobs.yaml changes. Cherry-picked from 07a6146#diff-693e3f6ce03f7b6994b1195bcced3d58c89f1c286667356f827c3aecd4518ef6 Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

Backport default.yaml split patches It backports following patches: #375 #383 #379 #384 #373 Reviewed-by: Ella Shulman <[email protected]> Reviewed-by: Cédric Jeanneret

openstack-k8s-operators/architecture#383 drops defaults.yaml scenario file in favor of openstack-k8s-operators/architecture#375. #2267 suggests to use Use cifmw_architecture_scenario to set proper automation file. Whereever default.yaml is used, the job broke. This pr fixes the same using cifmw_architecture_scenario file. Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

openstack-k8s-operators/architecture#383 drops defaults.yaml scenario file in favor of openstack-k8s-operators/architecture#375. openstack-k8s-operators#2267 suggests to use Use cifmw_architecture_scenario to set proper automation file. Whereever default.yaml is used, the job broke. This pr fixes the same using cifmw_architecture_scenario file. Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

openstack-k8s-operators/architecture#383 drops defaults.yaml scenario file in favor of openstack-k8s-operators/architecture#375. #2267 suggests to use Use cifmw_architecture_scenario to set proper automation file. Whereever default.yaml is used, the job broke. This pr fixes the same using cifmw_architecture_scenario file. Signed-off-by: Chandan Kumar (raukadah) <[email protected]>

openshift-ci bot requested review from abays and cjeanner August 27, 2024 13:43

openshift-ci bot added the approved label Aug 27, 2024

raukadah force-pushed the split_default_automation branch from ab6d3e1 to 499cc3f Compare August 27, 2024 14:12

This was referenced Aug 27, 2024

Use cifmw_architecture_scenario to set proper automation file openstack-k8s-operators/ci-framework#2267

Merged

Enable Downstream trigger job in the architecture repo #374

Merged

raukadah force-pushed the split_default_automation branch 3 times, most recently from 3bc9ccd to 8380941 Compare August 29, 2024 07:55

cjeanner mentioned this pull request Aug 29, 2024

Ensure we have one scenario per automation file #379

Merged

raukadah force-pushed the split_default_automation branch from 8380941 to 0e73ae8 Compare August 30, 2024 05:06

raukadah force-pushed the split_default_automation branch from 0e73ae8 to 216f1cc Compare August 30, 2024 06:23

cjeanner mentioned this pull request Sep 3, 2024

Use cifmw_architecture_scenario to set proper automation file openstack-k8s-operators/ci-framework#2292

Merged

raukadah mentioned this pull request Sep 10, 2024

Backport default.yaml split patches #394

Merged

raukadah mentioned this pull request Sep 10, 2024

Use cifmw_architecture_scenario to set proper automation file openstack-k8s-operators/ci-framework#2333

Merged

raukadah mentioned this pull request Sep 12, 2024

Use cifmw_architecture_scenario file in set va scenario openstack-k8s-operators/ci-framework#2342

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move automation default vars into seperate scenario files #375

Move automation default vars into seperate scenario files #375

raukadah commented Aug 27, 2024

abays commented Aug 29, 2024

cjeanner commented Aug 29, 2024

abays commented Aug 29, 2024

cjeanner commented Aug 29, 2024

tosky commented Aug 29, 2024

cjeanner commented Aug 29, 2024

cjeanner commented Aug 29, 2024

abays commented Aug 29, 2024

cjeanner commented Aug 30, 2024

raukadah commented Aug 30, 2024

ciecierski commented Sep 9, 2024

fultonj commented Sep 9, 2024

raukadah commented Sep 10, 2024

raukadah commented Sep 10, 2024 •

edited

Loading

raukadah commented Sep 10, 2024

Move automation default vars into seperate scenario files #375

Move automation default vars into seperate scenario files #375

Conversation

raukadah commented Aug 27, 2024

abays commented Aug 29, 2024

cjeanner commented Aug 29, 2024

abays commented Aug 29, 2024

cjeanner commented Aug 29, 2024

tosky commented Aug 29, 2024

cjeanner commented Aug 29, 2024

cjeanner commented Aug 29, 2024

abays commented Aug 29, 2024

cjeanner commented Aug 30, 2024

raukadah commented Aug 30, 2024

ciecierski commented Sep 9, 2024

fultonj commented Sep 9, 2024

raukadah commented Sep 10, 2024

raukadah commented Sep 10, 2024 • edited Loading

raukadah commented Sep 10, 2024

raukadah commented Sep 10, 2024 •

edited

Loading