cleanup test resources

[DaanHoogland]

This is to clean-up tests that create resources and do not clean-up after themselves. The idea is that the entire test-suite should be able to be run in a production environment without harming or messing up. In smoke tests running in Trillian we found the following tests to leave stuff around (the checks fixed in scope of this pr are just for simulator):

• test_accounts cleanup for resources left by test_accounts and test_project #3657 test cleanup #4473
• test_affinity_groups_projects test cleanup #4473
• test_async_job test cleanup #4473
• test_deploy_vm_iso
• test_deploy_vms_with_varied_deploymentplanners default teardown methods with reversed() handling #4289
• test_deploy_vm_with_userdata
• test_diagnostics default teardown methods with reversed() handling #4289
• test_domain_service_offerings
• test_internal_lb Marvin3 #4727
• test_list_ids_parameter
• test_loadbalance
• test_metrics_api
• test_multipleips_per_nic
• test_nested_virtualization
• test_network_acl
• test_network
• test_nic_adapter_type
• test_password_server
• test_portforwardingrules
• test_privategw_acl
• test_projects cleanup for resources left by test_accounts and test_project #3657
• test_reset_vm_on_reboot
• test_resource_accounting
• test_router_dhcphosts cleanup of unused code and cleanup of cleanup procedure #5562
• test_router_dns
• test_router_dnsservice
• test_routers_iptables_default_policy
• test_routers_network_ops
• test_routers default teardown methods with reversed() handling #4289
• test_service_offerings
• test_snapshots
• test_templates
• test_vm_deployment_planner smoke test: fix test_vm_deployment_planner #7183 - some work might be remaining due to the way VMs are created in this test-suite
• test_vpc_redundant
• test_vpc_router_nics
• test_vpc_vpn default teardown methods with reversed() handling #4289
• test_hostha_kvm cleanup some smoke tests #8196 - disabling and enabling host HA, not sure if this should be a smoke test!?

ISSUE TYPE

• Bug Report
• Other: test improvements

[rohityadavcloud]

@DaanHoogland is this still valid or can be closed?

[DaanHoogland]

no @rhtyd , it is still very valid. It is also limited to simulator tests and should be expanded to all tests. I will expand on this the coming weeks (given 4.15 is not asking more attention)

[nvazquez]

@DaanHoogland is this still valid?

[DaanHoogland]

yes it is, there is a lot of antiquated code in the smoke tests still, as they are all fixed/passing it is low priority.

[DaanHoogland]

also it has been touched since a year ago, so removing the 1year old label

[rohityadavcloud]

Maybe review/close or update this issue @DaanHoogland ?

[DaanHoogland]

@rhtyd @nvazquez this will be open for a long time unless it's getting priority, which it won't as it concerned data and entities being left around in test environments when running smoke tests. I've updated the list and am addressing some for component tests and will address any further I encounter during other work.
Maybe there should be a milestone "ongoing" or something?

[nvazquez]

Thanks @DaanHoogland, I've added the 'work in-progress' tag and moved it for next milestone

[nvazquez]

Hi @DaanHoogland any plans to work on this on 4.17?

[DaanHoogland]

Maybe given time I will pick up a few files again, @nvazquez

[shwstppr]

@DaanHoogland do you have any work planned for 4.17.1 or should this be moved to the next milestone?

[DaanHoogland]

moving to the next milestone @shwstppr even if I may do some files. I will attach the PR to a feasible milestone.