Skip to content

Commit

Permalink
Fix flaky TFESAMLSettings resource tests, add comments
Browse files Browse the repository at this point in the history
These tests flake violently, because they're competing for control of global
singleton state. The flakes show a distinctive pattern: they only flake when at
least two of the resource test funcs end up on separate test runner partitions
in CI, and re-running a single one failed partition alone always succeeds
(because there's no concurrent competition anymore).

The fix is to make sure all of these tests ALWAYS run in serial on a single test
runner partition. Smooshing into a single outer func and calling t.Run without
t.Parallel seems like it should fit the bill.
  • Loading branch information
nfagerlund committed Oct 25, 2023
1 parent 0f88ce7 commit d992cd3
Show file tree
Hide file tree
Showing 2 changed files with 213 additions and 194 deletions.
6 changes: 6 additions & 0 deletions internal/provider/data_source_saml_settings_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,12 @@ import (
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/resource"
)

// FLAKE ALERT: SAML settings are a singleton resource shared by the entire TFE
// instance, and any test touching them is at high risk to flake.
// This test is fine, because it only checks that the attributes have SOME
// value. Testing for any _particular_ value would not be viable, because
// `resource_tfe_saml_settings_test.go` exists. See that file for more color on
// this.
func TestAccTFESAMLSettingsDataSource_basic(t *testing.T) {
resourceAddress := "data.tfe_saml_settings.foobar"
resource.Test(t, resource.TestCase{
Expand Down
Loading

0 comments on commit d992cd3

Please sign in to comment.