adjust timeouts so tests have better chance of succeeding #3549

leehinman · 2023-10-05T18:45:23Z

What does this PR do?

adjust timeouts on the Coordinator tests so they have a better chance of succeeding in CI

Why is it important?

Flaky tests in CI

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have made corresponding change to the default configuration files
I have added tests that prove my fix is effective or that my feature works
I have added an entry in ./changelog/fragments using the changelog tool
I have added an integration test or an E2E test

Author's Checklist

[ ]

How to test this PR locally

Related issues

Flaky TestCoordinatorShutdownErrorAllResponses #3538

Use cases

Screenshots

Logs

Questions to ask yourself

How are we going to support this in production?
How are we going to measure its adoption?
How are we going to debug this?
What are the metrics I should take care of?
...

elasticmachine · 2023-10-05T18:45:26Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

elasticmachine · 2023-10-05T19:11:51Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-10-05T21:11:53.232+0000
Duration: 27 min 30 sec

Test stats 🧪

Test	Results
Failed	0
Passed	6485
Skipped	59
Total	6544

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages.
run integration tests : Run the Elastic Agent Integration tests.
run end-to-end tests : Generate the packages and run the E2E Tests.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2023-10-05T19:11:59Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	98.78% (`81/82`)	👍
Files	67.003% (`199/297`)	👍
Classes	65.642% (`363/553`)	👍
Methods	52.93% (`1147/2167`)	👍
Lines	38.475% (`13080/33996`)	👍 0.012
Conditionals	100.0% (`0/0`)	💚

cmacknz · 2023-10-05T20:53:05Z

The buildkite failure is unrelated:

        Enrolling Elastic Agent with Fleet.....{"log.level":"info","@timestamp":"2023-10-05T19:36:58.307Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":479},"message":"Starting enrollment to URL: https://7faba8a44384409e8c7f091cb2c4948a.fleet.us-central1.gcp.foundit.no:443/","ecs.version":"1.6.0"}
        ...{"log.level":"info","@timestamp":"2023-10-05T19:36:59.521Z","log.origin":{"file.name":"cmd/enroll_cmd.go","file.line":275},"message":"Elastic Agent might not be running; unable to trigger restart","error":{"message":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial unix /opt/Elastic/Agent/data/tmp/elastic-agent-control.sock: connect: no such file or directory\""},"ecs.version":"1.6.0"}
        Successfully enrolled the Elastic Agent.

This failure is being addressed in #3207

cmacknz · 2023-10-05T20:54:52Z

internal/pkg/agent/application/coordinator/coordinator_test.go

@@ -335,7 +335,6 @@ func TestCoordinatorShutdownErrorOneResponse(t *testing.T) {
 }

 func TestCoordinatorShutdownErrorAllResponses(t *testing.T) {
-	CoordinatorShutdownTimeout = time.Millisecond


Doesn't this make the value of the shutdown timeout dependent on the order in which tests are executed?

nice catch, I added these back and set to what the default is during normal operations

elastic-sonarqube · 2023-10-05T21:23:41Z

SonarQube Quality Gate

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

* adjust timeouts so tests have better chance of succeeding (cherry picked from commit dcf2263)

) * adjust timeouts so tests have better chance of succeeding (cherry picked from commit dcf2263) Co-authored-by: Lee E Hinman <[email protected]>

adjust timeouts so tests have better chance of succeeding

d104f9f

leehinman added bug Something isn't working Team:Elastic-Agent Label for the Agent team flaky-test Unstable or unreliable test cases. skip-changelog backport-v8.11.0 Automated backport with mergify labels Oct 5, 2023

leehinman requested a review from a team as a code owner October 5, 2023 18:45

leehinman requested review from blakerouse and faec October 5, 2023 18:45

mergify bot assigned leehinman Oct 5, 2023

cmacknz reviewed Oct 5, 2023

View reviewed changes

reset CoordinatorShutdownTimeout to default value

ca6ffb2

faec approved these changes Oct 6, 2023

View reviewed changes

cmacknz approved these changes Oct 6, 2023

View reviewed changes

leehinman merged commit dcf2263 into elastic:main Oct 6, 2023
24 checks passed

leehinman deleted the coordinator_test_fix branch October 6, 2023 20:38

mergify bot pushed a commit that referenced this pull request Oct 6, 2023

adjust timeouts so tests have better chance of succeeding (#3549)

377a112

* adjust timeouts so tests have better chance of succeeding (cherry picked from commit dcf2263)

mergify bot mentioned this pull request Oct 6, 2023

[8.11](backport #3549) adjust timeouts so tests have better chance of succeeding #3561

Merged

pierrehilbert mentioned this pull request Oct 8, 2023

Flaky TestCoordinatorShutdownErrorAllResponses #3538

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

adjust timeouts so tests have better chance of succeeding #3549

adjust timeouts so tests have better chance of succeeding #3549

leehinman commented Oct 5, 2023

elasticmachine commented Oct 5, 2023

elasticmachine commented Oct 5, 2023 •

edited

Loading

Build stats

Test stats 🧪

elasticmachine commented Oct 5, 2023 •

edited

Loading

cmacknz commented Oct 5, 2023

cmacknz Oct 5, 2023

leehinman Oct 5, 2023

elastic-sonarqube bot commented Oct 5, 2023

adjust timeouts so tests have better chance of succeeding #3549

adjust timeouts so tests have better chance of succeeding #3549

Conversation

leehinman commented Oct 5, 2023

What does this PR do?

Why is it important?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

Questions to ask yourself

elasticmachine commented Oct 5, 2023

elasticmachine commented Oct 5, 2023 • edited Loading

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented Oct 5, 2023 • edited Loading

🌐 Coverage report

cmacknz commented Oct 5, 2023

cmacknz Oct 5, 2023

Choose a reason for hiding this comment

leehinman Oct 5, 2023

Choose a reason for hiding this comment

elastic-sonarqube bot commented Oct 5, 2023

elasticmachine commented Oct 5, 2023 •

edited

Loading

elasticmachine commented Oct 5, 2023 •

edited

Loading