[8.11](backport #3765) Fix index pattern when querying ES and condition when searching logs #3829
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![mergify[bot]](https://avatars.githubusercontent.com/in/10562?s=60&v=4){kind=small}
mergify
bot
added
the
conflicts
💚 Build Succeeded
Expand to view the summary
Build stats
❕ Flaky test reportNo test was executed to be analysed. 🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
pierrehilbert
requested review from
belimawr
and removed request for
michalpristas and
pchila
|
Pinging @elastic/elastic-agent (Team:Elastic-Agent) |
…3765) Two issues were causing flakiness on integration tests and are fixed by this PR: 1. The pattern used to query ES was not working on serverless, this commit updates it to a pattern that works on both stateful and serverless as well as make it more specific to the indexes/data streams we want to query 2. `findESDocs` did not wait for the data to be indexed, only to a successful query on ES. In some cases the documents the test wanted were not indexed yet, leading to 0 documents being returned and the test failing with no error in the logs/diagnostics. This is fixed by waiting for a document count > 0 and no error. (cherry picked from commit a03aa9c) # Conflicts: # pkg/testing/tools/estools/elasticsearch.go # testing/integration/logs_ingestion_test.go
The `add_cloud_metadata` and some other `add_*_metadata` processors are expected to log some errors if they cannot fetch the necessary information. It is normal to find their error logs in pretty much any deployment, some examples: - When Docker is not installed/running `add_docker_metadata` will log some errors - When the Elastic-Agent is deployed in a non-cloud VM `add_cloud_metadata` will log some errors. This commit removes all those processors from the queries for log errors, as they're expected.
belimawr
force-pushed
the
mergify/bp/8.11/pr-3765
branch
from
ea9aabc
to
9faef34
Compare
|
buildkite test this |
Add a 2h timeout for `go test` when run on remote hosts.
|
SonarQube Quality Gate
|
AndersonQ
approved these changes
Nov 30, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why so many changes on this backport?
Even though this PR is a backport, there were quite some changes to the integration tests and tooling in
main, because of conflicts I took the path of least resistance and copied the new tests and some of the tooling around them, in the end we get some extra test coverage.Another change I had to make was to introduce 9faef34 excluding some
add_*_metadataprocessors from the default query that searches for error logs.add_cloud_metadatawas always logging some errors and I have seen other similar processors logging errors almost every time I run the Elastic-Agent. So I decided to exclude them all by default.The automated message
fixes #3831
This is an automatic backport of pull request #3765 done by Mergify.
Cherry-pick of a03aa9c has failed:
To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally
Mergify commands and options
More conditions and actions can be found in the documentation.
You can also trigger Mergify actions by commenting on this pull request:
@Mergifyio refreshwill re-evaluate the rules@Mergifyio rebasewill rebase this PR on its base branch@Mergifyio updatewill merge the base branch into this PR@Mergifyio backport <destination>will backport this PR on<destination>branchAdditionally, on Mergify dashboard you can:
Finally, you can contact us on https://mergify.com