Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.11](backport #3765) Fix index pattern when querying ES and condition when searching logs #3829

Merged
merged 4 commits into from
Nov 30, 2023

Commits on Nov 28, 2023

  1. Fix index pattern when querying ES and condition when searching logs (#…

    …3765)
    
    Two issues were causing flakiness on integration tests and are fixed by this PR:
    1. The pattern used to query ES was not working on serverless, this commit updates it to a pattern that works on both stateful and serverless as well as make it more specific to the indexes/data streams we want to query
    2. `findESDocs` did not wait for the data to be indexed, only to a successful query on ES. In some cases the documents the test wanted were not indexed yet, leading to 0 documents being returned and the test failing with no error in the logs/diagnostics. This is fixed by waiting for a document count > 0 and no error.
    
    (cherry picked from commit a03aa9c)
    
    # Conflicts:
    #	pkg/testing/tools/estools/elasticsearch.go
    #	testing/integration/logs_ingestion_test.go
    belimawr committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    6b34f6c View commit details
    Browse the repository at this point in the history
  2. fix merge conflicts

    belimawr committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    d0b8138 View commit details
    Browse the repository at this point in the history
  3. Remove failing processors from the query for errors

    The `add_cloud_metadata` and some other `add_*_metadata` processors
    are expected to log some errors if they cannot fetch the necessary
    information. It is normal to find their error logs in pretty much any
    deployment, some examples:
     - When Docker is not installed/running `add_docker_metadata` will log
     some errors
     - When the Elastic-Agent is deployed in a non-cloud VM
     `add_cloud_metadata` will log some errors.
    
    This commit removes all those processors from the queries for log
    errors, as they're expected.
    belimawr committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    9faef34 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. add timeout for go test

    Add a 2h timeout for `go test` when run on remote hosts.
    belimawr committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    856565c View commit details
    Browse the repository at this point in the history