-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding allowlist setting for ingest-useragent and ingest-geoip processors #15325
Adding allowlist setting for ingest-useragent and ingest-geoip processors #15325
Conversation
… ingest-common. Signed-off-by: Sarat Vemulapalli <[email protected]>
❌ Gradle check result for 3494bb3: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
.../ingest-common/src/test/java/org/opensearch/ingest/common/IngestCommonModulePluginTests.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Sarat Vemulapalli <[email protected]>
Signed-off-by: Sarat Vemulapalli <[email protected]>
❌ Gradle check result for 324be45: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
❌ Gradle check result for 1120f02: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #15325 +/- ##
============================================
- Coverage 71.96% 71.91% -0.06%
+ Complexity 63300 63238 -62
============================================
Files 5214 5214
Lines 295943 295979 +36
Branches 42754 42756 +2
============================================
- Hits 212977 212843 -134
- Misses 65531 65659 +128
- Partials 17435 17477 +42 ☔ View full report in Codecov by Sentry. |
Signed-off-by: Sarat Vemulapalli <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
d5a6c0b
into
opensearch-project:main
The backport to
To backport manually, run these commands in your terminal: # Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.x
# Create a new branch
git switch --create backport/backport-15325-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 d5a6c0bffe405e97021bf7405a92d2c9b050245c
# Push it to GitHub
git push --set-upstream origin backport/backport-15325-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.x Then, create a pull request where the |
…sors (opensearch-project#15325) * Adding allowlist setting for user-agent, geo-ip and updated tests for ingest-common. Signed-off-by: Sarat Vemulapalli <[email protected]> * Remove duplicate test in ingest-common Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> --------- Signed-off-by: Sarat Vemulapalli <[email protected]>
…st-geoip processors (#15374) * Adding allowlist setting for ingest-useragent and ingest-geoip processors (#15325) * Adding allowlist setting for user-agent, geo-ip and updated tests for ingest-common. Signed-off-by: Sarat Vemulapalli <[email protected]> * Remove duplicate test in ingest-common Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> --------- Signed-off-by: Sarat Vemulapalli <[email protected]> * Fixing spotlesscheck Signed-off-by: Sarat Vemulapalli <[email protected]> * Fixing merge conflicts Signed-off-by: Sarat Vemulapalli <[email protected]> * Spotless Again Signed-off-by: Sarat Vemulapalli <[email protected]> --------- Signed-off-by: Sarat Vemulapalli <[email protected]>
* Optimize global ordinal includes/excludes for prefix matching (opensearch-project#14371) * Optimize global ordinal includes/excludes for prefix matching If an aggregration specifies includes or excludes based on a regular expression, and the regular expression has a finite expansion followed by .*, then we can optimize the global ordinal filter. Specifically, in this case, we can expand the matching prefixes, then include/exclude the range of global ordinals that start with each prefix. Signed-off-by: Michael Froh <[email protected]> * Add unit test Signed-off-by: Michael Froh <[email protected]> * Add changelog entry Signed-off-by: Michael Froh <[email protected]> * Improve test coverage Updated the unit test to be functionally equivalent, but it covers more of the regex logic. Signed-off-by: Michael Froh <[email protected]> * Improve test coverage Signed-off-by: Michael Froh <[email protected]> * Fix bug in exclude-only case with no doc values in segment Signed-off-by: Michael Froh <[email protected]> * Address comments from @mch2 Signed-off-by: Michael Froh <[email protected]> --------- Signed-off-by: Michael Froh <[email protected]> * Adding access to noSubMatches and noOverlappingMatches in Hyphenation… (opensearch-project#13895) * Adding access to noSubMatches and noOverlappingMatches in HyphenationCompoundWordTokenFilter Signed-off-by: Evan Kielley <[email protected]> * Add Changelog Entry Signed-off-by: Mohammad Hasnain Mohsin Rajan <[email protected]> * test: add hyphenation decompounder tests Signed-off-by: Mohammad Hasnain <[email protected]> * test: refactor tests Signed-off-by: Mohammad Hasnain <[email protected]> * test: reformat test files Signed-off-by: Mohammad Hasnain <[email protected]> * chore: add changelog entry for 2.X Signed-off-by: Mohammad Hasnain <[email protected]> * chore: remove 3.x changelog Signed-off-by: Mohammad Hasnain <[email protected]> * chore: commonify settingsarr Signed-off-by: Mohammad Hasnain <[email protected]> * chore: commonify settingsarr Signed-off-by: Mohammad Hasnain <[email protected]> * chore: linting Signed-off-by: Mohammad Hasnain <[email protected]> --------- Signed-off-by: Evan Kielley <[email protected]> Signed-off-by: Mohammad Hasnain Mohsin Rajan <[email protected]> Signed-off-by: Mohammad Hasnain <[email protected]> Co-authored-by: Evan Kielley <[email protected]> * Add Settings related to Workload Management feature (opensearch-project#15028) * add QeryGroup Service tests Signed-off-by: Ruirui Zhang <[email protected]> * add PR to changelog Signed-off-by: Ruirui Zhang <[email protected]> * change the test directory Signed-off-by: Ruirui Zhang <[email protected]> * modify comments to be more specific Signed-off-by: Ruirui Zhang <[email protected]> * add test coverage Signed-off-by: Ruirui Zhang <[email protected]> * remove QUERY_GROUP_RUN_INTERVAL_SETTING as we'll define it in QueryGroupService Signed-off-by: Ruirui Zhang <[email protected]> * address comments Signed-off-by: Ruirui Zhang <[email protected]> * Update affiliation for @nknize. (opensearch-project#15322) Signed-off-by: dblock <[email protected]> * Add log when download completes with file size (opensearch-project#15224) Signed-off-by: Gaurav Bafna <[email protected]> * Support Filtering on Large List encoded by Bitmap (version update) (opensearch-project#15352) Signed-off-by: Andriy Redko <[email protected]> * Add support for index level slice count setting (opensearch-project#15336) Signed-off-by: Ganesh Ramadurai <[email protected]> * Adding allowlist setting for ingest-useragent and ingest-geoip processors (opensearch-project#15325) * Adding allowlist setting for user-agent, geo-ip and updated tests for ingest-common. Signed-off-by: Sarat Vemulapalli <[email protected]> * Remove duplicate test in ingest-common Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> --------- Signed-off-by: Sarat Vemulapalli <[email protected]> * Add Delete QueryGroup API Logic (opensearch-project#14735) * Add Delete QueryGroup API Logic Signed-off-by: Ruirui Zhang <[email protected]> * modify changelog Signed-off-by: Ruirui Zhang <[email protected]> * include comments from create pr Signed-off-by: Ruirui Zhang <[email protected]> * remove delete all Signed-off-by: Ruirui Zhang <[email protected]> * rebase and address comments Signed-off-by: Ruirui Zhang <[email protected]> * rebase Signed-off-by: Ruirui Zhang <[email protected]> * address comments Signed-off-by: Ruirui Zhang <[email protected]> * address comments Signed-off-by: Ruirui Zhang <[email protected]> * address comments Signed-off-by: Ruirui Zhang <[email protected]> * add UT coverage Signed-off-by: Ruirui Zhang <[email protected]> * [Star Tree] Lucene Abstractions for Star Tree File Formats (opensearch-project#15278) --------- Signed-off-by: Sarthak Aggarwal <[email protected]> * [Star tree] Changes to handle derived metrics such as avg as part of star tree mapping (opensearch-project#15152) --------- Signed-off-by: Bharathwaj G <[email protected]> * relaxing the join validation for nodes which have only store disabled but only publication enabled * relaxing the join validation for nodes which have only store disabled but only publication enabled Signed-off-by: Rajiv Kumar Vaidyanathan <[email protected]> --------- Signed-off-by: Michael Froh <[email protected]> Signed-off-by: Evan Kielley <[email protected]> Signed-off-by: Mohammad Hasnain Mohsin Rajan <[email protected]> Signed-off-by: Mohammad Hasnain <[email protected]> Signed-off-by: dblock <[email protected]> Signed-off-by: Gaurav Bafna <[email protected]> Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Ganesh Ramadurai <[email protected]> Signed-off-by: Sarat Vemulapalli <[email protected]> Signed-off-by: Rajiv Kumar Vaidyanathan <[email protected]> Co-authored-by: Michael Froh <[email protected]> Co-authored-by: Mohammad Hasnain Mohsin Rajan <[email protected]> Co-authored-by: Evan Kielley <[email protected]> Co-authored-by: Ruirui Zhang <[email protected]> Co-authored-by: Daniel (dB.) Doubrovkine <[email protected]> Co-authored-by: Gaurav Bafna <[email protected]> Co-authored-by: Andriy Redko <[email protected]> Co-authored-by: Ganesh Krishna Ramadurai <[email protected]> Co-authored-by: Sarat Vemulapalli <[email protected]> Co-authored-by: Sarthak Aggarwal <[email protected]> Co-authored-by: Bharathwaj G <[email protected]> Co-authored-by: Rajiv Kumar Vaidyanathan <[email protected]>
…sors (opensearch-project#15325) * Adding allowlist setting for user-agent, geo-ip and updated tests for ingest-common. Signed-off-by: Sarat Vemulapalli <[email protected]> * Remove duplicate test in ingest-common Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> --------- Signed-off-by: Sarat Vemulapalli <[email protected]>
…sors (opensearch-project#15325) * Adding allowlist setting for user-agent, geo-ip and updated tests for ingest-common. Signed-off-by: Sarat Vemulapalli <[email protected]> * Remove duplicate test in ingest-common Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> --------- Signed-off-by: Sarat Vemulapalli <[email protected]>
…sors (opensearch-project#15325) * Adding allowlist setting for user-agent, geo-ip and updated tests for ingest-common. Signed-off-by: Sarat Vemulapalli <[email protected]> * Remove duplicate test in ingest-common Signed-off-by: Sarat Vemulapalli <[email protected]> * Adding changelog Signed-off-by: Sarat Vemulapalli <[email protected]> --------- Signed-off-by: Sarat Vemulapalli <[email protected]>
Description
Added allow list setting to enable/disable processors for ingest-useragent and ingest-geoip.
Also, added another test to verify invalid processors cannot be specified for ingest-common.
Related Issues
This is a follow up of : #14479
The original PR only covered ingest-common, while this PR completes all ingest modules included from OpenSearch.
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.