[Backport 2.x] SearchBackPressure Policy/Decider Generic Framework Added #497
Commits on Oct 5, 2023
-
SearchBackPressure Policy/Decider Generic Framework Added (#461)
* Remove log files and add DCO (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Remove extra files (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Remove styling difference (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Remove unnecessary file changes (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add RCA_Decider (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Extract Heap Usage from SQlitedb (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Extract required searchbp metrics for deciders (signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add SearchBackPressureRCA Metric (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Use SearchBackPressureRCAMetrics to aggregate metrics (signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add the conf file extracted part for SearchBackPressureRcaConfig.java (signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add MinMaxSlidingWindow in OldGenRca (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Rename SearchBackPressureClusterRCA and add it to AnalysisGraph (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add basic UTs for SearchBackPressureRCA cluster/node level (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add unhealthy/healthy stats UTs for SearchBackPressureRCA cluster/node level (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add healthy resource unit UT (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add UT s both shard/task level (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add a new SearchBp Resource Unit (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add UTs to test shard/task level resource include-ness (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Remove styling changes for Version.java (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add metadata to resourceSummary (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Update to more general framework (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Refactor the MinMaxSlidingWindow and bug fix (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Refactor Heap Stats Metrics Getter(Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Refactor HeapUsed and HeapMax Getters (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Refactor operate() (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Refactor operate() and remove dead comments (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add new ActionPojo for Searchbp (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add new ActionPojo for Searchbp#2 (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Update SearchBackPressureAction (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add Searchbp Decider (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add Searchbp Policy and Config (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add dummy Searchbp Policy and Config in OpenSearchAnalysis Graph (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Update SearchBackpressure Policy (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Update SearchBackpressure Policy (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Merged Main (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add new AlarmMonitor (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Workable decider (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Workable decider (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Workable pipeline (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Workable pipeline (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Update ActionPojo (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Framework can read from config (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add Policy Increase/Decrease Alarm (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Generic Framework can generate shard/task and increase/decrease actions (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Generic Framework can generate specific actions and read from config (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * removed dead comment for SearchBpActionConfig.java (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * removed dead comment for action/polict (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * removed dead comment for action/policy (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add increase/decrease direction for ActionPojo (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * remove trailing (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Restore to workable solution Signed-off-by: CoderJeffrey <[email protected]> * Restore to workable solution (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * remove hourly window size and bucket size (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add hourMonitorConfig to set up alarm monitor (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Remove unused counter (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * change field description (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * refactor (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * enum added for direction and shard/task dimension (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Use enum for Dimension/Direction for Searchbp Action (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Add lambda function for next() in OldGenRCA (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Merged main with build.gradle (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * resolve nit#1 (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * resolve nit#2 (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * refactor OpenSearchAnalysisGraph (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * stream() refactor (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * null check refactor (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Change LOG to debug level for unnecessary info (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * refactor shar/task issue (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * nit fix (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Added JavaDoc for Searchbp Action (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * SearchBackPressureIssue Interface created for refactor (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> * Remove dead comment (Signed-off-by: Jeffrey Liu [email protected]) Signed-off-by: CoderJeffrey <[email protected]> --------- Signed-off-by: CoderJeffrey <[email protected]> (cherry picked from commit 749da27)
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.