Register agent API support for MLClient #1647

arjunkumargiri · 2023-11-16T01:16:40Z

Description

Register agent API support for MLClient. This will allow flow framework to integrate with register agent API.

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

opensearch-project#581) * return circuit breaker name in error messages Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Xun Zhang <[email protected]>

opensearch-project#583) Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

…ensearch-project#584) * chunk upload api Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Bhavana Ramaram <[email protected]>

…ct#510) (opensearch-project#586) Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Xun Zhang <[email protected]>

…) (opensearch-project#593)

…h-project#598) * use small model to run integ test Signed-off-by: Yaliang Wu <[email protected]> * tune custom model integ test Signed-off-by: Yaliang Wu <[email protected]> * close file stream Signed-off-by: Yaliang Wu <[email protected]> * decrease disk circuit breaker limit Signed-off-by: Yaliang Wu <[email protected]> * add more test cases for model manager Signed-off-by: Yaliang Wu <[email protected]> * tune model profile verification Signed-off-by: Yaliang Wu <[email protected]> * tune ML engine and integ test Signed-off-by: Yaliang Wu <[email protected]> * add integ test on multi data node cluster Signed-off-by: Yaliang Wu <[email protected]> * add unit test for forward action Signed-off-by: Yaliang Wu <[email protected]> * add predict request duration in profile response Signed-off-by: Yaliang Wu <[email protected]> * ignore custom integ test Signed-off-by: Yaliang Wu <[email protected]> * fix ml engine after merging 2.x Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

…ct#515) (opensearch-project#600)

…mmons pakage (opensearch-project#517) (opensearch-project#601) Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Xun Zhang <[email protected]>

…rch-project#604)

…-project#516) (opensearch-project#605)

…when upload model; tune log level) (opensearch-project#606)

…ject#530) (opensearch-project#607) * change the max_ml_task_per_node into dynamic settings Signed-off-by: Sicheng Song <[email protected]> * Trying to bypassing the flaky test Signed-off-by: Sicheng Song <[email protected] Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Sicheng Song <[email protected] Signed-off-by: Sicheng Song <[email protected]>

opensearch-project#608) Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Xun Zhang <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Xun Zhang <[email protected]>

…search-project#536) (opensearch-project#609)

…arch-project#610)

…oject#540) (opensearch-project#612) Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

… (opensearch-project#613) * fix running tasks when circuit breaker is open Signed-off-by: Yaliang Wu <[email protected]> * fix log error message Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

…) (opensearch-project#614) Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

…arch-project#615) Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Bhavana Ramaram <[email protected]>

…t#551) (opensearch-project#616)

…nsearch-project#561) (opensearch-project#617) Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

…roject#573) (opensearch-project#618) Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Bhavana Ramaram <[email protected]>

…oject#580) (opensearch-project#619) Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

…earch-project#620) Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

…pensearch-project#621) * unit tests coverage for load/unload/syncup Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Bhavana Ramaram <[email protected]>

Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Yaliang Wu <[email protected]>

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>

opensearch-project#659) Signed-off-by: dblock <[email protected]>

owaiskazi19

LGTM

ylwu-amzn

LGTM. Thanks a lot for the change

ylwu-amzn · 2023-11-16T23:24:44Z

BTW, the Github CI failed, maybe need to rebase ?

codecov · 2023-11-17T00:48:23Z

Codecov Report

Attention: 32 lines in your changes are missing coverage. Please review.

Comparison is base (1b76eb1) 71.53% compared to head (594c73c) 67.25%.
Report is 324 commits behind head on feature/agent_framework_dev.

❗ Current head 594c73c differs from pull request most recent head 5759bf2. Consider uploading reports for the commit 5759bf2 to get more accurate results

Files	Patch %	Lines
...pensearch/ml/client/MachineLearningNodeClient.java	77.96%	24 Missing and 2 partials ⚠️
...rg/opensearch/ml/client/MachineLearningClient.java	75.00%	6 Missing ⚠️

Additional details and impacted files

@@                        Coverage Diff                        @@
##             feature/agent_framework_dev    #1647      +/-   ##
=================================================================
- Coverage                          71.53%   67.25%   -4.29%     
- Complexity                           802     2504    +1702     
=================================================================
  Files                                 86      227     +141     
  Lines                               3267    12239    +8972     
  Branches                             309     1263     +954     
=================================================================
+ Hits                                2337     8231    +5894     
- Misses                               813     3455    +2642     
- Partials                             117      553     +436

Flag	Coverage Δ
ml-commons	`67.25% <77.46%> (-4.29%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

b4sjoo and others added 30 commits November 29, 2022 12:49

changes to custom model chunk upload api (opensearch-project#495) (op…

0689ee8

…ensearch-project#584) * chunk upload api Signed-off-by: Bhavana Ramaram <[email protected]> Signed-off-by: Sicheng Song <[email protected]> Co-authored-by: Bhavana Ramaram <[email protected]>

add test coverage to transportUploadModelAction (opensearch-project#511…

99306dc

…) (opensearch-project#593)

support multi-gpu; fix inference duration queue bug (opensearch-proje…

d52e80e

…ct#515) (opensearch-project#600)

add trusted URL regex (opensearch-project#518) (opensearch-project#602)

77b1459

Add test coverage to common package (opensearch-project#514) (opensea…

b3f390c

…rch-project#604)

add unit tests to improve test coverage in plugin package (opensearch…

ac57983

…-project#516) (opensearch-project#605)

Forwarding port changes in 2.4 to main branch (add description field …

63e02af

…when upload model; tune log level) (opensearch-project#606)

test coverage ratio changes for build.gradlew in plugin package (open…

75522ab

…search-project#536) (opensearch-project#609)

add ml-commons release notes for 2.4 (opensearch-project#538) (opense…

534b9e2

…arch-project#610)

wait for upload task to complete in security tests (opensearch-projec…

7b2f38f

…t#551) (opensearch-project#616)

Add .whitesource configuration file (opensearch-project#611)

339176b

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>

Update dependency junit:junit to v4.13.1 (opensearch-project#625)

606226f

Co-authored-by: mend-for-github-com[bot] <50673670+mend-for-github-com[bot]@users.noreply.github.com>

Updated MAINTAINERS.md to match recommended opensearch-project format. (

098de57

opensearch-project#659) Signed-off-by: dblock <[email protected]>

arjunkumargiri had a problem deploying to ml-commons-cicd-env November 16, 2023 22:41 — with GitHub Actions Failure

arjunkumargiri requested review from ylwu-amzn and owaiskazi19 November 16, 2023 22:54

owaiskazi19 approved these changes Nov 16, 2023

View reviewed changes

ylwu-amzn approved these changes Nov 16, 2023

View reviewed changes

arjunkumargiri force-pushed the client_local branch from a845401 to 594c73c Compare November 17, 2023 00:30

arjunkumargiri temporarily deployed to ml-commons-cicd-env November 17, 2023 00:30 — with GitHub Actions Inactive

arjunkumargiri had a problem deploying to ml-commons-cicd-env November 17, 2023 00:31 — with GitHub Actions Failure

arjunkumargiri had a problem deploying to ml-commons-cicd-env November 17, 2023 00:31 — with GitHub Actions Error

arjunkumargiri temporarily deployed to ml-commons-cicd-env November 17, 2023 00:31 — with GitHub Actions Inactive

arjunkumargiri force-pushed the client_local branch from 594c73c to 26a6914 Compare November 17, 2023 00:31

arjunkumargiri had a problem deploying to ml-commons-cicd-env November 17, 2023 00:31 — with GitHub Actions Error

arjunkumargiri had a problem deploying to ml-commons-cicd-env November 17, 2023 00:31 — with GitHub Actions Failure

arjunkumargiri force-pushed the client_local branch from 26a6914 to 5759bf2 Compare November 17, 2023 00:49

arjunkumargiri temporarily deployed to ml-commons-cicd-env November 17, 2023 00:49 — with GitHub Actions Inactive

arjunkumargiri had a problem deploying to ml-commons-cicd-env November 17, 2023 00:49 — with GitHub Actions Failure

arjunkumargiri temporarily deployed to ml-commons-cicd-env November 17, 2023 00:50 — with GitHub Actions Inactive

arjunkumargiri had a problem deploying to ml-commons-cicd-env November 17, 2023 00:50 — with GitHub Actions Error

arjunkumargiri closed this Nov 17, 2023

arjunkumargiri deleted the client_local branch November 17, 2023 00:57

peterzhuamazon had a problem deploying to ml-commons-cicd-env November 20, 2023 19:48 — with GitHub Actions Error

peterzhuamazon had a problem deploying to ml-commons-cicd-env November 20, 2023 19:48 — with GitHub Actions Failure

peterzhuamazon had a problem deploying to ml-commons-cicd-env November 20, 2023 19:48 — with GitHub Actions Error

peterzhuamazon had a problem deploying to ml-commons-cicd-env November 20, 2023 19:49 — with GitHub Actions Error

peterzhuamazon had a problem deploying to ml-commons-cicd-env November 20, 2023 19:49 — with GitHub Actions Failure

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Register agent API support for MLClient #1647

Register agent API support for MLClient #1647

arjunkumargiri commented Nov 16, 2023 •

edited

Loading

owaiskazi19 left a comment

ylwu-amzn left a comment

ylwu-amzn commented Nov 16, 2023

codecov bot commented Nov 17, 2023 •

edited

Loading

Register agent API support for MLClient #1647

Register agent API support for MLClient #1647

Conversation

arjunkumargiri commented Nov 16, 2023 • edited Loading

Description

Check List

owaiskazi19 left a comment

Choose a reason for hiding this comment

ylwu-amzn left a comment

Choose a reason for hiding this comment

ylwu-amzn commented Nov 16, 2023

codecov bot commented Nov 17, 2023 • edited Loading

Codecov Report

arjunkumargiri commented Nov 16, 2023 •

edited

Loading

codecov bot commented Nov 17, 2023 •

edited

Loading