Implemented throttling for max workflows to be created

[owaiskazi19]

Description

Implemented throttling for max workflows to be created.

For testing purpose. Set the max limit to 70.

curl -XPOST -u 'full_access:ollyframework@123' --insecure -H "Content-Type: application/json" 'https://localhost:9200/_plugins/_flow_framework/workflow/_search?filter_path=hits.total' -d '{"query":{"match_all":{}}}'
{"hits":{"total":{"value":70,"relation":"eq"}}}ubuntu@ip-172-31-56-214:~/OpenSearch$ curl -XPOST -u 'full_access:ollyframework@123' --insecure -H "Content-Type: application/json" 'https://l' --data '{"name":"create-connector-register-deploy-model","description":"test case","use_case":"TEST_CASE","version":{"template":"1.0.0","compatibility":["2.12.0","3.0.0"]},"workflows":{"provision":{"nodes":[{"id":"workflow_step_1","type":"create_connector","user_inputs":{"name":"OpenAI Chat Connector","description":"The connector to public OpenAI model service for GPT 3.5","version":"1","protocol":"http","parameters":{"endpoint":"api.openai.com","model":"gpt-3.5-turbo"},"credential":{"openAI_key":"..."},"actions":[{"action_type":"predict","method":"POST","url":"https://${parameters.endpoint}/v1/chat/completions"}]}},{"id":"workflow_step_2","type":"register_model","previous_node_inputs":{"workflow_step_1":"connector_id"},"user_inputs":{"name":"openAI-gpt-3.5-turbo","function_name":"remote","description":"test model"}},{"id":"workflow_step_3","type":"deploy_model","previous_node_inputs":{"workflow_step_2":"model_id"}}],"edges":[{"source":"workflow_step_1","dest":"workflow_step_2"},{"source":"workflow_step_2","dest":"workflow_step_3"}]}}}'


{"error":"Maximum workflows limit reached 70"}

Tried to create workflow again

curl -XPOST -u 'full_access:ollyframework@123' --insecure -H "Content-Type: application/json" 'https://localhost:9200/_plugins/_flow_framework/workflow' --data '{"name":"create-connector-register-deploy-model","description":"test case","use_case":"TEST_CASE","version":{"template":"1.0.0","compatibility":["2.12.0","3.0.0"]},"workflows":{"provision":{"nodes":[{"id":"workflow_step_1","type":"create_connector","user_inputs":{"name":"OpenAI Chat Connector","description":"The connector to public OpenAI model service for GPT 3.5","version":"1","protocol":"http","parameters":{"endpoint":"api.openai.com","model":"gpt-3.5-turbo"},"credential":{"openAI_key":"..."},"actions":[{"action_type":"predict","method":"POST","url":"https://${parameters.endpoint}/v1/chat/completions"}]}},{"id":"workflow_step_2","type":"register_model","previous_node_inputs":{"workflow_step_1":"connector_id"},"user_inputs":{"name":"openAI-gpt-3.5-turbo","function_name":"remote","description":"test model"}},{"id":"workflow_step_3","type":"deploy_model","previous_node_inputs":{"workflow_step_2":"model_id"}}],"edges":[{"source":"workflow_step_1","dest":"workflow_step_2"},{"source":"workflow_step_2","dest":"workflow_step_3"}]}}}'

{"error":"Maximum workflows limit reached 70"}

Verified that no new workflow has been created as the count remains 70

curl -XPOST -u 'full_access:ollyframework@123' --insecure -H "Content-Type: application/json" 'https://localhost:9200/_plugins/_flow_framework/workflow/_search?filter_path=hits.total' -d '{"query":{"match_all":{}}}'
{"hits":{"total":{"value":70,"relation":"eq"}}}

Issues Resolved

Closes #150

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[joshpalis]

Initial pass

[codecov]

Codecov Report

Merging #151 (2eb03a8) into main (1de0ca5) will increase coverage by 0.68%.
The diff coverage is 61.42%.

@@             Coverage Diff              @@
##               main     #151      +/-   ##
============================================
+ Coverage     67.70%   68.39%   +0.68%     
- Complexity      330      339       +9     
============================================
  Files            43       45       +2     
  Lines          1539     1585      +46     
  Branches        142      143       +1     
============================================
+ Hits           1042     1084      +42     
- Misses          448      452       +4     
  Partials         49       49              

Files	Coverage Δ
.../opensearch/flowframework/FlowFrameworkPlugin.java	100.00% <100.00%> (ø)
...ork/common/FlowFrameworkFeatureEnabledSetting.java	100.00% <ø> (ø)
...ch/flowframework/common/FlowFrameworkSettings.java	100.00% <100.00%> (ø)
...owframework/rest/AbstractSearchWorkflowAction.java	67.85% <ø> (ø)
...rch/flowframework/rest/AbstractWorkflowAction.java	100.00% <100.00%> (ø)
...lowframework/rest/RestProvisionWorkflowAction.java	56.66% <ø> (ø)
...rk/transport/ProvisionWorkflowTransportAction.java	49.27% <ø> (ø)
...pensearch/flowframework/util/RestHandlerUtils.java	0.00% <ø> (ø)
...h/flowframework/rest/RestCreateWorkflowAction.java	56.25% <50.00%> (ø)
...earch/flowframework/transport/WorkflowRequest.java	93.75% <91.66%> (-6.25%)	⬇️
... and 2 more

[joshpalis]

LGTM thanks for the changes

[dbwiddis]

LGTM!

I might quibble on the default max being so high, but OK for now.