Adding security enabled integration tests

[joshpalis]

Description

This PR aims to add security enabled integration tests to the CI in order to test various cluster permissions based on the opendistro security roles added by opensearch-project/security#3851. The tests themselves are relatively simple, we create two users, one with a flow_framework_full_access role, and one with a flow_framework_read_access role to determine if these users can hit our APIs.

The build.gradle has been modified to automatically configure an integration test cluster with or without the security plugin installed depending on the provided system properties. The following details the commands to run an integration test cluster with or without security :

To run a single-node/multi-node cluster locally with Flow Framework and ML Commons :  
./gradlew run
./gradlew run -DnumNodes=<number of nodes>

To run a single-node/multi-node cluster locally with Flow Framework, ML Commons and Security : 
./gradlew run -Dsecurity.enabled=true
./gradlew run -Dsecurity.enabled=true -DnumNodes=<number of nodes>

To run single-node non-security enabled integration tests :
./gradlew integTest

To run single-node security enabled integration tests : 
./gradlew integTest -Dsecurity.enabled=true

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

Attention: 16 lines in your changes are missing coverage. Please review.

Comparison is base (f8e822f) 72.14% compared to head (9402f58) 71.95%.

Files	Patch %	Lines
...ework/transport/CreateWorkflowTransportAction.java	36.36%	6 Missing and 1 partial ⚠️
...h/flowframework/rest/RestCreateWorkflowAction.java	0.00%	5 Missing ⚠️
...lowframework/rest/RestProvisionWorkflowAction.java	0.00%	4 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #400      +/-   ##
============================================
- Coverage     72.14%   71.95%   -0.20%     
  Complexity      613      613              
============================================
  Files            79       79              
  Lines          3070     3081      +11     
  Branches        238      238              
============================================
+ Hits           2215     2217       +2     
- Misses          751      760       +9     
  Partials        104      104              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[joshpalis]

Seems the client used to determine the security plugin is present in the cluster is refusing to connect, which is preventing our security integration tests to run :

Run security=`curl -XGET https://localhost:9[20](https://github.com/opensearch-project/flow-framework/actions/runs/7494086694/job/20401224256?pr=400#step:7:21)0/_cat/plugins?v -u admin:admin --insecure |grep opensearch-security|wc -l`
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (7) Failed to connect to localhost port 9200 after 0 ms: Connection refused
Security plugin is NOT available, skipping integration tests

[owaiskazi19]

Looks good overall with few comments

[owaiskazi19]

Looks good to me unless you are planning to add more tests in the same PR

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/flow-framework/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/flow-framework/backport-2.x
# Create a new branch
git switch --create backport/backport-400-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 74f42ba712bfc008ddc21ca18402f4dde541fd1a
# Push it to GitHub
git push --set-upstream origin backport/backport-400-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/flow-framework/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-400-to-2.x.