updated the cross site health check script to suit the new deployments

[kami619]

Fixes #980

The updated script works against the A/A deployments.

> ./cross-site-health-checks.sh -d ae3df5446dfc4f4e2.dualstack.awsglobalaccelerator.com \
-u developer -p $ISPN_PWD \
-s gh-keycloak-a.lsja.p3.openshiftapps.com \
-c 3 -n runner-keycloak
Verify the Keycloak Load Balancer health check
Checking health for: https://ae3df5446dfc4f4e2.dualstack.awsglobalaccelerator.com/lb-check
"HEALTHY"

Verify the Load Balancer health check on the Site
Checking health for: https://aaee0689882944bb780191449cbfd791-78a0ef5dd6a41017.elb.eu-west-1.amazonaws.com/lb-check
"HEALTHY"

Verify the default cache manager health in external ISPN
Checking health for: https://infinispan-external-runner-keycloak.apps.rosa.gh-keycloak-a.lsja.p3.openshiftapps.com/rest/v2/cache-managers/default/health/status
"HEALTHY"

Verify individual cache health
"HEALTHY"

ISPN Cluster Distribution
"HEALTHY"

ISPN Overall Status
"HEALTHY"

Verify for Keycloak condition in ROSA cluster
keycloak.k8s.keycloak.org/keycloak condition met
keycloak.k8s.keycloak.org/keycloak condition met

[ryanemerson]

@kami619 Do we need to document somewhere that this now needs to be executed against each site in order to verify both /lb-check?

[kami619]

@ryanemerson that's a good point, earlier we were getting the urls constructed from domain. But now its not the case. Should we just have the url as an input and give pointers on how to get it ? And the fact we have to run it twice on both clusters would be an implementation detail at that point. We could simply fetch this value as a precursor in our CI job and other SRE's could do the same. WDYT?

[ryanemerson]

I'm happy to have the URLs as an input. The method to retrieve the URL will differ for users based upon the namespace they use for their deployment, so extracting the retrieval from the script will make it more versatile.

[kami619]

thanks, @ryanemerson, I will work on the change.

[kami619]

@ryanemerson Please take another look at it, it should align with what we discussed earlier and the docs are updated for the same, I have added a note to indicate that this should be run twice or more times for all the clusters in the multi-site setup.

[ryanemerson]

Thanks @kami619