[Int tests] remove terraform , cloud engine and update README.md

[dkijania]

Removing cloud tests from integration tests with terraform files. Terraform files and cloud mode of integration tests were tightly coupled and used only for this purpose. Currently preferred way to run integration tests are docker.

Since not everyone has an access to our slack. I'm pasting chunks of discussion why removing mentioned files is a good idea:

Hey Hey Luis!
I am looking for your guidance regarding moving integration tests from cloud to docker. We still have terraform files in our repo and failing terrraform test on top of that (which is creating main.tf like TE and try to run terraform check and some linters). If we move all tests to docker , do we still need terraform? should we keep it just for keeping it up to date if we encoutner any issue on docker side?

ANSWER: We do not use Terraform for long-standing deployments and the way it looks now, we won't be needing them either for private cluster testing.
At the moment the only workflow that uses the Terraform workflow are the integration tests

and

Moving Intg tests from cloud to docker completely
I think there is only a little difference between running intg tests in a single cluster on cloud and docker. There were two (that i know) main reasons to keep cloud tests:
they are using terraform files which are used in production
they have higher latency than docker compose env.
I think first argument is obsolete since we are not using terraform when deploying testnets but helm files (+ argo cd). For the latency simulation we can use:
docker exec {node} tc qdisc add dev eth0 root netem delay 100ms
command which will delay network in a controlled way.
What you think ?
Edit: currently payment integration test is running on docker compose, others on cluster (edited)
5 replies

Luis:
Just to add on top of this, the current deployment approach opens the door for multi-cloud deployments and tests.
I think special tests under this realistic setup might be of interest.
my 2 cents

Alan:
I am all for trying this, it could save a lot in terms of execution as well as base infra cost

Matthew:
I don't think we're effectively leveraging the latency in the tests anyway. Good from my side to move any of them to docker

George:
Yes, please! Let's do it, I'm waiting for this to happen for a long time 😄
Latency isn't a viable argument, putting a few nodes in a single cluster is almost the same as if they were on the same hardware. If we want some latency later, we should simulate it programmatically just as was suggested in the message above.

[dkijania]

!ci-build-me

[dkijania]

!ci-build-me

[dkijania]

!ci-build-me

[georgeee]

What's purpose of the script?

[dkijania]

Good catch! Reverting file. This change is not related to the PR. The script is printing out dependencies tree for each package which then can be used to clean not used deps

[dkijania]

!ci-build-me

[dkijania]

!ci-nightly-me