Chore: Load Tests run in CI #324

wilwade · 2024-08-01T20:52:42Z

Problem

We want to run the k6 tests in CI so that they also function as semi-integration test system.

Closes: #317

Solution

Added a workflow for the k6 tests
Disabled one test I didn't want to get working without additional work. So added an issue for that: [Account Service] [k6] Enable account-service-load.k6.js #344

Run the tests Everything has tests now WIP WIP 2 Trying a different way to run the k6 tests

Give a bit more time before starting Fix health check urls Fix issue with GHA script Move just to node More documentation on the wait for ready check Javascript :( Change wait method Fix endpoint

Add the rust webhook server to the docker compose for k6 account service

wilwade · 2024-08-06T17:32:14Z

.github/workflows/load-tests.yml

This follows the pattern from the Unit test workflow

wilwade · 2024-08-06T17:33:31Z

.github/workflows/load-tests.yml

+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Install Node.js


Why node?

Because we need to run the setup script npm run main in tools/ci-k6

wilwade · 2024-08-06T17:35:57Z

docker-compose-k6.account.yaml

Each service has a docker-compose-k6.{service}.yaml. This overrides the base docker-compose.

Switches to use the production docker

configures other services and conditions as needed.

wilwade · 2024-08-06T17:36:12Z

docker-compose-k6.account.yaml

+      PROVIDER_BASE_URL: 'http://account-service-webhook:3001/webhooks/account-service'
+      WEBHOOK_BASE_URL: 'http://account-service-webhook:3001/webhooks'
+
+  account-service-webhook:


Great example of an additional service needed.

wilwade · 2024-08-06T17:36:42Z

docker-compose-k6.account.yaml

+    profiles:
+      - skip


This is so these services don't start by default. Easiest way to do this I found

wilwade · 2024-08-06T17:37:06Z

docker-compose.yaml

@@ -55,7 +56,7 @@ services:
      - redis_data:/data/redis

  frequency:
-    image: dsnp/instant-seal-node-with-deployed-schemas:latest
+    image: frequencychain/standalone-node:v1.13.0-rc3


Going ahead and switching to the new base image

wilwade · 2024-08-06T17:37:28Z

docker-compose.yaml

@@ -159,7 +160,6 @@ services:
      - graph_node_cache:/app/services/graph/node_modules
    depends_on:
      - redis
-      - ipfs


graph service doesn't need ipfs

wilwade · 2024-08-06T17:37:53Z

services/account/docs/account_service_arch.drawio

@@ -10,7 +10,7 @@
        <mxCell id="34" value="" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1">
          <mxGeometry x="280" y="330" width="390" height="330" as="geometry" />
        </mxCell>
-        <mxCell id="35" value="GET/api/health" style="endArrow=classic;html=1;rounded=0;fillColor=#d5e8d4;strokeColor=#82b366;strokeWidth=3;exitX=0.996;exitY=0.17;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.172;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="45" target="41" edge="1">
+        <mxCell id="35" value="GET/healthz" style="endArrow=classic;html=1;rounded=0;fillColor=#d5e8d4;strokeColor=#82b366;strokeWidth=3;exitX=0.996;exitY=0.17;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.172;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="45" target="41" edge="1">


I think this is the correct one, but I didn't generate the pngs for it. Let me know if it is correct.

wilwade · 2024-08-06T17:39:07Z

services/account/k6-test/README.md

@@ -1,7 +1,7 @@
 # Generated k6 script

-The `account-service-load.js` file contains most of the Swagger/OpenAPI specification and you can customize it to your needs.
-The `health-check.js` file contains a simple health check script that can be used to check the health of the service.
+The `account-service-load.k6.js` file contains most of the Swagger/OpenAPI specification and you can customize it to your needs.


First place this shows up...

So why?

I needed to be able to know when something was specifically for k6

This felt easier than trying to put all non-k6 test files into a subdirectory

wilwade · 2024-08-06T17:40:46Z

services/account/k6-test/README.md

 ```
+
+## Generating Test Data


Added a script to generate the information needed to run this test. I went for committing the generated code, as it works fine for running in CI and most runs. I'd be open to generating this as needed as well.

wilwade · 2024-08-06T17:41:39Z

services/account/k6-test/account-service-load.k6.skip.js

This file I needed to move to skip as it requires more setup than we currently have in place. Various options to do this setup (including generating data like we do for signup) to using the same setup script that creates the provider.

wilwade · 2024-08-06T17:41:56Z

services/account/k6-test/account-service-load.k6.skip.js

@@ -36,63 +36,7 @@ const SLEEP_DURATION = 0.1;
 // Global variables should be initialized.

 export default function () {
-  group('health', () => {


Health and siwf signup are in other test files now.

wilwade · 2024-08-06T17:42:53Z

services/account/k6-test/account-service-load.k6.skip.js

-    }
-  });
-
+  // Needs 255 generated accounts


Annotated these with what they need to get it unskipped.

wilwade · 2024-08-06T17:43:14Z

services/account/k6-test/generate/signups.mjs

Generator script

wilwade · 2024-08-06T17:43:43Z

services/account/k6-test/new-sign-up.k6.js

@@ -43,38 +46,39 @@ const CALLBACK_URL = 'http://localhost:3001/webhooks/account-service';
 const BASE_URL = 'http://localhost:3000';
 // Sleep duration between successive requests.
 const SLEEP_DURATION = 0.1;
-const BLOCKTIME_SECONDS = 12;
+const BLOCKTIME_SECONDS = 13; // Add 1 second for additional loop buffer


Perhaps there is a better way to do this, but it works consistently.

wilwade · 2024-08-06T17:44:35Z

services/account/rust-webhook-server/src/main.rs

@@ -22,7 +22,7 @@ const WEBHOOK_ENDPOINT: &str = "/account-service";

 #[actix_web::main]
 async fn main() -> Result<(), impl Error> {
-    const HOST: Ipv4Addr = Ipv4Addr::new(127, 0, 0, 1);
+    const HOST: Ipv4Addr = Ipv4Addr::new(0, 0, 0, 0);


Allows this to bind to the external interface in docker

wilwade · 2024-08-06T17:45:15Z

services/content-watcher/k6-test/README.md

Added the basic health check to the content watcher so everything has k6 tests

wilwade · 2024-08-06T17:45:52Z

tools/ci-k6/main.mjs

Here's the chain setup script

mattheworris · 2024-08-06T19:37:58Z

docker-compose-k6.account.yaml

+      ipfs:
+        condition: service_healthy
+      account-service-webhook:
+        condition: service_healthy


Cool, didn't know you could have conditions on dependencies.

mattheworris · 2024-08-06T19:54:29Z

services/account/k6-test/new-sign-up.k6.js

-    console.log('Block finalized. Checking callback...');
-    checkCallback();
+    console.log('Block finalized. Checking callback...', { referenceId });
+    checkCallback(referenceId);


Nice, love this improvement.

mattheworris

LGTM!!!

wilwade force-pushed the chore/load-test-ci-317 branch 5 times, most recently from 2e1e1ba to a9b3da4 Compare August 6, 2024 12:53

wilwade added 13 commits August 6, 2024 16:21

Initial work on the workflow

1d20d8e

Run the tests Everything has tests now WIP WIP 2 Trying a different way to run the k6 tests

Authenticate Docker

4a8d6c8

Remove ipfs requirement from graph service

c7a1df0

Wait for the API to be ready before running the tests

558c814

Give a bit more time before starting Fix health check urls Fix issue with GHA script Move just to node More documentation on the wait for ready check Javascript :( Change wait method Fix endpoint

Make k6 tests use .k6.js

8fb85bd

Updating k6 in CI params to better ones

873c24f

Health check was moved to a different file

9d51f53

Fix bad urls

bd16f70

rust webhook server should bind to all interfaces

befedca

Add the rust webhook server to the docker compose for k6 account service

Fix account new-sign-up k6 test callback check

9877ccd

Signups working

a9e6b39

Remove duplicate test

f45ba92

Disable account-service-load k6 test for now

bf36d15

wilwade force-pushed the chore/load-test-ci-317 branch from e8f6973 to bf36d15 Compare August 6, 2024 16:21

wilwade commented Aug 6, 2024

View reviewed changes

.github/workflows/load-tests.yml

Copy link

Contributor Author

wilwade Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This follows the pattern from the Unit test workflow

wilwade commented Aug 6, 2024

View reviewed changes

services/account/k6-test/generate/signups.mjs Outdated

Copy link

Contributor Author

wilwade Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generator script

wilwade commented Aug 6, 2024

View reviewed changes

tools/ci-k6/main.mjs Outdated

Copy link

Contributor Author

wilwade Aug 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's the chain setup script

Remove the unused wait for ready script and long sleep

c00ff89

wilwade requested a review from mattheworris August 6, 2024 17:48

wilwade marked this pull request as ready for review August 6, 2024 17:48

wilwade mentioned this pull request Aug 6, 2024

[Account Service] [k6] Enable account-service-load.k6.js #344

Open

1 task

mattheworris reviewed Aug 6, 2024

View reviewed changes

mattheworris approved these changes Aug 6, 2024

View reviewed changes

wilwade merged commit ff4539b into main Aug 6, 2024
11 checks passed

wilwade deleted the chore/load-test-ci-317 branch August 6, 2024 20:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Chore: Load Tests run in CI #324

Chore: Load Tests run in CI #324

wilwade commented Aug 1, 2024 •

edited

Loading

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

wilwade Aug 6, 2024

mattheworris Aug 6, 2024

mattheworris Aug 6, 2024

mattheworris left a comment

Chore: Load Tests run in CI #324

Chore: Load Tests run in CI #324

Conversation

wilwade commented Aug 1, 2024 • edited Loading

Problem

Solution

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mattheworris left a comment

Choose a reason for hiding this comment

wilwade commented Aug 1, 2024 •

edited

Loading