-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Chore: Load Tests run in CI #324
Changes from all commits
1d20d8e
4a8d6c8
c7a1df0
558c814
8fb85bd
873c24f
9d51f53
bd16f70
befedca
9877ccd
a9e6b39
f45ba92
bf36d15
c00ff89
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,106 @@ | ||
name: Load Tests | ||
|
||
on: | ||
pull_request: | ||
branches: | ||
- main | ||
push: | ||
branches: | ||
- main | ||
|
||
jobs: | ||
determine-service-matrix: | ||
runs-on: ubuntu-latest | ||
outputs: | ||
services: ${{ steps.determine-matrix.outputs.changes }} | ||
steps: | ||
- name: Check Out Repo | ||
uses: actions/checkout@v4 | ||
- uses: dorny/paths-filter@v3 | ||
id: determine-matrix | ||
with: | ||
# Note, 'common' is for future use--when code is factored out into common libraries, | ||
# but it's set up here so that the workflow logic could be tested with it present | ||
filters: | | ||
common: | ||
- '*' | ||
- '.github/**' | ||
account: | ||
- '*' | ||
- '.github/**' | ||
- 'services/account/**' | ||
graph: | ||
- '*' | ||
- '.github/**' | ||
- 'services/graph/**' | ||
content-publishing: | ||
- '*' | ||
- '.github/**' | ||
- 'services/content-publishing/**' | ||
content-watcher: | ||
- '*' | ||
- '.github/**' | ||
- 'services/content-watcher/**' | ||
|
||
build: | ||
name: '[${{ matrix.service }}] Load Test' | ||
runs-on: ubuntu-latest | ||
needs: determine-service-matrix | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
service: ${{ fromJson(needs.determine-service-matrix.outputs.services) }} | ||
exclude: | ||
- service: common | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Install Node.js | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why node? Because we need to run the setup script |
||
uses: actions/setup-node@v4 | ||
with: | ||
node-version: 20 | ||
cache: 'npm' | ||
registry-url: 'https://registry.npmjs.org' | ||
cache-dependency-path: tools/ci-k6/package-lock.json | ||
- name: Install dependencies | ||
working-directory: tools/ci-k6 | ||
run: npm ci | ||
|
||
- name: Set up Docker Buildx | ||
uses: docker/setup-buildx-action@v3 | ||
- name: Login to DockerHub | ||
uses: docker/login-action@v3 | ||
with: | ||
username: ${{secrets.DOCKERHUB_USERNAME}} | ||
password: ${{secrets.DOCKERHUB_TOKEN}} | ||
|
||
- name: Start Frequency | ||
run: | | ||
docker compose -f docker-compose.yaml -f docker-compose-k6.${{ matrix.service }}.yaml up -d frequency | ||
sleep 5 | ||
- name: Generate Provider and Capacity | ||
working-directory: tools/ci-k6 | ||
run: npm run main | ||
|
||
# Just start the services we need... | ||
- name: Start All Services | ||
run: docker compose -f docker-compose.yaml -f docker-compose-k6.${{ matrix.service }}.yaml up -d | ||
|
||
- name: Wait for API to be ready | ||
uses: cygnetdigital/[email protected] | ||
with: | ||
url: 'http://localhost:3000/readyz' | ||
responseCode: '200' | ||
timeout: 120000 | ||
interval: 2000 | ||
|
||
- name: Setup K6 | ||
uses: grafana/setup-k6-action@v1 | ||
- name: Run k6 Tests | ||
run: | | ||
for test_file in services/${{ matrix.service }}/k6-test/*.k6.js; do | ||
k6 run --no-color --quiet --no-usage-report "$test_file" || exit 1 | ||
done | ||
- name: Stop Docker Compose | ||
if: always() | ||
run: docker compose -f docker-compose.yaml -f docker-compose-k6.${{ matrix.service }}.yaml down |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Each service has a docker-compose-k6.{service}.yaml. This overrides the base
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# yaml-language-server: $schema=https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json | ||
|
||
services: | ||
account-service-api: | ||
pull_policy: never | ||
build: | ||
context: . | ||
dockerfile: ./Docker/Dockerfile.account | ||
command: ['api'] | ||
ports: | ||
- 3000:3000 | ||
volumes: !reset [] | ||
depends_on: !override | ||
redis: | ||
condition: service_started | ||
ipfs: | ||
condition: service_healthy | ||
account-service-webhook: | ||
condition: service_healthy | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Cool, didn't know you could have conditions on dependencies. |
||
environment: | ||
PROVIDER_BASE_URL: 'http://account-service-webhook:3001/webhooks/account-service' | ||
WEBHOOK_BASE_URL: 'http://account-service-webhook:3001/webhooks' | ||
|
||
account-service-worker: | ||
pull_policy: never | ||
build: | ||
context: . | ||
dockerfile: ./Docker/Dockerfile.account | ||
command: ['worker'] | ||
volumes: !reset [] | ||
depends_on: !override | ||
- redis | ||
- ipfs | ||
environment: | ||
PROVIDER_BASE_URL: 'http://account-service-webhook:3001/webhooks/account-service' | ||
WEBHOOK_BASE_URL: 'http://account-service-webhook:3001/webhooks' | ||
|
||
account-service-webhook: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Great example of an additional service needed. |
||
image: rust:1.80.0 | ||
volumes: | ||
- ./services/account/rust-webhook-server:/app | ||
ports: | ||
- '3001:3001' | ||
command: sh -c "cd /app && cargo run" | ||
healthcheck: | ||
test: ['CMD', 'curl', '-f', 'http://127.0.0.1:3001/webhooks/health'] | ||
interval: 30s | ||
timeout: 10s | ||
retries: 3 | ||
start_period: 40s | ||
networks: | ||
- amplica-gateway | ||
|
||
gateway-base: | ||
profiles: | ||
- skip | ||
Comment on lines
+55
to
+56
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is so these services don't start by default. Easiest way to do this I found |
||
content-publishing-service-worker: | ||
profiles: | ||
- skip | ||
content-publishing-service-api: | ||
profiles: | ||
- skip | ||
content-watcher-service: | ||
profiles: | ||
- skip | ||
graph-service-api: | ||
profiles: | ||
- skip | ||
graph-service-worker: | ||
profiles: | ||
- skip |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
# yaml-language-server: $schema=https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json | ||
|
||
services: | ||
content-publishing-service-api: | ||
pull_policy: never | ||
build: | ||
context: . | ||
dockerfile: ./Docker/Dockerfile.content-publishing | ||
command: ['api'] | ||
ports: | ||
- 3000:3000 | ||
volumes: !reset [] | ||
depends_on: !override | ||
- redis | ||
- ipfs | ||
|
||
content-publishing-service-worker: | ||
pull_policy: never | ||
build: | ||
context: . | ||
dockerfile: ./Docker/Dockerfile.content-publishing | ||
command: ['worker'] | ||
volumes: !reset [] | ||
depends_on: !override | ||
- redis | ||
- ipfs | ||
|
||
gateway-base: | ||
profiles: | ||
- skip | ||
account-service-worker: | ||
profiles: | ||
- skip | ||
account-service-api: | ||
profiles: | ||
- skip | ||
content-watcher-service: | ||
profiles: | ||
- skip | ||
graph-service-api: | ||
profiles: | ||
- skip | ||
graph-service-worker: | ||
profiles: | ||
- skip |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
# yaml-language-server: $schema=https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json | ||
|
||
services: | ||
content-watcher-service: | ||
pull_policy: never | ||
build: | ||
context: . | ||
dockerfile: ./Docker/Dockerfile.content-watcher | ||
ports: | ||
- 3000:3000 | ||
volumes: !reset [] | ||
depends_on: !override | ||
- redis | ||
- ipfs | ||
|
||
gateway-base: | ||
profiles: | ||
- skip | ||
content-publishing-service-worker: | ||
profiles: | ||
- skip | ||
content-publishing-service-api: | ||
profiles: | ||
- skip | ||
account-service-worker: | ||
profiles: | ||
- skip | ||
account-service-api: | ||
profiles: | ||
- skip | ||
graph-service-api: | ||
profiles: | ||
- skip | ||
graph-service-worker: | ||
profiles: | ||
- skip |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
# yaml-language-server: $schema=https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json | ||
|
||
services: | ||
graph-service-api: | ||
pull_policy: never | ||
build: | ||
context: . | ||
dockerfile: ./Docker/Dockerfile.graph | ||
command: ['api'] | ||
ports: | ||
- 3000:3000 | ||
volumes: !reset [] | ||
depends_on: !override | ||
- redis | ||
|
||
graph-service-worker: | ||
pull_policy: never | ||
build: | ||
context: . | ||
dockerfile: ./Docker/Dockerfile.graph | ||
command: ['worker'] | ||
volumes: !reset [] | ||
depends_on: !override | ||
- redis | ||
|
||
gateway-base: | ||
profiles: | ||
- skip | ||
content-publishing-service-worker: | ||
profiles: | ||
- skip | ||
content-publishing-service-api: | ||
profiles: | ||
- skip | ||
content-watcher-service: | ||
profiles: | ||
- skip | ||
account-service-api: | ||
profiles: | ||
- skip | ||
account-service-worker: | ||
profiles: | ||
- skip |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
# yaml-language-server: $schema=https://raw.githubusercontent.com/compose-spec/compose-spec/master/schema/compose-spec.json | ||
x-common-environment: &common-environment | ||
FREQUENCY_URL: ${FREQUENCY_URL:-ws://frequency:9944} | ||
FREQUENCY_HTTP_URL: ${FREQUENCY_HTTP_URL:-http://localhost:9944} | ||
|
@@ -55,7 +56,7 @@ services: | |
- redis_data:/data/redis | ||
|
||
frequency: | ||
image: dsnp/instant-seal-node-with-deployed-schemas:latest | ||
image: frequencychain/standalone-node:v1.13.0-rc3 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Going ahead and switching to the new base image |
||
# We need to specify the platform because it's the only image | ||
# built by Frequency at the moment, and auto-pull won't work otherwise | ||
platform: linux/amd64 | ||
|
@@ -159,7 +160,6 @@ services: | |
- graph_node_cache:/app/services/graph/node_modules | ||
depends_on: | ||
- redis | ||
- ipfs | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. graph service doesn't need |
||
- gateway-base | ||
networks: | ||
- amplica-gateway | ||
|
@@ -174,7 +174,6 @@ services: | |
- graph_node_cache:/app/services/graph/node_modules | ||
depends_on: | ||
- redis | ||
- ipfs | ||
- gateway-base | ||
networks: | ||
- amplica-gateway | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,7 +10,7 @@ | |
<mxCell id="34" value="" style="rounded=1;whiteSpace=wrap;html=1;" parent="1" vertex="1"> | ||
<mxGeometry x="280" y="330" width="390" height="330" as="geometry" /> | ||
</mxCell> | ||
<mxCell id="35" value="GET/api/health" style="endArrow=classic;html=1;rounded=0;fillColor=#d5e8d4;strokeColor=#82b366;strokeWidth=3;exitX=0.996;exitY=0.17;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.172;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="45" target="41" edge="1"> | ||
<mxCell id="35" value="GET/healthz" style="endArrow=classic;html=1;rounded=0;fillColor=#d5e8d4;strokeColor=#82b366;strokeWidth=3;exitX=0.996;exitY=0.17;exitDx=0;exitDy=0;exitPerimeter=0;entryX=0;entryY=0.172;entryDx=0;entryDy=0;entryPerimeter=0;" parent="1" source="45" target="41" edge="1"> | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is the correct one, but I didn't generate the pngs for it. Let me know if it is correct. |
||
<mxGeometry width="50" height="50" relative="1" as="geometry"> | ||
<mxPoint x="-260" y="400" as="sourcePoint" /> | ||
<mxPoint x="-140" y="400" as="targetPoint" /> | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
# Generated k6 script | ||
|
||
The `account-service-load.js` file contains most of the Swagger/OpenAPI specification and you can customize it to your needs. | ||
The `health-check.js` file contains a simple health check script that can be used to check the health of the service. | ||
The `account-service-load.k6.js` file contains most of the Swagger/OpenAPI specification and you can customize it to your needs. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. First place this shows up... So why?
|
||
The `health-check.k6.js` file contains a simple health check script that can be used to check the health of the service. | ||
|
||
Global header variables are defined at the top of the file, like `api_key`. Each path in the specification is converted into a [group](https://docs.k6.io/docs/tags-and-groups) in k6 and each group contains all the request methods related to that path. Path and query parameters are extracted from the specification and put at the start of the group. The URL is constructed from the base URL plus path and query. | ||
|
||
|
@@ -22,5 +22,11 @@ To run the script, you need to have k6 installed. You can download it from [here | |
To run the script, execute the following command: | ||
|
||
```bash | ||
k6 run health-check.js | ||
k6 run health-check.k6.js | ||
``` | ||
|
||
## Generating Test Data | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added a script to generate the information needed to run this test. I went for committing the generated code, as it works fine for running in CI and most runs. I'd be open to generating this as needed as well. |
||
|
||
So that k6 doesn't have to interact with the chain, we can generate data sepatately and then use it for the tests. | ||
|
||
`npm run generate:signup` for example will re-generate the `signups.gen.js` file with 100 generated valid signup payloads. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This follows the pattern from the Unit test workflow