diff --git a/building/config.json b/building/config.json index 9cee04c6..2a7de427 100644 --- a/building/config.json +++ b/building/config.json @@ -661,6 +661,12 @@ "path": "building/tracks/stories/tuples.santas-helper.md", "title": "Santa's Helper" }, + { + "uuid": "b99fb54b-a9ce-4a50-bca4-6a928cc77ec6", + "slug": "tracks/ci/workflows", + "path": "building/tracks/ci/workflows.md", + "title": "Workflows" + }, { "uuid": "191b0fa1-96e2-48a6-ad2e-c34f57443799", "slug": "tracks/ci/migrating-from-travis", diff --git a/building/tracks/ci/README.md b/building/tracks/ci/README.md index d50c46aa..639d6379 100644 --- a/building/tracks/ci/README.md +++ b/building/tracks/ci/README.md @@ -2,3 +2,7 @@ At Exercism, we use [GitHub Actions](https://github.com/features/actions) to handle our [continuous integration](https://en.wikipedia.org/wiki/Continuous_integration) (CI) and [continuous deployment](https://en.wikipedia.org/wiki/Continuous_deployment) (CD) needs. This includes running tests, formatting things, and deploying things. + +For more information, check: + +- [Workflows](/docs/building/tracks/ci/workflows) diff --git a/building/tracks/ci/workflows.md b/building/tracks/ci/workflows.md new file mode 100644 index 00000000..93c4d68e --- /dev/null +++ b/building/tracks/ci/workflows.md @@ -0,0 +1,41 @@ +# Workflows + +GitHub Actions uses the concept of _workflows_, which are scripts that run automatically whenever a specific event occurs (e.g. pushing a commit). + +Each GitHub Actions workflow is defined in a `.yml` file in the `.github/workflows` directory. +For information on workflows, check the following docs: + +- [Workflow syntax](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions) +- [Choosing when your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/triggering-a-workflow) +- [Choosing where your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-where-your-workflow-runs) +- [Choose what your workflow does](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does) +- [Writing workflows](https://docs.github.com/en/actions/writing-workflows) +- [Best practices](/docs/building/github/gha-best-practices) + +## Shared workflows + +Some workflows are shared across repositories. +These workflows _should not be changed_. + +### General workflows + +- `sync-labels.yml`: automatically syncs the repository's labels from a `labels.yml` file + +### Track-specific workflows + +- `configlet.yml`: runs the [configlet tool](/docs/building/configlet), which checks if a track's (configuration) files are properly structured - both syntactically and semantically +- `no-important-files-changed.yml`: checks if pull requests would cause all existing solutions of one or more changes exercises to be re-run +- `test.yml`: verify the track's exercises + +### Tooling-specific workflows + +- `deploy.yml`: deploy the tooling Docker image to Docker Hub and ECR + +## Custom workflows + +Maintainers are free to add custom workflows to their repos. +Examples of such workflows could be: + +- Linting of shell scripts ([example](https://github.com/exercism/configlet/blob/3baa09608c8ac327315c887608c13a68ae8ac359/.github/workflows/shellcheck.yml)) +- Auto-commenting on pull requests ([example](https://github.com/exercism/elixir/blob/b737f80cc93fcfdec6c53acb7361819834782470/.github/workflows/pr-comment.yml)) +- Etc. diff --git a/building/tracks/new/setup-continuous-integration.md b/building/tracks/new/setup-continuous-integration.md index 2feb2f15..a2df7279 100644 --- a/building/tracks/new/setup-continuous-integration.md +++ b/building/tracks/new/setup-continuous-integration.md @@ -6,39 +6,21 @@ Setting up Continuous Integration (CI) for your track is very important, as it h Our tracks (and other repositories) use [GitHub Actions](https://docs.github.com/en/actions) to run their CI. GitHub Actions uses the concept of _workflows_, which are scripts that run automatically whenever a specific event occurs (e.g. pushing a commit). - -Each GitHub Actions workflow is defined in a `.yml` file in the `.github/workflows` directory. -For information on workflows, check the following docs: - -- [Workflow syntax](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions) -- [Choosing when your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/triggering-a-workflow) -- [Choosing where your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-where-your-workflow-runs) -- [Choose what your workflow does](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does) -- [Writing workflows](https://docs.github.com/en/actions/writing-workflows) -- [Best practices](/docs/building/github/gha-best-practices) - -## Pre-defined workflows - -A track repository contains several pre-defined workflows: - -- `configlet.yml`: runs the [configlet tool](/docs/building/configlet), which checks if a track's (configuration) files are properly structured - both syntactically and semantically -- `no-important-files-changed.yml`: checks if pull requests would cause all existing solutions of one or more changes exercises to be re-run -- `sync-labels.yml`: automatically syncs the repository's labels from a `labels.yml` file -- `test.yml`: verify the track's exercises - -Of these workflows, _only_ the `test.yml` workflow requires manual work. -The other workflows should not be changed (we keep them up-to-date automatically). +For more information on workflow, check the [workflows docs](/docs/building/tracks/ci/workflows). ## Test workflow -The test workflow should verify the track's exercises. +Each track comes with a `test.yml` workflow. +This workflow should verify that the track's exercises are in proper shape. +The workflow is setup to run automatically (in GitHub Actions terminology: is _triggered_) when a push is made to the `main` branch or to a pull request's branch. + The workflow itself should not do much, except for: - Checking out the code (already implemented) - Installing dependencies (e.g. installing an SDK, optional) -- Running the script to verify the exercises (already implemented) +- Running the verify exercises script (already implemented) -### Verify exercises script +## Implement the verify exercises script As mentioned, the exercises are verified via a script, namely the `bin/verify-exercises` (bash) script. This script is _almost_ done, and does the following: @@ -51,7 +33,7 @@ This script is _almost_ done, and does the following: The `run_tests` and `unskip_tests` functions are the only things that you need to implement. -### Unskipping tests +### Unskip tests If your track supports skipping tests, we must ensure that no tests are skipped when verifying an exercise's example/exemplar solution. In general, there are two ways in which tracks support "unskipping" tests: @@ -69,7 +51,7 @@ The `unskip_test` function runs on a copy of an exercise directory, so feel free If unskipping tests requires an environment variable to be set, make sure that it is set in the `run_tests` function. -### Running tests +### Run tests The `run_tests` function is responsible for running the tests of an exercise. When the function is called, the example/exemplar files will already have been copied to (stub) solution files, so you only need to call the right command to run the tests. @@ -80,7 +62,10 @@ The function must return a zero as the exit code if all tests pass, otherwise re The `run_tests` function runs on a copy of an exercise directory, so feel free to modify the files as you see fit. ``` -### Example: Arturo track +### Option 1: use language tooling + +The default option for the verify exercises script is to use the language's tooling (SDK/binary/etc.). +It assumes (and possibly checks) that the language tooling is installed, a This is what the [`bin/verify-exercises` file](https://github.com/exercism/arturo/blob/79560f853f5cb8e2f3f0a07cbb8fcce8438ee996/bin/verify-exercises) looks file for the Arturo track: @@ -164,10 +149,22 @@ and runs the tests via the `arturo` command: arturo tester.art ``` +### Option 2: use the test runner Docker image + +In this option, we're using the fact that each track must have a test runner which already knows how to verify exercises. +To enable this option, we first need to download (pull) the track's test runner Docker image and then run the `bin/verify-exercises` script, which is modified to use the test runner Docker image to run the tests. + +```exercism/note +The main benefit of this approach is that it best mimics how tests are being run in production (on the website). +With the approach, it is less likely that things will fail in production that passed in CI. +The downside of this approach is that it likely is slower, due to having to pull the Docker image and the overhead of Docker. +``` + +```` + ## Implement the test workflow -The goal of the test workflow (defined in `.github/workflows/test.yml`) is to automatically verify that the track's exercises are in proper shape. -The workflow is setup to run automatically (in GitHub Actions terminology: is _triggered_) when a push is made to the `main` branch or to a pull request's branch. +Now that the `verify-exercises` script is There are three options when implementing this workflow: @@ -208,7 +205,7 @@ jobs: - name: Verify all exercises run: bin/verify-exercises -``` +```` #### Option 2: running the verify exercises script within test runner Docker image