Improve CI docs

building/config.json

@@ -661,6 +661,12 @@
     "path": "building/tracks/stories/tuples.santas-helper.md",
     "title": "Santa's Helper"
   },
+  {
+    "uuid": "b99fb54b-a9ce-4a50-bca4-6a928cc77ec6",
+    "slug": "tracks/ci/workflows",
+    "path": "building/tracks/ci/workflows.md",
+    "title": "Workflows"
+  },
   {
     "uuid": "191b0fa1-96e2-48a6-ad2e-c34f57443799",
     "slug": "tracks/ci/migrating-from-travis",

building/tracks/ci/README.md

@@ -2,3 +2,7 @@
 
 At Exercism, we use [GitHub Actions](https://github.com/features/actions) to handle our [continuous integration](https://en.wikipedia.org/wiki/Continuous_integration) (CI) and [continuous deployment](https://en.wikipedia.org/wiki/Continuous_deployment) (CD) needs.
 This includes running tests, formatting things, and deploying things.
+
+For more information, check:
+
+- [Workflows](/docs/building/tracks/ci/workflows)

building/tracks/ci/workflows.md

@@ -0,0 +1,41 @@
+# Workflows
+
+GitHub Actions uses the concept of _workflows_, which are scripts that run automatically whenever a specific event occurs (e.g. pushing a commit).
+
+Each GitHub Actions workflow is defined in a `.yml` file in the `.github/workflows` directory.
+For information on workflows, check the following docs:
+
+- [Workflow syntax](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions)
+- [Choosing when your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/triggering-a-workflow)
+- [Choosing where your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-where-your-workflow-runs)
+- [Choose what your workflow does](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does)
+- [Writing workflows](https://docs.github.com/en/actions/writing-workflows)
+- [Best practices](/docs/building/github/gha-best-practices)
+
+## Shared workflows
+
+Some workflows are shared across repositories.
+These workflows _should not be changed_.
+
+### General workflows
+
+- `sync-labels.yml`: automatically syncs the repository's labels from a `labels.yml` file
+
+### Track-specific workflows
+
+- `configlet.yml`: runs the [configlet tool](/docs/building/configlet), which checks if a track's (configuration) files are properly structured - both syntactically and semantically
+- `no-important-files-changed.yml`: checks if pull requests would cause all existing solutions of one or more changes exercises to be re-run
+- `test.yml`: verify the track's exercises
+
+### Tooling-specific workflows
+
+- `deploy.yml`: deploy the tooling Docker image to Docker Hub and ECR
+
+## Custom workflows
+
+Maintainers are free to add custom workflows to their repos.
+Examples of such workflows could be:
+
+- Linting of shell scripts ([example](https://github.com/exercism/configlet/blob/3baa09608c8ac327315c887608c13a68ae8ac359/.github/workflows/shellcheck.yml))
+- Auto-commenting on pull requests ([example](https://github.com/exercism/elixir/blob/b737f80cc93fcfdec6c53acb7361819834782470/.github/workflows/pr-comment.yml))
+- Etc.

building/tracks/new/setup-continuous-integration.md

@@ -6,39 +6,21 @@ Setting up Continuous Integration (CI) for your track is very important, as it h
 
 Our tracks (and other repositories) use [GitHub Actions](https://docs.github.com/en/actions) to run their CI.
 GitHub Actions uses the concept of _workflows_, which are scripts that run automatically whenever a specific event occurs (e.g. pushing a commit).
-
-Each GitHub Actions workflow is defined in a `.yml` file in the `.github/workflows` directory.
-For information on workflows, check the following docs:
-
-- [Workflow syntax](https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions)
-- [Choosing when your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-when-your-workflow-runs/triggering-a-workflow)
-- [Choosing where your workflow runs](https://docs.github.com/en/actions/writing-workflows/choosing-where-your-workflow-runs)
-- [Choose what your workflow does](https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does)
-- [Writing workflows](https://docs.github.com/en/actions/writing-workflows)
-- [Best practices](/docs/building/github/gha-best-practices)
-
-## Pre-defined workflows
-
-A track repository contains several pre-defined workflows:
-
-- `configlet.yml`: runs the [configlet tool](/docs/building/configlet), which checks if a track's (configuration) files are properly structured - both syntactically and semantically
-- `no-important-files-changed.yml`: checks if pull requests would cause all existing solutions of one or more changes exercises to be re-run
-- `sync-labels.yml`: automatically syncs the repository's labels from a `labels.yml` file
-- `test.yml`: verify the track's exercises
-
-Of these workflows, _only_ the `test.yml` workflow requires manual work.
-The other workflows should not be changed (we keep them up-to-date automatically).
+For more information on workflow, check the [workflows docs](/docs/building/tracks/ci/workflows).
 
 ## Test workflow
 
-The test workflow should verify the track's exercises.
+Each track comes with a `test.yml` workflow.
+This workflow should verify that the track's exercises are in proper shape.
+The workflow is setup to run automatically (in GitHub Actions terminology: is _triggered_) when a push is made to the `main` branch or to a pull request's branch.
+
 The workflow itself should not do much, except for:
 
 - Checking out the code (already implemented)
 - Installing dependencies (e.g. installing an SDK, optional)
-- Running the script to verify the exercises (already implemented)
+- Running the verify exercises script (already implemented)
 
-### Verify exercises script
+## Implement the verify exercises script
 
 As mentioned, the exercises are verified via a script, namely the `bin/verify-exercises` (bash) script.
 This script is _almost_ done, and does the following:
@@ -51,7 +33,7 @@ This script is _almost_ done, and does the following:
 
 The `run_tests` and `unskip_tests` functions are the only things that you need to implement.
 
-### Unskipping tests
+### Unskip tests
 
 If your track supports skipping tests, we must ensure that no tests are skipped when verifying an exercise's example/exemplar solution.
 In general, there are two ways in which tracks support "unskipping" tests:
@@ -69,7 +51,7 @@ The `unskip_test` function runs on a copy of an exercise directory, so feel free
 
 If unskipping tests requires an environment variable to be set, make sure that it is set in the `run_tests` function.
 
-### Running tests
+### Run tests
 
 The `run_tests` function is responsible for running the tests of an exercise.
 When the function is called, the example/exemplar files will already have been copied to (stub) solution files, so you only need to call the right command to run the tests.
@@ -80,7 +62,10 @@ The function must return a zero as the exit code if all tests pass, otherwise re
 The `run_tests` function runs on a copy of an exercise directory, so feel free to modify the files as you see fit.
 ```
 
-### Example: Arturo track
+### Option 1: use language tooling
+
+The default option for the verify exercises script is to use the language's tooling (SDK/binary/etc.).
+It assumes (and possibly checks) that the language tooling is installed, a
 
 This is what the [`bin/verify-exercises` file](https://github.com/exercism/arturo/blob/79560f853f5cb8e2f3f0a07cbb8fcce8438ee996/bin/verify-exercises) looks file for the Arturo track:
 
@@ -164,10 +149,22 @@ and runs the tests via the `arturo` command:
 arturo tester.art
 ```
 
+### Option 2: use the test runner Docker image
+
+In this option, we're using the fact that each track must have a test runner which already knows how to verify exercises.
+To enable this option, we first need to download (pull) the track's test runner Docker image and then run the `bin/verify-exercises` script, which is modified to use the test runner Docker image to run the tests.
+
+```exercism/note
+The main benefit of this approach is that it best mimics how tests are being run in production (on the website).
+With the approach, it is less likely that things will fail in production that passed in CI.
+The downside of this approach is that it likely is slower, due to having to pull the Docker image and the overhead of Docker.
+```
+
+````
+
 ## Implement the test workflow
 
-The goal of the test workflow (defined in `.github/workflows/test.yml`) is to automatically verify that the track's exercises are in proper shape.
-The workflow is setup to run automatically (in GitHub Actions terminology: is _triggered_) when a push is made to the `main` branch or to a pull request's branch.
+Now that the `verify-exercises` script is
 
 There are three options when implementing this workflow:
 
@@ -208,7 +205,7 @@ jobs:
 
       - name: Verify all exercises
         run: bin/verify-exercises
-```
+````
 
 #### Option 2: running the verify exercises script within test runner Docker image