Merge pull request #24 from buildkite/promote-test-spliiting-page-con…

…tent

Delayed Sept launch PR for Test Engine updates - Promote Test splitting page within the Test Engine docs + add Usage and billing page with Test executions content

app/models/beta_pages.rb

@@ -1,7 +1,8 @@
 class BetaPages
   def self.all
     [
-      'test-analytics/test-ownership'
+      'path/page-name',
+      'path/last-page-name-in-list-without-trailing-comma'
     ]
   end
 end

app/views/homepage/_popular_guides.html.erb

@@ -32,7 +32,7 @@
     },
     {
       id: "glossary",
-      title: "Glossary",
+      title: "Pipelines glossary",
       summary: "A list of common terms that describe key concepts",
       link: "/docs/pipelines/glossary",
     }

data/content/test_splitting_env.yaml

@@ -1,7 +1,7 @@
 predefined:
   - name: BUILDKITE_BUILD_ID
     desc:
-      - The UUID of the pipeline build. Test Splitter uses this UUID along with `BUILDKITE_STEP_ID` to uniquely identify the test plan.
+      - The UUID of the pipeline build. The Test Engine Client uses this UUID along with `BUILDKITE_STEP_ID` to uniquely identify the test plan.
   - name: BUILDKITE_JOB_ID
     desc:
       - The UUID of the job in the pipeline's build.
@@ -18,40 +18,90 @@ predefined:
       - Ensure you configure `parallelism` in your pipeline definition. Learn more about parallel build steps in [Concurrency and parallelism](https://buildkite.com/docs/pipelines/controlling-concurrency#concurrency-and-parallelism).
   - name: BUILDKITE_STEP_ID
     desc:
-      - The UUID of the step group in the pipeline build. Test Splitter uses this UUID along with `BUILDKITE_BUILD_ID` to uniquely identify the test plan.
+      - The UUID of the step group in the pipeline build. The Test Engine Client uses this UUID along with `BUILDKITE_BUILD_ID` to uniquely identify the test plan.
 
 mandatory:
-  - name: BUILDKITE_SPLITTER_API_ACCESS_TOKEN 
+  - name: BUILDKITE_TEST_ENGINE_API_ACCESS_TOKEN 
     desc:
-      - Buildkite API access token with `read_suites`, `read_test_plan`, and `write_test_plan` scopes. You can create an [API access token](https://buildkite.com/user/api-access-tokens) from **Personal Settings** > **API Access Tokens** in the Buildkite interface.
-  - name: BUILDKITE_SPLITTER_SUITE_SLUG 
+      - Buildkite API access token with `read_suites`, `read_test_plan`, and `write_test_plan` scopes. You can create an [API access token](https://buildkite.com/user/api-access-tokens) from **Personal Settings** > **API Access Tokens** in the Buildkite interface. To avoid this token being tied to any one employee or person, ideally, this token should be created for a bot user account within your Buildkite organization.
+  - name: BUILDKITE_TEST_ENGINE_RESULT_PATH
     desc:
-      - The slug of your Buildkite Test Engine test suite. You can find the suite slug in the url for your test suite. 
+      - The path to store the test result. The Test Engine Client uses this environment variable to tell the runner where to store the test result. The Test Engine Client reads the test result after each test run for retries and verification.
+      - For RSpec, the result is generated using the `--format json` and `--out` CLI options, while for Jest, it is generated using the `--json` and `--outputFile` options. We have included these options in the default test command for RSpec and Jest. If you need to customize your test command, make sure to append the CLI options to save the result to a file. 
+      - Please refer to the `BUILDKITE_TEST_ENGINE_TEST_CMD` environment variable for more details.
+    note:
+      - The Test Engine Client will not delete the file after running the test, however it will be deleted by Buildkite Agent as part of the build lifecycle.
+  - name: BUILDKITE_TEST_ENGINE_RUNNER
+    desc:
+      - The test runner to use for running tests. Currently `rspec` and `jest` are supported.
+  - name: BUILDKITE_TEST_ENGINE_SUITE_SLUG 
+    desc:
+      - The slug of your Test Engine test suite. You can find the suite slug in the url for your test suite. 
       - "For example, the slug for the url: `https://buildkite.com/organizations/my-organization/analytics/suites/my-suite` is `my-suite`."
 
 optional:
-  - name: BUILDKITE_SPLITTER_DEBUG_ENABLED 
-    default: false
-    desc:
-      - A flag to enable more verbose logging.
-  - name: BUILDKITE_SPLITTER_RETRY_COUNT 
-    default: 0
-    desc:
-      - The number of retries permitted. Test Splitter runs the test command defined in `BUILDKITE_SPLITTER_TEST_CMD`, and retries only the failing tests for a maximum of `BUILDKITE_SPLITTER_RETRY_COUNT` times. For RSpec, the Test Splitter runs `BUILDKITE_SPLITTER_TEST_CMD` with `--only-failures` as the retry command.
-  - name: BUILDKITE_SPLITTER_SPLIT_BY_EXAMPLE 
-    default: false
-    desc:
-      - A flag to enable split by example. When this option is `true`, the Test Splitter will split the execution of slow test files over multiple partitions.
-  - name: BUILDKITE_SPLITTER_TEST_CMD 
-    default: bundle exec rspec {{testExamples}}
-    desc:
-      - The test command to run your tests. The Test Splitter will replace and populate the `{{testExamples}}` placeholder with the test plan.
-  - name: BUILDKITE_SPLITTER_TEST_FILE_EXCLUDE_PATTERN 
-    desc: 
-      - The glob pattern to exclude certain test files or directories. The exclusion will be applied after discovering the test files using a pattern configured with `BUILDKITE_SPLITTER_TEST_FILE_PATTERN`.
-      - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._
-  - name: BUILDKITE_SPLITTER_TEST_FILE_PATTERN 
-    default: spec/**/*_spec.rb
-    desc:
-      - The glob pattern to discover test files. You can exclude certain test files or directories from the discovered test files using a pattern that can be configured with `BUILDKITE_SPLITTER_TEST_FILE_EXCLUDE_PATTERN`.
-      - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._
+  rspec:
+    - name: BUILDKITE_TEST_ENGINE_DEBUG_ENABLED 
+      default: false
+      desc:
+        - A flag to enable more verbose logging.
+    - name: BUILDKITE_TEST_ENGINE_RETRY_CMD
+      desc:
+        - The command to retry the failed tests. The Test Engine Client will replace the `{{testExamples}}` placeholder with the failed tests. If not set, the client will use the same command defined in `BUILDKITE_TEST_ENGINE_TEST_CMD`. 
+    - name: BUILDKITE_TEST_ENGINE_RETRY_COUNT 
+      default: 0
+      desc:
+        - The number of retries. The Test Engine Client runs the test command defined in `BUILDKITE_TEST_ENGINE_TEST_CMD` and retries only the failed tests up to `BUILDKITE_TEST_ENGINE_RETRY_COUNT` times, using the retry command defined in `BUILDKITE_TEST_ENGINE_RETRY_CMD`.
+    - name: BUILDKITE_TEST_ENGINE_SPLIT_BY_EXAMPLE 
+      default: false
+      desc:
+        - A flag to enable split by example. When this option is `true`, the Test Engine Client will split the execution of slow test files over multiple partitions.
+    - name: BUILDKITE_TEST_ENGINE_TEST_CMD 
+      default: bundle exec rspec --format progress --format json --out {{resultPath}} {{testExamples}}
+      desc:
+        - The test command to run your tests. The Test Engine Client will replace the `{{testExamples}}` placeholder with the test plan.
+      note:
+        - It is necessary to include `--format json --out {{resultPath}}` in the test command, because the Test Engine Client needs to read the result after each test run.
+        - Please refer to the `BUILDKITE_TEST_ENGINE_RESULT_PATH` environment variable for more details.
+    - name: BUILDKITE_TEST_ENGINE_TEST_FILE_EXCLUDE_PATTERN 
+      desc: 
+        - The glob pattern to exclude certain test files or directories. The exclusion will be applied after discovering the test files using a pattern configured with `BUILDKITE_TEST_ENGINE_TEST_FILE_PATTERN`.
+        - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._
+    - name: BUILDKITE_TEST_ENGINE_TEST_FILE_PATTERN 
+      default: spec/**/*_spec.rb
+      desc:
+        - The glob pattern to discover test files. You can exclude certain test files or directories from the discovered test files using a pattern that can be configured with `BUILDKITE_TEST_ENGINE_TEST_FILE_EXCLUDE_PATTERN`.
+        - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._
+  jest:
+    - name: BUILDKITE_TEST_ENGINE_DEBUG_ENABLED 
+      default: false
+      desc:
+        - A flag to enable more verbose logging.
+    - name: BUILDKITE_TEST_ENGINE_RETRY_CMD
+      default: yarn test --testNamePattern '{{testNamePattern}}' --json --testLocationInResults --outputFile {{resultPath}}
+      desc:
+        - The command to retry the failed tests. The Test Engine Client will replace the `{{testNamePattern}}` placeholder with the failed tests.
+      note:
+        - It is necessary to include `--json --testLocationInResults --outputFile {{resultPath}}` in the command, because the Test Engine Client needs to read the result after each test run.
+        - Please refer to the `BUILDKITE_TEST_ENGINE_RESULT_PATH` environment variable for more details.
+    - name: BUILDKITE_TEST_ENGINE_RETRY_COUNT 
+      default: 0
+      desc:
+        - The number of retries. The Test Engine Client runs the test command defined in `BUILDKITE_TEST_ENGINE_TEST_CMD` and retries only the failed tests up to `BUILDKITE_TEST_ENGINE_RETRY_COUNT` times, using the retry command defined in `BUILDKITE_TEST_ENGINE_RETRY_CMD`.
+    - name: BUILDKITE_TEST_ENGINE_TEST_CMD 
+      default: yarn test {{testExamples}} --json --testLocationInResults --outputFile {{resultPath}}
+      desc:
+        - The test command to run your tests. The Test Engine Client will replace and populate the `{{testExamples}}` placeholder with the test plan.
+      note:
+        - It is necessary to include `--json --testLocationInResults --outputFile {{resultPath}}` in the command, because the Test Engine Client needs to read the result after each test run.
+        - Please refer to the `BUILDKITE_TEST_ENGINE_RESULT_PATH` environment variable for more details.
+    - name: BUILDKITE_TEST_ENGINE_TEST_FILE_EXCLUDE_PATTERN
+      default: "node_modules"
+      desc: 
+        - The glob pattern to exclude certain test files or directories. The exclusion will be applied after discovering the test files using a pattern configured with `BUILDKITE_TEST_ENGINE_TEST_FILE_PATTERN`.
+        - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._
+    - name: BUILDKITE_TEST_ENGINE_TEST_FILE_PATTERN 
+      default: "**/{__tests__/**/*,*.spec,*.test}.{ts,js,tsx,jsx}"
+      desc:
+        - The glob pattern to discover test files. You can exclude certain test files or directories from the discovered test files using a pattern that can be configured with `BUILDKITE_TEST_ENGINE_TEST_FILE_EXCLUDE_PATTERN`.
+        - _This option accepts the pattern syntax supported by the [zzglob](https://github.com/DrJosh9000/zzglob?tab=readme-ov-file#pattern-syntax) library._

data/nav.yml

@@ -422,23 +422,28 @@
       children:
         - name: "Configuring test suites"
           path: "test-engine/test-suites"
-        - name: "Configuring test splitting"
-          path: "test-engine/test-splitting"
         - name: "Permissions"
           path: "test-engine/permissions"
         - name: "CI environment variables"
           path: "test-engine/ci-environments"
-        - name: "Test executions"
-          path: "test-engine/test-executions"
         - name: "Public test suites"
           path: "test-engine/public-test-suites"
         - name: "Test ownership"
-          pill: "beta"
           path: "test-engine/test-ownership"
         - name: "Flaky test assignment"
           path: "test-engine/flaky-test-assignment"
-    - name: "Languages"
+        - name: "Usage and billing"
+          path: "test-engine/usage-and-billing"
+    - name: "Test splitting"
       start_expanded: true
+      children:
+        - name: "Overview"
+          path: "test-engine/test-splitting"
+        - name: "Configuring"
+          path: "test-engine/test-splitting/configuring"
+        - name: "Test Engine Client installation"
+          path: "test-engine/test-splitting/client-installation"
+    - name: "Languages"
       children:
         - name: "Ruby"
           path: "test-engine/ruby-collectors"
@@ -463,7 +468,6 @@
         - name: "Other languages"
           path: "test-engine/other-collectors"
     - name: "References"
-      start_expanded: true
       children:
         - name: "Importing JUnit XML"
           path: "test-engine/importing-junit-xml"

data/tiles.yml

@@ -6,6 +6,9 @@ test_engine_features:
   - title: "Find and fix flaky tests"
     url: "/docs/test-engine/test-suites#detecting-flaky-tests"
     desc: "Quickly identify which tests are the most disruptive for your team, and get a head-start on fixing them."
+  - title: "Reduce build times with test splitting"
+    url: "/docs/test-engine/test-splitting"
+    desc: "Split tests evenly across agents to reduce overall pipeline build times, especially for highly complex test suites."
 test_engine_guides:
   - title: "Languages"
     links:

pages/test_engine.md

@@ -19,7 +19,7 @@ _Buildkite Test Engine_ was previously called _Buildkite Test Analytics_.
 
 ## Get started
 
-Run through the 'Getting started' section of these Test Engine docs, beginning with [Configuring test suites](/docs/test-engine/test-suites) for an overview of Test Engine's concepts and functionality, followed by the appropriate test collector for project's language:
+To get started with Test Engine, and to begin setting up your _test suites_, configure the relevant supported _test collectors_ for your development project.
 
 <!-- vale off -->
 
@@ -45,6 +45,8 @@ Run through the 'Getting started' section of these Test Engine docs, beginning w
 
 You can also upload test results by importing [JSON](/docs/test-engine/importing-json) or [JUnit XML](/docs/test-engine/importing-junit-xml).
 
+Once your test collectors have been set up, you can begin configuring your test suites by running through the relevant 'Getting started' sections, beginning with [Configuring test suites](/docs/test-engine/test-suites) for an overview of Test Engine's concepts and functionality.
+
 <br/>
 
 <%= tiles "test_engine_features" %>

pages/test_engine/test_ownership.md

@@ -1,6 +1,6 @@
 # Test ownership
 
-Test ownership is critical in adopting a healthy testing culture at your organization. Defining one or more teams as test owners allows these teams to become accountable for maintaining a fast and reliable test suite, ensuring confidence when you deploy your code.
+Test ownership is critical in adopting a healthy testing culture at your organization. Defining one or more teams as test owners allows these teams to become accountable for maintaining tests within your test suite, ensuring it is fast and reliable, and providing confidence when you deploy your code.
 
 Customers on the [Pro and Enterprise plans](https://buildkite.com/pricing) can assign test ownership to [teams](/docs/test-engine/permissions#manage-teams-and-permissions).