d2l-test-reporting

Helper package for generating reports for use with the D2L test reporting framework. This package is meant to be used in conjunction with this GitHub Action.

Note

If you have any questions, concerns or just want to chat feel free to reach out in #test-reporting (D2L employee accessible only).

Installation

npm install d2l-test-reporting

Usage

This library provides a reporters for many of the test execution frameworks we use, if one for your test runner framework isn't provided please file an issue so we can look into adding it to our set of reporters.

Reporters

Mocha

Please consult the official documentation for Mocha to see how to use reporters. Below is an example of how to add the reporter provided by this package.

module.exports = {
  spec: 'test/*.test.js',
  reporter: 'd2l-test-reporting/reporters/mocha.js',
  reporterOptions: [
    'reportPath=./d2l-test-report.json', // optional
    'reportConfigurationPath=./d2l-test-reporting.config.json' // optional
  ]
};

Inputs

• reportPath: path to output the reporter to, relative to current working directory. Not required. Defaults to ./d2l-test-report.json.
• reportConfigurationPath: path to the D2L test reporting configuration file for mapping test type, experience and tool to test code. Not required. Defaults to ./d2l-test-reporting.config.json.

Playwright

Please consult the official documentation for Playwright to see how to use reporters. Below is an example of how to add the reporter provided by this package.

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  reporter: [
    [
      'd2l-test-reporting/reporters/playwright.js',
      {
        reportPath: './d2l-test-report.json', // optional
        reportConfigurationPath: './d2l-test-reporting.config.json' // optional
      }
    ],
    ['list']
  ],
  testDir: '../',
  testMatch: '*.test.js',
  projects: [{
    name: 'firefox',
    use: devices['Desktop Firefox'],
    testMatch: 'firefox/*.test.js'
  }]
});

Warning

Currently the merge-reports command is not fully supported due to a lack of browser/launcher information preservation with the blob reporter. If you are using a GitHub matrix run this may result in partial data showing in the reporting dashboard as it becomes available.

Inputs

• reportPath: path to output the reporter to, relative to current working directory. Not required. Defaults to ./d2l-test-report.json.
• reportConfigurationPath: path to the D2L test reporting configuration file for mapping test type, experience and tool to test code. Not required. Defaults to ./d2l-test-reporting.config.json.

@web/test-runner

Please consult the official documentation for @web/test-runner to see how to use reporters. Below is an example of how to add the reporter provided by this package.

import { defaultReporter } from '@web/test-runner';
import { reporter } from 'd2l-test-reporting/reporters/web-test-runner.js';

export default {
  reporters: [
    defaultReporter(),
    reporter({
      reportPath: './d2l-test-report.json', // optional
      reportConfigurationPath: './d2l-test-reporting.config.json' // optional
    })
  ],
  files: 'test/component-*.test.js',
  groups: [{
    name: 'group',
    files: 'test/group/component-*.test.js'
  }]
};

Inputs

• reportPath: path to output the reporter to, relative to current working directory. Not required. Defaults to ./d2l-test-report.json.
• reportConfigurationPath: path to the D2L test reporting configuration file for mapping test type, experience and tool to test code. Not required. Defaults to ./d2l-test-reporting.config.json.

Configuration

To have the test type, experience and tool mapped to test code, a D2L test reporting configuration file is required when using one of the reporters provided in this package.

Below are examples of how to create the config file. Note that the type field will end up lowercase in the report.

Please see Automated Testing Definitions on confluence for the list of test types that should be used when creating the D2L test reporting configuration file.

{
  "type": "UI Visual Diff",
  "experience": "Experience",
  "tool": "Tool",
}

{
  "type": "UI E2E",
  "overrides": [
    {
      "pattern": "tests/account-settings/**/*",
      "experience": "Administration",
      "tool": "Account Settings"
    },
    {
      "pattern": "tests/announcements/**/*",
      "experience": "Teaching & Learning",
      "tool": "Announcements"
    },
    {
      "pattern": "tests/rubrics.test.js",
      "experience": "Assessment",
      "tool": "Rubrics"
    }
  ]
}

Report Format

For details on what the schema that the various custom reporters output looks like see Report Format.

Note

The report format is required in order to upload items to the back-end using the corresponding GitHub Action.

Developing

After cloning the repository make sure to install dependencies.

npm ci

Linting

# currently only eslint
npm run lint

# eslint only
npm run lint:eslint

Fixing

# currently only eslint
npm run fix

# eslint only
npm run fix:eslint

Testing

# lint, unit tests and integration tests
npm test

# unit tests and integration tests only
npm run test:all

# unit tests only
npm run test:unit

# integration tests only
npm run test:integration