Docs: Replace Vitest plugin with Test addon #29315

kylegach · 2024-10-09T22:09:37Z

(Part of #29088)

What I did

Replace /docs/writing-tests/vitest-plugin with /docs/writing-tests/test-addon
- Redirect added here
Revise and add to the content to cover the Storybook UI enhancements and other details

Checklist for Contributors

Manual testing

In this repo, check out this branch, addon-test-docs
In the web repo
1. Run npm run sync-docs
2. In another terminal tab, run npm run dev
3. Open the docs page http://localhost:3000/docs/writing-tests/test-addon

Documentation

Add or update documentation reflecting your changes
If you are deprecating/removing a feature, make sure to update
MIGRATION.MD

Checklist for Maintainers

When this PR is ready for testing, make sure to add ci:normal, ci:merged or ci:daily GH label to it to run a specific set of sandboxes. The particular set of sandboxes can be found in code/lib/cli/src/sandbox-templates.ts
Make sure this PR contains one of the labels below:
Available labels
- bug: Internal changes that fixes incorrect behavior.
- maintenance: User-facing maintenance tasks.
- dependencies: Upgrading (sometimes downgrading) dependencies.
- build: Internal-facing build tooling & test updates. Will not show up in release changelog.
- cleanup: Minor cleanup style change. Will not show up in release changelog.
- documentation: Documentation only changes. Will not show up in release changelog.
- feature request: Introducing a new feature.
- BREAKING CHANGE: Changes that break compatibility in some way with current major version.
- other: Changes that don't fit in the above categories.

Greptile Summary

This pull request replaces the Vitest plugin documentation with the new Test addon documentation, providing a comprehensive guide for using the Test addon in Storybook.

Renamed docs/writing-tests/vitest-plugin.mdx to docs/writing-tests/test-addon.mdx
Updated content to cover Storybook UI enhancements for running tests directly in the UI
Added sections on installation, setup, usage, configuration, and comparison with the test runner
Included new information on debugging and running tests in CI environments
Updated docs/_snippets/vitest-plugin-vitest-workspace.md with renderer-specific configurations

kylegach · 2024-10-09T22:12:08Z

docs/writing-tests/test-addon.mdx

+#### Debugging
+
+While the plugin does not require Storybook to run when testing, you may still want to run Storybook to debug your tests. To enable this, provide the [`storybookScript` option](#storybookscript) in the plugin configuration. When you run Vitest in watch mode, the plugin will start Storybook using this script and provide links to the story in the output on test failures. This allows you to quickly jump to the story in Storybook to debug the issue.
+
+You can also provide a [`storybookUrl` option](#storybookurl) to the plugin configuration. When you're not using watch mode and tests fail, the plugin will provide a link to the story using this URL in the output. This is useful when [running tests in CI](#in-ci) or other environments where Storybook is not already running.
+
+![Screenshot of test failure in the console, showing a failure with a link to the story](../_assets/writing-tests/vitest-plugin-test-failure.png)


This is the same content as the previous debugging section that was immediately under Usage. I moved it to be under CLI because the page now presents the SB UI usage as primary and this section only pertains to CLI.

nx-cloud · 2024-10-09T22:13:53Z

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 15c9433. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this CI Pipeline Execution

✅ Successfully ran 1 target

nx run-many -t build --parallel=3

Sent with 💌 from NxCloud.

docs/writing-tests/test-addon.mdx

kylegach · 2024-10-18T16:19:11Z

docs/writing-tests/test-addon.mdx

+### What happens if Vitest itself has an error?
+
+Sometimes tests can fail because of errors within Vitest itself. When this happens, the test module in the Storybook UI will alert you to the error, and you can click a link to see more details. The error will also be logged to the console.
+
+{/* TODO: screenshot of test module with Vitest error */}
+
+Vitest offers [troubleshooting help for common errors](https://vitest.dev/guide/common-errors.html).
+
+### What happens when there are different test results in multiple environments?
+
+When you run tests with this addon, they are run as Vitest tests with whatever configuration you have set up in your project. By default, they will run in browser mode, using Playwright's Chromium browser. Sometimes, tests will fail when run in the addon (or via CLI), but then pass when viewed in the Component tests addon panel (or vice-versa). This can happen because the tests are run in different environments, which can have different behaviors.


@ghengeveld @valentinpalkovic @yannbf — Is there more help we can offer for either of these sections?

greptile-apps

_{2 file(s) reviewed, 1 comment(s)}
_{Edit PR Review Bot Settings | Greptile}

docs/writing-tests/test-addon.mdx

jonniebigodes

This is shaping up nicely, I left some items to be looked into when possible so that we can get this one and published

docs/writing-tests/test-addon.mdx

jonniebigodes · 2024-10-22T11:52:53Z

docs/writing-tests/test-addon.mdx

+After running your tests, you will now see status indicators on stories and components for their pass, fail, or error state. You can press on these indicators to see more details and jump straight to the failure in the Component tests addon panel. That panel provides an interactive debugger for your component test, allowing you to step through each simulated behavior or assertion.
+
+<Callout variant="info" icon="ℹ️">
+
+The Component tests addon panel replaces the [Interactions addon panel](../essentials/interactions.mdx) when the Test addon is installed. While the testing mechanism is different, the functionality of the addon panel itself remains the same.
+
+</Callout>
+
+The test module will also show you the total number of tests run, the number of tests that passed, and the number of tests that failed or errored. You can press the failure number to filter the sidebar to only those stories that failed.
+
+<Video src="../_assets/writing-tests/addon-test-filter-sidebar.mp4" />


I believe this should be reframed and included inside a section on previewing the test results to educate the user on how to do it. There's a lot of information for the user to digest, which can result in confusion. The bonus is that we can use said section in the future for other learning materials, blogs, etc.

I disagree. The section first describes how to run tests and then how to review them. One must come after the other. Splitting that into two sub-sections would be overhead.

I'm going to go ahead and merge this into the parent PR so that it goes out with the actual feature.

If you really feel strongly about this feedback (or anything else), please ping me in a comment over there.

Co-authored-by: Yann Braga <[email protected]>

Co-authored-by: Michael Shilman <[email protected]>

kylegach added the documentation label Oct 9, 2024

kylegach self-assigned this Oct 9, 2024

kylegach commented Oct 9, 2024

View reviewed changes

yannbf reviewed Oct 10, 2024

View reviewed changes

docs/writing-tests/test-addon.mdx Outdated Show resolved Hide resolved

yannbf reviewed Oct 11, 2024

View reviewed changes

docs/writing-tests/test-addon.mdx Outdated Show resolved Hide resolved

shilman reviewed Oct 11, 2024

View reviewed changes

docs/writing-tests/test-addon.mdx Outdated Show resolved Hide resolved

kylegach mentioned this pull request Oct 15, 2024

Add redirect /writing-tests/vitest-plugin -> /writing-tests/test-addon storybookjs/web#231

Closed

kylegach commented Oct 18, 2024

View reviewed changes

kylegach added the ci:docs Run the CI jobs for documentation checks only. label Oct 18, 2024

kylegach marked this pull request as ready for review October 18, 2024 16:19

kylegach requested a review from jonniebigodes as a code owner October 18, 2024 16:19

kylegach requested review from jonniebigodes and removed request for jonniebigodes October 18, 2024 16:19

greptile-apps bot reviewed Oct 18, 2024

View reviewed changes

jonniebigodes reviewed Oct 22, 2024

View reviewed changes

docs/writing-tests/test-addon.mdx Outdated Show resolved Hide resolved

jonniebigodes requested changes Oct 22, 2024

View reviewed changes

kylegach changed the base branch from next to unified-ui-testing October 23, 2024 04:45

kylegach and others added 9 commits October 22, 2024 22:51

Docs: Replace Vitest plugin with Test addon

f728e29

Update required Vitest version

7960bdd

Co-authored-by: Yann Braga <[email protected]>

Add Vitest attribution in experimental callout

4fc0b7c

Co-authored-by: Michael Shilman <[email protected]>

Correct status indicator guidance

524cec4

Clarify Next.js requirements

1220592

Add assets

20879cb

Add Vitest error screenshot

e40db8b

Address feedback

bcecdb9

Improve videos

15c9433

kylegach force-pushed the addon-test-docs branch from a97f63a to 15c9433 Compare October 23, 2024 04:52

kylegach merged commit f3b479a into unified-ui-testing Oct 23, 2024
7 checks passed

kylegach deleted the addon-test-docs branch October 23, 2024 04:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docs: Replace Vitest plugin with Test addon #29315

Docs: Replace Vitest plugin with Test addon #29315

kylegach commented Oct 9, 2024 •

edited

Loading

kylegach Oct 9, 2024

nx-cloud bot commented Oct 9, 2024 •

edited

Loading

kylegach Oct 18, 2024

greptile-apps bot left a comment

jonniebigodes left a comment

jonniebigodes Oct 22, 2024

kylegach Oct 22, 2024

kylegach Oct 23, 2024

Docs: Replace Vitest plugin with Test addon #29315

Docs: Replace Vitest plugin with Test addon #29315

Conversation

kylegach commented Oct 9, 2024 • edited Loading

What I did

Checklist for Contributors

Manual testing

Documentation

Checklist for Maintainers

Greptile Summary

kylegach Oct 9, 2024

Choose a reason for hiding this comment

nx-cloud bot commented Oct 9, 2024 • edited Loading

☁️ Nx Cloud Report

kylegach Oct 18, 2024

Choose a reason for hiding this comment

greptile-apps bot left a comment

Choose a reason for hiding this comment

jonniebigodes left a comment

Choose a reason for hiding this comment

jonniebigodes Oct 22, 2024

Choose a reason for hiding this comment

kylegach Oct 22, 2024

Choose a reason for hiding this comment

kylegach Oct 23, 2024

Choose a reason for hiding this comment

kylegach commented Oct 9, 2024 •

edited

Loading

nx-cloud bot commented Oct 9, 2024 •

edited

Loading