Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(atomic): run only affected E2E tests in atomic #4484

Merged
merged 31 commits into from
Oct 10, 2024

Conversation

y-lakhdar
Copy link
Contributor

@y-lakhdar y-lakhdar commented Oct 2, 2024

This pull request aims to optimize the E2E tests by running only the affected tests.

Identifying only the affected tests by listing the changed files can be tricky in the case of E2E testing. However, since we are testing components in isolation, we can use this approach along with TypeScript AST (Abstract Syntax Tree) to build a tree representation of the atomic code structure. This allows us to create a dependency map between each E2E test file and all the source files that might affect them.

Rules

  • If a change is detected in a source file that is a dependency (direct or indirect) of an E2E test, the dependent test file will be added to the list of tests to run.
  • If a change is detected in an E2E test file (or any test-related file), it will be added to the list of tests to run.
  • If the changed file has no effect on E2E tests (e.g., updating the README), no tests will be executed.
  • If changes are detected in a different package (e.g., /headless), all tests will be executed.
  • If an error occurs in the script (e.g., no associated E2E test for a change made in an atomic component), a warning will be thrown and all tests will be executed.

Shard reduction

I also adjusted the number of shards based on the number of results. 1 shard per test suite (24 shard max)

https://coveord.atlassian.net/browse/KIT-3634

@y-lakhdar y-lakhdar changed the title poc(scope e2e test): run only affected E2E tests in atomic poc(atomic): run only affected E2E tests in atomic Oct 2, 2024
Copy link

github-actions bot commented Oct 2, 2024

Pull Request Report

PR Title

✅ Title follows the conventional commit spec.

Live demo links

Bundle Size

File Old (kb) New (kb) Change (%)
case-assist 236.6 236.6 0
commerce 339.4 339.4 0
search 412 412 0
insight 400 400 0
recommendation 248.8 248.8 0
ssr 405.7 405.7 0
ssr-commerce 351.6 351.6 0

SSR Progress

Use case SSR (#) CSR (#) Progress (%)
search 39 44 89
recommendation 0 4 0
case-assist 0 6 0
insight 0 27 0
commerce 0 15 0
Detailed logs search : buildInteractiveResult
search : buildInteractiveInstantResult
search : buildInteractiveRecentResult
search : buildInteractiveCitation
search : buildGeneratedAnswer
recommendation : missing SSR support
case-assist : missing SSR support
insight : missing SSR support
commerce : missing SSR support

@y-lakhdar y-lakhdar marked this pull request as ready for review October 3, 2024 15:41
@y-lakhdar y-lakhdar requested a review from a team as a code owner October 3, 2024 15:41
@y-lakhdar y-lakhdar changed the title poc(atomic): run only affected E2E tests in atomic ci(atomic): run only affected E2E tests in atomic Oct 8, 2024
Copy link
Collaborator

@louis-bompart louis-bompart left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's try it!

Copy link
Contributor

@alexprudhomme alexprudhomme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice, ran 1 out of 1 shard 😄

@louis-bompart louis-bompart merged commit 497ede3 into master Oct 10, 2024
92 checks passed
@louis-bompart louis-bompart deleted the KIT-3560-scope-tests branch October 10, 2024 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants