Feature: BeforeFeature / AfterFeature hooks

[vitalets]

The problem
Sometimes we need to run code before / after particular feature file. Currently it can be achieved in playwright-bdd only with some hacks.
In CucumberJS it's not possible - there were several related requests:

• Before/After feature hooks and hacks cucumber/cucumber-js#914
• Full tagging support cucumber/cucumber-js#1284
• BeforeAll executes without considering the tags provided cucumber/cucumber-js#1447

CucumberJS FAQ states that these hooks are not possible, because runner does not consider features when running scenarios.

A solution
Playwright provides a direct way to run code before particular file - test.beforeAll / test.afterAll. We can just utilize that.
SpecFlow also provides BeforeFeature / AfterFeature hooks.

Binding to features can be done by tags - the same as for scenario hooks. Example code:

const { BeforeFeature, AfterFeature } = createBdd();

BeforeFeature({ tags: '@auth' }, async () => { 
  // run setup before features with @auth tag 
});

This also matches the proposed tags from path approach.
You can put feature files and BeforeFeature hooks into @-prefixed directory, and they will be bound automatically:

features
└── @auth
    ├── signin.feature
    └── hooks.ts

Hooks naming
Hooks naming between Cucumber and Playwright is a bit confusing:

• Playwright's BeforeEach / AfterEach = Cucumber's Before / After - run before/after each scenario
• Playwright's BeforeAll / AfterAll = (no alternative in CucumberJS) - run before/after each feature
• Playwright's worker-scoped auto-fixtures = Cucumber's BeforeAll / AfterAll - run before/after each worker

I like the explicit naming in SpecFlow:

• BeforeScenario / AfterScenario
• BeforeFeature / AfterFeature
• BeforeWorker / AfterWorker

I think, we can use these aliases in playwright-bdd.

Running once
There is a request from community to run beforeFeature hooks only once, even in fullyParallel mode / in case when some tests are failing and new worker is created.
Keep in mind during implementation, see microsoft/playwright#22520 (comment)

I you have any thoughts / concerns here, feel free to share in comments.
Or just add your 👍

[vitalets]

What should be a default behavior of BeforeFeature / AfterFeature hooks?

1. Run once in each worker: this is how Playwright's test.beforeAll / test.afterAll currently behave. If some test fails, new worker is created and hooks re-run again.
2. Run once in a whole test run: this is not how Playwright's test.beforeAll / test.afterAll behave, but according to playwright#22520 it is often requested (to setup some database stuff, etc).

What do you think?

[alescinskis]

I vote for 1st option

[vitalets]

What should be a default behavior of BeforeFeature / AfterFeature hooks?

I thought more about that question. If BeforeFeature runs in each worker, it's actually not BeforeFeature but BeforeWorker. A confusion also goes from naming: while feature is related to BDD level, worker is related to runner. Mixing these entities in hooks naming does not make it clear.

As of now, I'm thinking about the following changes:

1. Alias existing Before / After hooks as BeforeScenario / AfterScenario and recommend using the latter, as they are more explicit
2. Alias existing BeforeAll / AfterAll as BeforeWorker / AfterWorker and recommend using the latter, as they are more explicit
3. Allow to pass tags option to BeforeWorker / AfterWorker. If tags are passed, the hook runs only if matched features are being executed (so actually behaves like BeforeFeature / AfterFeature). If a new worker is created for matched feature, the hook runs again. If tags are not passed, the hook runs before / after each worker (current behavior).
4. Allow to pass once option to BeforeWorker / AfterWorker. If once: true is passed, the hook runs only once during the whole test run session (so actually behave like Playwright's global setup / teardown). If additionally filtered by tags, the hook runs once and only if matched features are being executed - this is the main benefit for setting up some data once for the particular feature.
5. No BeforeFeature / AfterFeature hooks

Example code:

const { BeforeWorker } = createBdd();

BeforeWorker({ tags: '@auth', once: true }, async () => { 
  // heavy setup of test user in database for checking all auth stuff
});

Feature file can contain @auth tag on the top level:

@auth
Feature: check auth

  Scenario: check login

Or on the scenario level:

Feature: check auth

  @auth
  Scenario: check login

If at least one scenario of feature has the matching tag, BeforeWorker hook will run before that feature.