Feature: Pytest Slow Mark

[JakobEliasWagner]

Feature: Pytest Slow Mark

Description

The current test suite does not differentiate between standard tests and slow tests. As a result, all tests, irrespective of their execution time, are treated equally and run in every test cycle. This leads to increased execution time, especially in local development environments. This issue aims to introduce a classification system for our tests, specifically by marking slow tests so they can be selectively run on the CI pipeline and at predetermined times, thereby optimizing the development workflow and CI pipeline efficiency.

Tests are categorized into two categories:

• Milliseconds to 1 second: Tests running in this time-frame are quick enough to allow for rapid iteration and feedback during development. They can be run frequently without significantly impacting productivity. They do not need to be marked.
• 1 second to 5 seconds: Tests running in this time-frame might be acceptable in some cases. However, test in this range should be scrutinized to ensure they cannot be optimized or refactored for speed. They should be marked as slow using the @pytest.mark.slow wrapper.
• Longer than 5 seconds: Any unit test taking longer than 5 seconds should be classified as an integration test. These tests should be marked as slow with the @pytest.mark.slow wrapper.

Now running tests with the flag -m "not slow" invokes only tests which have not been marked as slow.

Which issue does this PR tackle?

• Currently all tests are run always, which leads to slow tests and slow iteration circle.
• Fixes Optimize Test Suite by Introducing Test Classification with Marks for Slow Tests #36

How does it solve the problem?

• Implements Slow Mark.
• Marks current slow tests as slow.
  • Test Optuna.
  • Convergence test on DeepONet.
  • Covergence test on NerualOperator.

Checklist for Contributors

• Scope: This PR tackles exactly one problem.
• Conventions: The branch follows the feature/title-slug convention.
• Conventions: The PR title follows the Bugfix: Title convention.
• Coding style: The code passes all pre-commit hooks.
• Documentation: All changes are well-documented.
• Tests: New features are tested and all tests pass successfully.
• Changelog: Updated CHANGELOG.md for new features or breaking changes.
• Review: A suitable reviewer has been assigned.

Checklist for Reviewers:

• The PR solves the issue it claims to solve and only this one.
• Changes are tested sufficiently and all tests pass.
• Documentation is complete and well-written.
• Changelog has been updated, if necessary.

[JakobEliasWagner]

@samuelburbulla Is it necessary to add a section to the documentation with this? Currently this feature can be discovered by looking at the pyproject.toml.