Add -category argument to TestRunner #5358
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Looks like in this test, turning off the release build performance tests took 700 seconds vs 1300 seconds. I'm not sure whether that means it's worth it to turn them off |
|
I guess it depends on what we're trying to optimise - time to first green check, or time to all green checks. If it's all green checks then we might as well keep running the performance tests on release builds, because they'll still be finishing faster than the linux debug build (although we might want to drop them from windows?). If it's time to first green check then it might be worth dropping. I think I'd be inclined to keep, at least on Linux. |
danieldresser-ie
force-pushed
the
testCategory
branch
from
4dd9fdf
to
7d5c3a0
Compare
johnhaddon
added a commit
to johnhaddon/gaffer
that referenced
this pull request
Aug 17, 2023
This builds on the work done in GafferHQ#5358, allowing tests to be assigned to arbitrary categories using a new `CategorisedTestMethod` decorator. It also offers improved filtering by category in `gaffer test` by supporting wildcards and adding a `-excludedCategories` argument. This will allow us to use test categories to group tests that are related in some way and are commonly run together during development.
johnhaddon
added a commit
to johnhaddon/gaffer
that referenced
this pull request
Aug 17, 2023
This builds on the work done in GafferHQ#5358, allowing tests to be assigned to arbitrary categories using a new `CategorisedTestMethod` decorator. It also offers improved filtering by category in `gaffer test` by supporting wildcards and adding a `-excludedCategories` argument. This will allow us to use test categories to group tests that are related in some way and are commonly run together during development.
johnhaddon
added a commit
to johnhaddon/gaffer
that referenced
this pull request
Aug 18, 2023
This builds on the work done in GafferHQ#5358, allowing tests to be assigned to arbitrary categories using a new `CategorisedTestMethod` decorator. It also offers improved filtering by category in `gaffer test` by supporting wildcards and adding a `-excludedCategories` argument. This will allow us to use test categories to group tests that are related in some way and are commonly run together during development.
johnhaddon
added a commit
to johnhaddon/gaffer
that referenced
this pull request
Aug 18, 2023
This builds on the work done in GafferHQ#5358, allowing tests to be assigned to arbitrary categories using a new `CategorisedTestMethod` decorator. It also offers improved filtering by category in `gaffer test` by supporting wildcards and adding a `-excludedCategories` argument. This will allow us to use test categories to group tests that are related in some way and are commonly run together during development.
johnhaddon
added a commit
to johnhaddon/gaffer
that referenced
this pull request
Aug 22, 2023
This builds on the work done in GafferHQ#5358, allowing tests to be assigned to arbitrary categories using a new `CategorisedTestMethod` decorator. It also offers improved filtering by category in `gaffer test` by supporting wildcards and adding a `-excludedCategories` argument. This will allow us to use test categories to group tests that are related in some way and are commonly run together during development.
johnhaddon
added a commit
to johnhaddon/gaffer
that referenced
this pull request
Aug 23, 2023
This builds on the work done in GafferHQ#5358, allowing tests to be assigned to arbitrary categories using a new `CategorisedTestMethod` decorator. It also offers improved filtering by category in `gaffer test` by supporting wildcards and adding a `-excludedCategories` argument. This will allow us to use test categories to group tests that are related in some way and are commonly run together during development.
johnhaddon
added a commit
to johnhaddon/gaffer
that referenced
this pull request
Aug 23, 2023
This builds on the work done in GafferHQ#5358, allowing tests to be assigned to arbitrary categories using a new `CategorisedTestMethod` decorator. It also offers improved filtering by category in `gaffer test` by supporting wildcards and adding a `-excludedCategories` argument. This will allow us to use test categories to group tests that are related in some way and are commonly run together during development.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As discussed, replace -performanceOnly with a more flexible argument.
Since we haven't decided yet whether to set release builds to
-category standard, I tried setting up a temp build with it set for comparison purposes ... we'll see if I botched that syntax.