-
-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
adding package-benchmark sub-project #64
Conversation
An example benchmark run (on my laptop - an M1 macbook pro) of this update: Baseline 'Current_run'
ECSBenchmarkTraitMatching
TypedFamilyEntities
TypedFamilyEntityFiveComponents
TypedFamilyEntityFourComponents
TypedFamilyEntityOneComponent
TypedFamilyEntityThreeComponents
TypedFamilyEntityTwoComponents
TypedFamilyFiveComponents
TypedFamilyFourComponents
TypedFamilyOneComponent
TypedFamilyThreeComponents
TypedFamilyTwoComponents
|
Thanks for the contribution @heckj 👍 LGTM How about we extend the CI pipeline with a benchmark step that's either run pre-release and/or when triggered manually or even on every push (at least on the free machines)? |
4fb8c06
to
39820d7
Compare
We absolutely can tweak up the CI to do runs, make comparisons, etc. package-benchmark is great for that. The Linux (and macOS, if desired) CI will need jemalloc installed to be able to get the memory allocations information, as its not available stock - but more importantly if you want to compare on CI we'll want to leverage only the metrics that are "what CPU did you give me" agnostic, which is a more limited set (and not at easily understood in regressions from what I've seen) - macOS (Darwin) has some interesting "instructions executed" markers that can be pulled, the allocations are a good choice, etc. Otherwise you'll be getting some notably wide variances, especially with free resources in Github Resources, and even with the paid versions. |
Ok, so let's skip extending CI for that for now. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #64 +/- ##
=======================================
Coverage 96.91% 96.91%
=======================================
Files 24 24
Lines 1069 1069
=======================================
Hits 1036 1036
Misses 33 33 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please run make precommit
to let the linter and formatter do it's job
39820d7
to
186860c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we also should exclude the Benchmarks/
from the code coverage report.
Please just add that to .codecov.yml
Head branch was pushed to by a user without write access
Description
adds a package within the project to capture a set up benchmarks based on what's in FirebladeECSPerformanceTests/TypedFamilyPerformanceTests.swift, converted to leverage package-benchmark.
There's no functional change or external API change with this update, it's more of a prep piece to allow me to more easily compare future changes.
Testing
To use this new sub project:
swift package benchmark
Further details are in the README within the sub project.
Checklist