chore: rerun baseline benchmark as part of workflow

[MarkMcCulloh]

Now the benchmark runs twice: Once for the latest version and once for the PR. They run on the same machine in the same job, so there are less variables to worry about.

By submitting this pull request, I confirm that my contribution is made under the terms of the Wing Cloud Contribution License.

[monadabot]

Console preview environment is available at https://wing-console-pr-4373.fly.dev 🚀

Last Updated (UTC) 2023-10-02 16:22

[monadabot]

Benchmarks

Comparison to Baseline ⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜⬜

Benchmark	Before	After	Change
version	86ms±2.77	90ms±3.82	+4ms (+4.22%)⬜
functions_10.test.w -t sim	769ms±9.48	779ms±19.31	+10ms (+1.28%)⬜
functions_10.test.w -t tf-aws	4737ms±32.04	4727ms±38.37	-10ms (-0.21%)⬜
jsii_big.test.w -t sim	4500ms±33.32	4472ms±34.56	-28ms (-0.62%)⬜
jsii_big.test.w -t tf-aws	4619ms±37.45	4620ms±33.19	+1ms (+0.03%)⬜
hello_world.test.w -t sim	679ms±9.07	676ms±7.82	-3ms (-0.51%)⬜
hello_world.test.w -t tf-aws	5624ms±46.6	5608ms±36.97	-16ms (-0.28%)⬜
empty.test.w -t sim	628ms±13.67	625ms±7.61	-4ms (-0.56%)⬜
empty.test.w -t tf-aws	804ms±17.57	796ms±11.95	-8ms (-0.99%)⬜
jsii_small.test.w -t sim	626ms±7.56	620ms±7.96	-5ms (-0.87%)⬜
jsii_small.test.w -t tf-aws	810ms±9.19	789ms±10.2	-21ms (-2.64%)⬜
functions_1.test.w -t sim	722ms±14.87	714ms±16.76	-7ms (-1.03%)⬜
functions_1.test.w -t tf-aws	2530ms±101.89	2522ms±87.15	-8ms (-0.32%)⬜

⬜ Within 1.5 standard deviations
🟩 Faster, Above 1.5 standard deviations
🟥 Slower, Above 1.5 standard deviations

Benchmarks may vary outside of normal expectations, especially when running in GitHub Actions CI.

Results

name	mean	min	max	moe	sd
version	90ms	87ms	102ms	4ms	5ms
functions_10.test.w -t sim	779ms	758ms	842ms	19ms	27ms
functions_10.test.w -t tf-aws	4727ms	4626ms	4815ms	38ms	54ms
jsii_big.test.w -t sim	4472ms	4431ms	4571ms	35ms	48ms
jsii_big.test.w -t tf-aws	4620ms	4508ms	4681ms	33ms	46ms
hello_world.test.w -t sim	676ms	662ms	691ms	8ms	11ms
hello_world.test.w -t tf-aws	5608ms	5522ms	5697ms	37ms	52ms
empty.test.w -t sim	625ms	613ms	635ms	8ms	11ms
empty.test.w -t tf-aws	796ms	779ms	824ms	12ms	17ms
jsii_small.test.w -t sim	620ms	610ms	636ms	8ms	11ms
jsii_small.test.w -t tf-aws	789ms	777ms	813ms	10ms	14ms
functions_1.test.w -t sim	714ms	688ms	774ms	17ms	23ms
functions_1.test.w -t tf-aws	2522ms	2343ms	2820ms	87ms	122ms

Last Updated (UTC) 2023-10-02 16:32

[staycoolcall911]

This is great @MarkMcCulloh! Elegant and smart change, making benchmarks very useful

[mergify]

Thanks for contributing, @MarkMcCulloh! This PR will now be added to the merge queue, or immediately merged if mark/baseline-bench is up-to-date with main and the queue is empty.

[monadabot]

Congrats! 🚀 This was released in Wing 0.34.10.