feat(sdk): stream logs in wing test

[Chriscbr]

This PR introduces log streaming for the wing test command is used with the simulator. By default, any log statements printed by the user will be automatically streamed while the tests are running. (The first line of each log includes a string identifying which test it's associated with). More granular trace events from the individual resources can be displayed with the --debug flag.

The order of logs can vary and they each include timestamps, so the --no-stream flag can be passed to fall back to the old output style. This is used within the monorepo in places where we're snapshotting CLI output for catching regressions.

log-streaming.mov

Closes #2096

Checklist

• Title matches Winglang's style guide
• Description explains motivation and solution
• Tests added (always)
• Docs updated (only required for features)
• Added pr/e2e-full label if this feature requires end-to-end testing

By submitting this pull request, I confirm that my contribution is made under the terms of the Wing Cloud Contribution License.

[github-actions]

Thanks for opening this pull request! 🎉
Please consult the contributing guidelines for details on how to contribute to this project.
If you need any assistence, don't hesitate to ping the relevant owner over Slack.

Topic	Owner
Wing SDK and utility APIs	@chriscbr
Wing Console	@ainvoner, @skyrpex, @polamoros
JSON, structs, primitives and collections	@hasanaburayyan
Platforms and plugins	@hasanaburayyan
Frontend resources (website, react, etc)	@tsuf239
Language design	@chriscbr
VSCode extension and language server	@markmcculloh
Compiler architecture, inflights, lifting	@yoav-steinberg
Wing Testing Framework	@tsuf239
Wing CLI	@markmcculloh
Build system, dev environment, releases	@markmcculloh
Library Ecosystem	@chriscbr
Documentation	@hasanaburayyan
SDK test suite	@tsuf239
Examples	@hasanaburayyan
Wing Playground	@eladcon

[monadabot]

Console preview environment is available at https://wing-console-pr-6205.fly.dev 🚀

Last Updated (UTC) 2024-04-12 22:23

[MarkMcCulloh]

Stoked for this. We should check that the output looks legible for the following cases

• long/multiline
• color disabled (e.g. NO_COLOR env set)

[monadabot]

Benchmarks

Comparison to Baseline ⬜⬜⬜⬜⬜⬜⬜⬜⬜🟥⬜⬜⬜

Benchmark	Before	After	Change
version	60ms±0.47	60ms±1.15	+0ms (+0.08%)⬜
functions_1.test.w -t sim	399ms±10.34	398ms±5.96	-1ms (-0.15%)⬜
functions_1.test.w -t tf-aws	817ms±3.8	813ms±7.63	-4ms (-0.48%)⬜
hello_world.test.w -t sim	397ms±3.62	401ms±13.18	+4ms (+1.03%)⬜
hello_world.test.w -t tf-aws	1513ms±16.36	1517ms±22.79	+4ms (+0.25%)⬜
jsii_big.test.w -t sim	2779ms±16.12	2783ms±7.81	+4ms (+0.14%)⬜
jsii_big.test.w -t tf-aws	3005ms±19.06	2993ms±18.73	-12ms (-0.4%)⬜
functions_10.test.w -t sim	485ms±12.59	498ms±9.48	+13ms (+2.76%)⬜
functions_10.test.w -t tf-aws	2005ms±9.94	2021ms±17.39	+16ms (+0.79%)⬜
jsii_small.test.w -t sim	362ms±3.06	370ms±3.58	+9ms (+2.44%)🟥
jsii_small.test.w -t tf-aws	601ms±3.46	601ms±4.44	+1ms (+0.09%)⬜
empty.test.w -t sim	359ms±4.06	362ms±3.77	+4ms (+1.1%)⬜
empty.test.w -t tf-aws	588ms±5.25	593ms±5.5	+5ms (+0.92%)⬜

⬜ Within 1.5 standard deviations
🟩 Faster, Above 1.5 standard deviations
🟥 Slower, Above 1.5 standard deviations

Benchmarks may vary outside of normal expectations, especially when running in GitHub Actions CI.

Results

name	mean	min	max	moe	sd
version	60ms	58ms	64ms	1ms	2ms
functions_1.test.w -t sim	398ms	391ms	419ms	6ms	8ms
functions_1.test.w -t tf-aws	813ms	795ms	831ms	8ms	11ms
hello_world.test.w -t sim	401ms	385ms	451ms	13ms	18ms
hello_world.test.w -t tf-aws	1517ms	1480ms	1599ms	23ms	32ms
jsii_big.test.w -t sim	2783ms	2763ms	2806ms	8ms	11ms
jsii_big.test.w -t tf-aws	2993ms	2954ms	3041ms	19ms	26ms
functions_10.test.w -t sim	498ms	479ms	516ms	9ms	13ms
functions_10.test.w -t tf-aws	2021ms	1972ms	2053ms	17ms	24ms
jsii_small.test.w -t sim	370ms	364ms	379ms	4ms	5ms
jsii_small.test.w -t tf-aws	601ms	594ms	610ms	4ms	6ms
empty.test.w -t sim	362ms	354ms	373ms	4ms	5ms
empty.test.w -t tf-aws	593ms	587ms	612ms	6ms	8ms

Last Updated (UTC) 2024-04-12 22:28

[Chriscbr]

Here are some screenshots of the latest revised formatting. The output is "short" when you run wing test without any flags:

and "full" when you run wing test with any debug flags:

[monadabot]

❌ Unstable Self-Mutation ❌

Self-mutation has run twice in a row. There may be a something non-deterministic in the build or test process.
Check the last mutation commit (7c9d968) for suspicious changes.
This is typically caused by:

• Absolute paths
• Timestamps
• Random values
• Flakey tests (relying on one of the above)

[mergify]

Thanks for contributing, @Chriscbr! This PR will now be added to the merge queue, or immediately merged if rybickic/log-streaming is up-to-date with main and the queue is empty.

[monadabot]

Congrats! 🚀 This was released in Wing 0.70.6.