Fix encoding mismatch between python child process and uv #7757
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes #7733. According to CPython documentation on
sys.stdout
, whenstdout
/stderr
is non-character device like pipe, the encoding will be set to system locale on windows. However, on the Rust sidestdout_reader
andstderr_reader
expect them to be encoded in UTF-8 and will fail when child process write non-ASCII character to stdout/stderr, e.g., build directory name containing non-ASCII character.Both CPython3 and PyPy support environment variable
PYTHONIOENCODING
. When it is set toutf-8
, python will use UTF-8 encoding forstdin
/stdout
/stderr
. Sincestdin
is not used by the spawned python process and we expectstdout
/stderr
to use UTF-8, this fix should work as expected.Test Plan
I only tested it on my computer with CPython 3.12 and 3.7. With the fix applied I confirmed that the case I described is fixed.
I'm using Windows 11 with system locale set to code page 936.