Skip to content

Commit

Permalink
Fix encoding mismatch between python child process and uv (#7757)
Browse files Browse the repository at this point in the history
## Summary

This PR fixes #7733. According to [CPython documentation on
`sys.stdout`](https://docs.python.org/3.12/library/sys.html#sys.stdout),
when `stdout`/`stderr` is non-character device like pipe, the encoding
will be set to system locale on windows. However, on the Rust side
`stdout_reader` and `stderr_reader` expect them to be encoded in UTF-8
and will fail when child process write non-ASCII character to
stdout/stderr, e.g., build directory name containing non-ASCII
character.

Both
[CPython3](https://docs.python.org/3.12/using/cmdline.html#envvar-PYTHONIOENCODING)
and [PyPy](https://doc.pypy.org/en/default/man/pypy3.1.html#environment)
support environment variable `PYTHONIOENCODING`. When it is set to
`utf-8`, python will use UTF-8 encoding for `stdin`/`stdout`/`stderr`.
Since `stdin` is not used by the spawned python process and we expect
`stdout`/`stderr` to use UTF-8, this fix should work as expected.
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

I only tested it on my computer with CPython 3.12 and 3.7. With the fix
applied I confirmed that [the case I
described](#7733 (comment))
is fixed.

I'm using Windows 11 with system locale set to code page 936.
  • Loading branch information
kahojyun committed Sep 28, 2024
1 parent 313612d commit 1cae78d
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions crates/uv-build-frontend/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -939,6 +939,7 @@ impl PythonRunner {
.env("PATH", modified_path)
.env("VIRTUAL_ENV", venv.root())
.env("CLICOLOR_FORCE", "1")
.env("PYTHONIOENCODING", "utf-8")
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
Expand Down

0 comments on commit 1cae78d

Please sign in to comment.