Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bazel] Add PYTHONUSERBASE to list of non-hermetic env vars #18689

Closed
wants to merge 1 commit into from

Conversation

jwnrt
Copy link
Contributor

@jwnrt jwnrt commented May 19, 2023

We use python modules from outside of Bazel's "hermetic seal".

Without allowing this variable through we always assume they're in the default location.

For environments which set PYTHONUSERBASE, this leads to confusing behaviour where modules are installed and work outside of Bazel, but not inside.

@jwnrt jwnrt requested a review from cfrantz as a code owner May 19, 2023 10:20
@jwnrt jwnrt requested review from pamaury and HU90m May 19, 2023 10:20
@jwnrt
Copy link
Contributor Author

jwnrt commented May 19, 2023

@HU90m am I misremembering that you were working on bringing the python modules into the Bazel environment? If that work's ongoing, it can replace this.

Copy link
Member

@HU90m HU90m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha LGTM

Does it need a comment? I guess it's pretty self explanatory so people can look it up.

@HU90m
Copy link
Member

HU90m commented May 19, 2023

@HU90m am I misremembering that you were working on bringing the python modules into the Bazel environment? If that work's ongoing, it can replace this.

I was considering trying to make it a little more hermetic, but there are bigger fish to fry so I doubt I'll ever get round to it.

@jwnrt
Copy link
Contributor Author

jwnrt commented May 19, 2023

Does it need a comment?

Sure, added a comment

@jwnrt

This comment was marked as resolved.

@jwnrt

This comment was marked as resolved.

We use python modules from outside of Bazel's "hermetic seal". Without
allowing this variable through we always assume they're in the default
location.

Signed-off-by: James Wainwright <[email protected]>
@jwnrt
Copy link
Contributor Author

jwnrt commented May 22, 2023

CI failures were an upstream problem. Rebased on the fix and it's passing now.

@a-will
Copy link
Contributor

a-will commented Jun 29, 2023

The "real fix" would probably be to eliminate system Python use in the build. I'm not sure how big of a problem that is, though. Some work needs to be done with our fusesoc generators so they use the bazel-provided Python, for example.

Edit: ...I tried just passing the interpreter around by modifying the PATH in a shim py_binary for fusesoc, and while that handled calling the correct interpreter for the generators, that Python environment didn't come with many of the modules available from @ot_python_deps 😅 . primgen would immediately fail to load mako as a result.

Stuffing in all_requirements into that py_binary shim's deps then did the trick. Considering this represents much more than fusesoc's direct dependencies, it still doesn't feel quite right, but this mechanism is probably closer to where we'd want to go.

@jwnrt
Copy link
Contributor Author

jwnrt commented Aug 31, 2023

@a-will do you have any objection to landing this as a stop gap between having python managed by Bazel?

@a-will
Copy link
Contributor

a-will commented Sep 14, 2023

@jwnrt The work for fusesoc was done in #19476. Are there any remaining uses of Python in bazel where PYTHONUSERBASE would be needed?

@jwnrt
Copy link
Contributor Author

jwnrt commented Nov 10, 2023

Nope, I've been going fine for a while without this env var propagated. Thanks @a-will.

@jwnrt jwnrt closed this Nov 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants