Editable install does not work

[matthiasdiener]

Describe the bug
When using an editable install, pyvisfile fails to load:

$ python -c 'import pyvisfile'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1138, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1078, in _find_spec
  File "/Users/mdiener/Work/e4/miniforge3/envs/ceesd/lib/python3.11/site-packages/_pyvisfile_editable_loader.py", line 310, in find_spec
    tree = self._rebuild()
           ^^^^^^^^^^^^^^^
  File "/Users/mdiener/Work/e4/miniforge3/envs/ceesd/lib/python3.11/site-packages/_pyvisfile_editable_loader.py", line 343, in _rebuild
    subprocess.run(self._build_cmd, cwd=self._build_path, env=env, stdout=subprocess.DEVNULL)
  File "/Users/mdiener/Work/e4/miniforge3/envs/ceesd/lib/python3.11/subprocess.py", line 548, in run
    with Popen(*popenargs, **kwargs) as process:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mdiener/Work/e4/miniforge3/envs/ceesd/lib/python3.11/subprocess.py", line 1026, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/Users/mdiener/Work/e4/miniforge3/envs/ceesd/lib/python3.11/subprocess.py", line 1953, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/2f/56g48t855fq44c1p370wvfn40000gn/T/pip-build-env-l1574hnj/overlay/bin/ninja'

To Reproduce
Steps to reproduce the behavior:
Consider this script:

$ git clone <pyvisfile>
$ cd pyvisfile
$ pip install -e . -v
$ python -c 'import pyvisfile'

I would expect this to:

• not fail

Instead, it:

• failed

Expected behavior
A clear and concise description of what you expected to happen.

Environment (please complete the following information):

• OS: MacOS
• PyVisfile version: git
• Python version: 3.11

Additional context
Add any other context about the problem here.

cc illinois-ceesd/mirgecom#1044

[alexfikl]

Just doing a pip install --editable . will do the install in a local environment that gets cleaned up when it's done. You need to install the build dependencies and use --no-build-isolation. This works just fine for me in a new venv

python -m pip install --upgrade meson-python ninja
python -m pip install --no-build-isolation --editable .
python -c 'import pyvisfile'

I think scikit-build-core works in the same way. For more info, see
https://meson-python.readthedocs.io/en/latest/how-to-guides/editable-installs.html

[matthiasdiener]

I think scikit-build-core works in the same way.

Kind of. By default, editable rebuilds are off in scikit-build-core, such that editable installs work without --no-build-isolation.

Edit:
There seems to be no way to disable editable rebuilds in meson-python.

[alexfikl]

Edit: There seems to be no way to disable editable rebuilds in meson-python.

Hm, yeah, they seem to always call ninja to do a build. From a quick browse through their issues, it doesn't seem like they're interested in offering a way to disable that besides just setting --no-build-isolation..

[alexfikl]

@matthiasdiener Another little workaround for this is to set the NINJA env variable. Without build isolation, meson-python uses the ninja installed in the local environment created by pip, i.e.

/private/var/folders/2f/56g48t855fq44c1p370wvfn40000gn/T/pip-build-env-l1574hnj/overlay/bin/ninja

This works for me from your original post:

export NINJA=/usr/bin/ninja
python -m pip install --editable .
python -c "import pyvisfile"

You still need to have ninja installed globally (or in the venv somewhere and point at that) + the rebuilds are still happening at import time.. but it's not crashing 😁