No install under Python 3.9

[pikeas]

I recently added a Python 3.9 to my system and tried to update my pipx-in-pipx to match. However, nothing seems to happen. I've searched throughout the Python installation and there are no files relating to pipx.

The install completes successfully:

$ pip -V
pip 20.3.3 from /Users/me/.asdf/installs/python/3.9.1/lib/python3.9/site-packages/pip (python 3.9)
$ pip install pipx-in-pipx
Collecting pipx-in-pipx
  Using cached pipx-in-pipx-1.0.1.tar.gz (11 kB)
Using legacy 'setup.py install' for pipx-in-pipx, since package 'wheel' is not installed.
Installing collected packages: pipx-in-pipx
    Running setup.py install for pipx-in-pipx ... done
Successfully installed pipx-in-pipx

There's no bootstrapping output, is it possible that the pipx-in-pipx setup entrypoint isn't being run?

[pikeas]

After pip install wheel, the install proceeds further but is unable to complete:

~ ❯ pip3.9 install pipx-in-pipx
Collecting pipx-in-pipx
  Using cached pipx-in-pipx-1.0.1.tar.gz (11 kB)
Building wheels for collected packages: pipx-in-pipx
  Building wheel for pipx-in-pipx (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/me/.asdf/installs/python/3.9.1/bin/python3.9 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/85/39v1lc455z77f146j3lzm1jr0000gn/T/pip-install-h59_9upk/pipx-in-pipx_110322f4ee524f69a0e59b7429311c75/setup.py'"'"'; __file__='"'"'/private/var/folders/85/39v1lc455z77f146j3lzm1jr0000gn/T/pip-install-h59_9upk/pipx-in-pipx_110322f4ee524f69a0e59b7429311c75/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/85/39v1lc455z77f146j3lzm1jr0000gn/T/pip-wheel-sy40n7rb
       cwd: /private/var/folders/85/39v1lc455z77f146j3lzm1jr0000gn/T/pip-install-h59_9upk/pipx-in-pipx_110322f4ee524f69a0e59b7429311c75/
  Complete output (57 lines):
  running bdist_wheel
  running build
  installing to build/bdist.macosx-10.15-x86_64/wheel
  running install
  ################# Building bootstrapping venv ##################
  Collecting pip
    Using cached pip-20.3.3-py2.py3-none-any.whl (1.5 MB)
  Installing collected packages: pip
    Attempting uninstall: pip
      Found existing installation: pip 20.2.3
      Uninstalling pip-20.2.3:
        Successfully uninstalled pip-20.2.3
  Successfully installed pip-20.3.3

  Collecting pipx
    Using cached pipx-0.15.6.0-py3-none-any.whl (43 kB)
  Collecting userpath
    Using cached userpath-1.4.1-py2.py3-none-any.whl (14 kB)
  Collecting argcomplete<2.0,>=1.9.4
    Using cached argcomplete-1.12.2-py2.py3-none-any.whl (38 kB)
  Collecting packaging>=20.0
    Using cached packaging-20.8-py2.py3-none-any.whl (39 kB)
  Collecting pyparsing>=2.0.2
    Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
  Collecting click
    Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
  Installing collected packages: pyparsing, click, userpath, packaging, argcomplete, pipx
  Successfully installed argcomplete-1.12.2 click-7.1.2 packaging-20.8 pipx-0.15.6.0 pyparsing-2.4.7 userpath-1.4.1

  ######## Installing local pipx using bootstrapping pipx ########
  creating virtual environment...
  installing pipx...


  /Users/me/.local/pipx/venvs/pipx/bin/python: No module named pip
  Error installing pipx.

  ############ Verifying that PATH includes local bin ############
  The directory `/Users/me/.local/bin` is already in PATH! If you are sure you want to proceed, try again with the -f/--force flag.

  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/85/39v1lc455z77f146j3lzm1jr0000gn/T/pip-install-h59_9upk/pipx-in-pipx_110322f4ee524f69a0e59b7429311c75/setup.py", line 108, in <module>
      setup(name=name, **SETUP_KWARGS)
    File "/Users/me/.asdf/installs/python/3.9.1/lib/python3.9/site-packages/setuptools/__init__.py", line 165, in setup
      return distutils.core.setup(**attrs)
    File "/Users/me/.asdf/installs/python/3.9.1/lib/python3.9/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/Users/me/.asdf/installs/python/3.9.1/lib/python3.9/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/Users/me/.asdf/installs/python/3.9.1/lib/python3.9/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/Users/me/.asdf/installs/python/3.9.1/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 351, in run
      self.egg2dist(self.egginfo_dir, distinfo_dir)
    File "/Users/me/.asdf/installs/python/3.9.1/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 461, in egg2dist
      raise ValueError(err)
  ValueError: Egg metadata expected at build/bdist.macosx-10.15-x86_64/wheel/pipx_in_pipx-1.0.1-py3.9.egg-info but not found
  ----------------------------------------
  ERROR: Failed building wheel for pipx-in-pipx
  Running setup.py clean for pipx-in-pipx
Failed to build pipx-in-pipx
Installing collected packages: pipx-in-pipx
    Running setup.py install for pipx-in-pipx ... done
Successfully installed pipx-in-pipx

[rpdelaney]

On my system pip3 --isolated install --user pipx_in_pipx gives the second output. pipx is not in the path.

This part looks interesting:

/usr/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-43rf94ig/pipx-in-pipx/setup.py'"'"'; __file__='"'"'/tmp/pip-install-4
3rf94ig/pipx-in-pipx/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file
__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-73hid9qh

Here is that code formatted as a python 3 script:

import sys, setuptools, tokenize

sys.argv[0] = """tmp/pip-install-43rf94ig/pipx-in-pipx/setup.py"""
__file__ = """/tmp/pip-install-4 3rf94ig/pipx-in-pipx/setup.py"""
f = getattr(tokenize, """open""", open)(__file__)
code = f.read().replace("""\r\n""", """\n""")
f.close()
exec(compile(code, __file__, """exec"""))

It seems as if the install script cannot find the pip module?

[rpdelaney]

This may be a problem with pipx:

$ pip3 --isolated install --user pipx
...
$ pipx install --force youtube-dl
Installing to existing directory '/home/ryan/.local/pipx/venvs/youtube-dl'
/home/ryan/.local/pipx/shared/bin/python: No module named pip
Failed to upgrade shared libraries
Traceback (most recent call last):
  File "/home/ryan/.local/lib/python3.9/site-packages/pipx/shared_libs.py", line 73, in upgrade
    run(
  File "/home/ryan/.local/lib/python3.9/site-packages/pipx/util.py", line 135, in run
    raise PipxError(f"{cmd_str!r} failed")
pipx.util.PipxError: '/home/ryan/.local/pipx/shared/bin/python -m pip --disable-pip-version-check install -q --upgrade pip setuptools wheel' failed
/home/ryan/.local/pipx/venvs/youtube-dl/bin/python: No module named pip

Not removing existing venv /home/ryan/.local/pipx/venvs/youtube-dl because it was not created in this session
Error installing youtube-dl.
$

[rpdelaney]

The README has the fix. I'm surprised I stepped on this rake again:

Because all pipx-managed packages use the Python in the pipx virtualenv, you can change the Python for all packages by installing pipx-in-pipx again.

[mattsb42]

Sorry folks, I've been kinda out of it the last couple months.

@pikeas the interesting bit to me in the error output is:

/Users/me/.local/pipx/venvs/pipx/bin/python: No module named pip

That is odd to me, as it should be doing effectively the same thing as the bootstrapping step, which worked successfully. Did you already have a pipx managed pipx installed when you ran this? It should have just overridden the existing install if so, but that seems like the most likely issue.

[pikeas]

@mattsb42 Thanks for replying! Here's a more complete run-through that throws the same error:

$ rm -rf ~/.local/pipx/venvs/pipx

$ ~/.asdf/installs/python/3.9.1/bin/pip install pipx-in-pipx
Collecting pipx-in-pipx
  Using cached pipx-in-pipx-1.0.1.tar.gz (11 kB)
Using legacy 'setup.py install' for pipx-in-pipx, since package 'wheel' is not installed.
Installing collected packages: pipx-in-pipx
    Running setup.py install for pipx-in-pipx ... done
Successfully installed pipx-in-pipx

$ ls -l ~/.local/pipx/venvs/pipx
lsd: /Users/me/.local/pipx/venvs/pipx: No such file or directory (os error 2)

$ ~/.asdf/installs/python/3.9.1/bin/pip install wheel
Collecting wheel
  Using cached wheel-0.36.2-py2.py3-none-any.whl (35 kB)
Installing collected packages: wheel
Successfully installed wheel-0.36.2

$ ~/.asdf/installs/python/3.9.1/bin/pip install pipx-in-pipx
[...]
  Building wheel for pipx-in-pipx (setup.py) ... error
  ERROR: Command errored out with exit status 1:
[...]
  ######## Installing local pipx using bootstrapping pipx ########
  creating virtual environment...
  installing pipx...

  /Users/me/.local/pipx/venvs/pipx/bin/python: No module named pip
  Error installing pipx.
[...]
  ValueError: Egg metadata expected at build/bdist.macosx-10.15-x86_64/wheel/pipx_in_pipx-1.0.1-py3.9.egg-info but not found
  ----------------------------------------
  ERROR: Failed building wheel for pipx-in-pipx
  Running setup.py clean for pipx-in-pipx
Failed to build pipx-in-pipx
Installing collected packages: pipx-in-pipx
    Running setup.py install for pipx-in-pipx ... done
Successfully installed pipx-in-pipx

$ ls -l ~/.local/pipx/venvs/pipx
lsd: /Users/me/.local/pipx/venvs/pipx: No such file or directory (os error 2)

[pikeas]

For now, I'm using this workaround:

cd ~/.local/pipx/venvs/
python -m venv pipx
./pipx/bin/pip install pipx

This pipx is mostly functional but is unable to self-manage, it will break when reinstalling itself.

[rpdelaney]

I wound up listing the contents of ~/.local/pipx/venvs into a file, nuking pipx and reinstalling everything:

while read -r pkg ; do pipx install "$pkg" || break ; done < pipx_things

Not exactly the most comfortable upgrade path, but not the most painful either.

[mattsb42]

This pipx is mostly functional but is unable to self-manage, it will break when reinstalling itself.

This is expected of pipx-managed-pipx installs, as is noted in the readme. "Reinstall" just means "uninstall then install". In the case of uninstalling pipx, there is now no pipx to use for the subsequent install.

If you want to reinstall to update references to a new Python install, see the comments in the readme[1]. If you want to reinstall for some other reason, I'd be interested to understand what that reason is.

[1] https://github.com/mattsb42-meta/pipx-in-pipx#which-python