Several issues when installing packages via pip

[rnixx]

Versions

• Python: all
• OS: all
• Kivy: 1.10.1
• Cython: 0.28.2

Description

When defining requirements for p4a which get installed via pip, there are several issues:

• If requirements list gets modified and p4a re-runs, packages containing namespace packages (https://packaging.python.org/guides/packaging-namespace-packages/) are not installed properly (although they are if deleting .buildozer folder and build from scratch).

• Package install dependencies are ignored. They get installed in the local python, but are never synced to .buildozer/android/platform/build/build/python-installs/APPNAME/ unless defined explicitely. Some kind of pip freeze call before syncing or similar might be needed in order to collect dependencies.

• Package sync to .buildozer/android/platform/build/build/python-installs/APPNAME/ should happen at each p4a run to ensure sane python package structure if anything changes.

• zope and zope_interface recipes are obsolete. zope.interface installation works properly via pip (i think since @opacam refactoring). The fact that namespace packages contain no __init__.py is perfectly fine, the relevant information lives in related *-nspkg.pth files (e.g. zope.interface-4.6.0-py3.5-nspkg.pth) which get copied properly.

[ghost]

Package install dependencies are ignored. They get installed in the local python, but are never synced to .buildozer/android/platform/build/build/python-installs/APPNAME/ unless defined explicitely. Some kind of pip freeze call before syncing or similar might be needed in order to collect dependencies.

Hopefully will be solved if you use a setup.py and we get around to merging this: #1625 (in which case you simply don't use --requirements anymore. I don't think it makes too much sense to fix the option itself, since the setup.py way is more standard anyway and achieves the same result with the same required complexity for the dependency scan)

If requirements list gets modified and p4a re-runs, packages containing namespace packages (https://packaging.python.org/guides/packaging-namespace-packages/) are not installed properly (although they are if deleting .buildozer folder and build from scratch).

Can you name a specific package to make it easier for someone of us to look into this if we get the time? Is it a recipe package or an external/pip one?

[rnixx]

since the setup.py way is more standard anyway and achieves the same result with the same required complexity for the dependency scan

i have not taken a look into #1625, but with just guessing i fear this does not resolve the syncing issue (i may be wrong)

Can you name a specific package

e.g. node.ext.directory. Expected dependencies are:

• node
• odict
• plumber
• zope.component
• zope.deferredimport
• zope.deprecation
• zope.event
• zope.hookable
• zope.interface
• zope.lifecycleevent
• zope.proxy

[ghost]

@rnixx what are the steps to reproduce? build anything with --requirements node.ext.directory and run the build twice, then check in the site packages if they're around? Maybe a separate ticket for this particular issue with reproduction steps would be useful, because that sounds like an interesting corner case that may require a bit of debugging on our side to fix

[rnixx]

• run with 'node' as requirements

  • you will see that no dependencies are synced to .buildozer/android/platform/build/build/python-installs/APPNAME/
  • the dependencies are defined in node/setup.py (but not all dependencies, subsequent dependencies are defined in the respective dependency packages setup.py files

• run again with 'node.ext.directory' as requirements

  • you will see that 'node' directory will not contain 'ext/directory' as expected.
  • the dependency related nspkg.pth file should be synced properly
  • consider re-syncing the whole python-installs every time?

• removing dependencies from requirements not removes them from .buildozer/android/platform/build/build/python-installs/APPNAME/

  • consider re-syncing the whole python-installs every time?

[ghost]

run again with 'node.ext.directory' as requirements

Did you specify --requirements=node.ext.directory only? Without also specifying node itself, is that supposed to be working correctly? You really need to specify all indirect dependencies at this point in time, if you don't and things break that really is expected. Did you specify them all?

That old packages are left around is indeed a little unfortunate, and should be fixable

[rnixx]

• As described, if i define the entire dependency list and build from scratch, everything works.
• Subsequent installation of namespace packages fails (e.g. adding node.ext.directory after node)
• And as pointed out setup dependencies fails (but are installed correctly in build python before syncing). And in my understanding, installing packages via pip should behave like that, means setup dependencies are resolved properly

[Julian-O]

Because this contains multiple issues that have multiple solutions, this issue is going to be impossible to close - we will never know if it is all fixed.

@rnixx: Can you please raise one issue for each remaining problem?

[github-actions]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have the means to take action. Please reach out if you have or find the answers we need so that we can investigate further.