Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tox / Docker / Dependency updates #406

Merged
merged 6 commits into from
Sep 19, 2023

Conversation

nkemnitz
Copy link
Contributor

@nkemnitz nkemnitz commented Jun 29, 2022

Updating PyPI requirements.txt:

  • remove superfluous dependencies:
    • urllib3
    • certifi
    • jsonschema
    • importlib-metadata
  • document known dependency pins:
    • protobuf~=3.20
  • remove likely outdated dependency pins - if still necessary, should document:
    • cloud-volume==6.1.1
    • task-queue==1.0.0
    • dracopy<=0.0.19
    • google-cloud-datastore>=1.8<=2.0dev
    • network==2.1
  • add missing dependencies:
    • uwsgi
    • werkzeug

tox:

  • switch to py3.11
  • graph-tool now available as conda-forge package
  • add tox-conda plugin + requirements.yml
  • remove hacks previously added to link the tox environment to system dist-packages
  • still requires installing conda and cbtemulator :/

Multi-stage Dockerfile:

  • update .dockerignore to match .gitignore (+ .git/)
  • build image for PyChunkedGraph dependencies in separate environment (including graph-tool)
  • build image for Bigtable Emulator (gets rid off 1 GB Google Cloud SDK)
  • adding shim for Bigtable Emulator call and pointing uwsgi to new Python environment

Fixes:

  • When PCG internal checks already pre-filter columns, column_filter can end up as empty list. Bigtable however interprets this as "no filters applied". This PR changes behavior to raise an exception if an ambiguous, empty sequence is passed as column_filter, and fixes one such case captured by existing tests.

@nkemnitz nkemnitz marked this pull request as ready for review July 21, 2022 08:19
@nkemnitz nkemnitz requested a review from sdorkenw July 21, 2022 08:26
Dockerfile Show resolved Hide resolved
@fcollman
Copy link
Collaborator

@nkemnitz can i tag this version for a test deployment?

@fcollman fcollman changed the base branch from main to tox-update September 19, 2023 14:36
@fcollman fcollman merged commit 1bd3713 into CAVEconnectome:tox-update Sep 19, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants