Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update builtin collections imports to be forward-compatible with Python3.10+ #253

Merged
merged 25 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
6772632
Update workflow.py to be python3.10+ compatible
AndroxxTraxxon May 2, 2023
c030c33
Update base.py to be python3.10+ compatible
AndroxxTraxxon May 2, 2023
524eff2
Update base.py to be compatible back to 3.6
AndroxxTraxxon May 2, 2023
0d75e0b
Update workflow.py to also be backwards compatible to python 3.6
AndroxxTraxxon May 2, 2023
5fda72b
Update tox.yml
AndroxxTraxxon May 2, 2023
f105c00
Update tox.yml for old Python compatibility
AndroxxTraxxon May 2, 2023
79eb36c
Update tox.yml with more comprehensive test matrix to fit dependency …
AndroxxTraxxon May 2, 2023
209f988
Update tox.yml -- move exclusions to inside the strategy matrix
AndroxxTraxxon May 2, 2023
4167882
Update tox.yml - removing Python3.10 testing because nose tests are s…
AndroxxTraxxon May 2, 2023
d0d977d
Update test_task_rendering_for_with_items.py
AndroxxTraxxon May 2, 2023
97e3244
Update CHANGELOG.rst
AndroxxTraxxon Sep 20, 2023
e916994
Update CHANGELOG.rst
AndroxxTraxxon Sep 21, 2023
4cdefb6
Merge branch 'master' into master
AndroxxTraxxon Oct 10, 2023
5bb925b
Update tox.yml
AndroxxTraxxon Oct 10, 2023
15b4ee7
Update tox.yml
AndroxxTraxxon Oct 10, 2023
9836b06
Update requirements-ci.txt
AndroxxTraxxon Oct 10, 2023
7ba729d
Update tox.yml
AndroxxTraxxon Oct 10, 2023
1a70b40
replacing nose with pytest, testing python 3.8-3.11
AndroxxTraxxon Oct 10, 2023
015f618
correcting gitlab ci matrix
AndroxxTraxxon Oct 10, 2023
91106ff
tweak the pytest command
AndroxxTraxxon Oct 10, 2023
410b034
attempting python path patching for docs schema generation
AndroxxTraxxon Oct 10, 2023
abda276
switching to use development mode install for docs generation
AndroxxTraxxon Oct 10, 2023
e5917ae
applying formatting, attempting more path shenanigans to get docs wor…
AndroxxTraxxon Oct 10, 2023
f20280b
doing some cleanup for consistency
AndroxxTraxxon Oct 10, 2023
12ee053
re-add Python 3.6 to the test matrix for the patch release before dro…
AndroxxTraxxon Oct 11, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .coveragerc
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,5 @@ exclude_lines =
if __name__ == .__main__.:

ignore_errors = True
omit = orquesta/tests/*

omit =
tests/*
6 changes: 2 additions & 4 deletions .github/workflows/tox.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,10 @@ on:

jobs:
tests:
#python 3.6 not available on ubuntu-latest
runs-on: ubuntu-20.04
strategy:
matrix:
python-version: ["3.6", "3.8"]
python-version: ["3.6", "3.8", "3.9", "3.10", "3.11"]
steps:
- name: Checkout repository
uses: actions/checkout@v4
Expand All @@ -30,5 +29,4 @@ jobs:
pip install -r requirements-ci.txt
make clean reqs schemas
- name: "Run tox for ${{ matrix.python-version }}"
run: |
make check
run: make check
6 changes: 6 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,12 @@ In Development
Changed
~~~~~~~

* Update deprecated `collections` imports to `collections.abc` to be forward-compatible with Python3.10
Contributed by @AndroxxTraxxon
* Migrate from `nosetest` to `pytest` for Python test runner.
Contributed by @AndroxxTraxxon
* Add Python versions 3.9, 3.10, and 3.11 to the test matrix
Contributed by @AndroxxTraxxon
* Update networkx >=2.6 for Python 3.8 to fix insecure deserialization #255 (improvement)
Contributed by @Stealthii
* Update jsonschema requirements to allow 3.2 (improvement)
Expand Down
1 change: 0 additions & 1 deletion orquesta/conducting.py
Original file line number Diff line number Diff line change
Expand Up @@ -370,7 +370,6 @@ def log_entry(
result=None,
data=None,
):

# Check entry type.
if entry_type not in ["info", "warn", "error"]:
raise exc.WorkflowLogEntryError('The log entry type "%s" is not valid.' % entry_type)
Expand Down
6 changes: 3 additions & 3 deletions orquesta/expressions/functions/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import collections
import collections.abc

from orquesta import constants
from orquesta import exceptions as exc
Expand Down Expand Up @@ -117,8 +117,8 @@ def item_(context, key=None):
if not key:
return current_item

if not isinstance(current_item, collections.Mapping):
raise exc.ExpressionEvaluationException("Item is not type of collections.Mapping.")
if not isinstance(current_item, collections.abc.Mapping):
raise exc.ExpressionEvaluationException("Item is not type of collections.abc.Mapping.")

if key not in current_item:
raise exc.ExpressionEvaluationException('Item does not have key "%s".' % key)
Expand Down
4 changes: 2 additions & 2 deletions orquesta/specs/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.

import collections
import collections.abc
import inspect
import json
import jsonschema
Expand Down Expand Up @@ -613,7 +613,7 @@ def update(self, *args, **kwargs):
raise NotImplementedError()


class SequenceSpec(Spec, collections.MutableSequence):
class SequenceSpec(Spec, collections.abc.MutableSequence):
def __init__(self, spec, name=None, member=False):
super(SequenceSpec, self).__init__(spec, name=name, member=member)

Expand Down
2 changes: 0 additions & 2 deletions orquesta/tests/hacking/import_aliases_rule.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@


def get_alias(logical_line):

parts = logical_line.split()

if (
Expand All @@ -68,7 +67,6 @@ def get_alias(logical_line):
and parts[1] != "__future__"
and not core.is_import_exception(parts[1])
):

# from path.to.module import module
if len(parts) == 4:
return ".".join([parts[1], parts[3]]), None
Expand Down
2 changes: 0 additions & 2 deletions orquesta/tests/unit/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,6 @@ def format_task_item(
items_count=None,
items_concurrency=None,
):

if not actions and items_count is None:
actions = [{"action": spec.action, "input": spec.input}]

Expand Down Expand Up @@ -285,7 +284,6 @@ def assert_task_items(
concurrency=None,
mock_ac_ex_results=None,
):

# Set up test cases.
tests = list(zip(mock_ac_ex_statuses, expected_task_statuses, expected_workflow_statuses))
tk_ex_result = [None] * len(items)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ def test_bad_item_type(self):
"type": "error",
"message": (
"YaqlEvaluationException: Unable to evaluate expression '<% item(x) %>'. "
"ExpressionEvaluationException: Item is not type of collections.Mapping."
"ExpressionEvaluationException: Item is not type of collections.abc.Mapping."
),
"task_id": "task1",
"route": 0,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -211,8 +211,7 @@ def test_init_test_spec_with_mock_action_execution_yml_result_path(self):
def test_init_test_spec_with_base_path(self):
shutil.copy(self.get_wf_file_path("sequential"), "/tmp/sequential.yaml")

fd, path = tempfile.mkstemp(suffix=".json")

fd, path = tempfile.mkstemp(suffix=".json", dir="/tmp")
with os.fdopen(fd, "w") as tmp:
tmp.write('{"foo": "bar"}\n')

Expand Down
2 changes: 1 addition & 1 deletion requirements-ci.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Different versions of tox are required by python version
tox-gh-actions
tox==3.28.0 ; python_version == "3.6"
tox==4.6.4 ; python_version == "3.8"
tox==4.11.3; python_version >= "3.8"
9 changes: 0 additions & 9 deletions requirements-test-py27.txt

This file was deleted.

4 changes: 2 additions & 2 deletions requirements-test.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ coverage
flake8<3.9.0,>=3.8.0
hacking
mock>=1.0
nose
nosexcover
pytest
pytest-cov
pep8>=1.6.0,<1.7
pylint>=2.5.2,<2.6
twine
Expand Down
14 changes: 10 additions & 4 deletions tox.ini
Original file line number Diff line number Diff line change
@@ -1,23 +1,27 @@
[tox]
envlist = py36,py38,pep8,docs
envlist = py36,py38,py39,py310,py311,pep8,docs
minversion = 1.6
skipsdist = True

[gh-actions]
python =
3.6: py36,pep8,docs
3.8: py38
3.6: py36
3.8: py38,pep8,docs
3.9: py39
3.10: py310
3.11: py311

[testenv]
usedevelop = True
deps =
-r{toxinidir}/requirements.txt
-r{toxinidir}/requirements-test.txt
commands =
nosetests -sv --with-xcoverage --cover-package=orquesta orquesta.tests
pytest --cov=orquesta --cov-report=term orquesta/tests

[testenv:pep8]
deps =
-r{toxinidir}/requirements.txt
-r{toxinidir}/requirements-test.txt
commands =
pylint -E --rcfile={toxinidir}/.pylintrc orquesta
Expand All @@ -26,7 +30,9 @@ commands =

[testenv:docs]
deps =
-r{toxinidir}/requirements.txt
-r{toxinidir}/requirements-docs.txt
-e{toxinidir}
commands =
python {toxinidir}/bin/orquesta-generate-schemas
sphinx-build -W -b html docs/source docs/build/html