Build fails against setuptools 67.0.0

[stanislavlevin]

setuptools 67 vendors packaging 23.0:
https://setuptools.pypa.io/en/stable/history.html#v67-0-0

pypa/setuptools#3790: Bump vendored version of packaging to 23.0 (pyparsing is no longer required and was removed). As a consequence, users will experience a more strict parsing of requirements. Specifications that don’t comply with PEP 440 and PEP 508 will result in build errors.

While packaging 22.0 removed LegacyVersion:
https://packaging.pypa.io/en/stable/changelog.html
pypa/packaging#407

So, build of lib389 fails

generating man/dsconf.8
Traceback (most recent call last):
  File "/usr/src/RPM/BUILD/389-ds-base-2.2.4/src/lib389/lib389/utils.py", line 46, in <module>
    from pkg_resources.extern.packaging.version import LegacyVersion
ImportError: cannot import name 'LegacyVersion' from 'pkg_resources.extern.packaging.version' (/usr/lib64/python3/site-packages/pkg_resources/_vendor/packaging/version.py)

and all the clients of lib389 are broken:

    from ipaserver.install import ipa_restore
../ipaserver/install/ipa_restore.py:44: in <module>
    from ipaserver.install import dsinstance, httpinstance, cainstance, krbinstance
../ipaserver/install/dsinstance.py:29: in <module>
    from lib389 import DirSrv
/usr/lib/python3/site-packages/lib389/__init__.py:54: in <module>
    from lib389._entry import Entry
/usr/lib/python3/site-packages/lib389/_entry.py:20: in <module>
    from lib389.utils import (ensure_str, ensure_bytes, ensure_list_bytes, display_log_data)
/usr/lib/python3/site-packages/lib389/utils.py:49: in <module>
    from packaging.version import LegacyVersion
E   ImportError: cannot import name 'LegacyVersion' from 'packaging.version' (/usr/lib/python3/site-packages/packaging/version.py)

[svenstaro]

Hit this too. This is currently blocking packaging in Arch Linux.

[vashirov]

We check for DS version to gate some features in lib389 and skip/xfail tests.
First we used string literal comparison, but it doesn't work in certain cases:

>>> '1.3.10.1' > '1.3.2.1'
False

So we switched to version.parse(). But it doesn't work with the development versions:

>>> from packaging import version
>>> x = '2.3.2.202302121950git1b4f5a5bf'
>>> y = '2.3.0'
>>> version.parse(x) > version.parse(y)
False
>>> version.parse(x)
<LegacyVersion('2.3.2.202302121950git1b4f5a5bf')>
>>> version.parse(y)
<Version('2.3.0')>

It returns different objects, and Version is always higher than LegacyVersion, because the latter has epoch -1.
So we switched to LegacyVersion instead, and now it's gone.

And in packaging 23.0 it fails, because our version is not PEP 440 compliant:

>>> version.parse('2.3.2.202302121950git1b4f5a5bf')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.11/site-packages/packaging/version.py", line 52, in parse
    return Version(version)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/packaging/version.py", line 197, in __init__
    raise InvalidVersion(f"Invalid version: '{version}'")
packaging.version.InvalidVersion: Invalid version: '2.3.2.202302121950git1b4f5a5bf'

There is a new package packvers that brings LegacyVersion back. But it's not packaged for any distributions, and I doubt it will ever be.
So I think we need to write our own class for DS version instead of LegacyVersion and use it in lib389.

[mreynolds389]

I'm not sure we even need this version checking anymore...

In nss_ssl.py we use it to check if openssl is greater than 1.0 (current versions are 3.x). Then we use the version check for DS for various tests, but this was used when lib389 was NOT tied to a specific version of DS. Now lib389 is directly packaged with DS. So I don't think we need any of this anymore.

What I don't know if how other distros package openssl. Maybe this check for openssl is still needed on other platforms?

[vashirov]

We use version checking in lib389 in a few places (mainly because of schema differences between 1.3.x and 1.4.x).
And it is used extensively in tests. Recently we had to run some regression tests on RHEL7 (for password storage scheme changes) from the main branch.

[mreynolds389]

We use version checking in lib389 in a few places (mainly because of schema differences between 1.3.x and 1.4.x). And it is used extensively in tests. Recently we had to run some regression tests on RHEL7 (for password storage scheme changes) from the main branch.

Then maybe we should update lib389 in 1.3.10, and remove all this version checking in main?

[vashirov]

We might need to run latest upstream tests on RHEL7 in the future again, so I'd like to keep openssl version check until RHEL7 Z-Stream EOL (June 30, 2024)
I think backporting lib389 to 1.3.10 is a big change, since we never officially supported it on that version. And it doesn't work without changing dse.ldif templates and importing 1.4.x schema anyway.

In the meantime I prepared a patch to replace LegacyVersion: vashirov@67adb7e
I'm currently running more tests in my fork, and I'll open a PR when they pass.

[svenstaro]

Thanks! That patch will work for us. I'd still appreciate it if this were merged.

[vashirov]

@svenstaro, sorry, missed your comment. This is in main branch now: c0e2f68

[svenstaro]

Could you cut a release with this in it?

[mreynolds389]

Could you cut a release with this in it?

We don't do Arch Linux "releases", just Fedora releases. Maybe you mean tag? Well there are a few fixes out for review at the moment, and once they are merged we will do a Fedora release/tag of main branch.

[svenstaro]

Yeah, I do mean a tag which I believe equates to a release of the software in most cases.

[Firstyear]

@mreynolds389 Is there any reason not to backport this to 2.2 / 2.3? We have just hit this in SUSE on those versions.

[vashirov]

We only hit it with Fedora Rawhide (F39, 389-ds-base-2.3) because there was a rebase to setuptools 67: https://bugzilla.redhat.com/show_bug.cgi?id=2183375#c3

389-ds-base-2.3 and main has the fix. And I've just backported it to 2.2:

22c174a..1eeaedf 389-ds-base-2.2 -> 389-ds-base-2.2

[Firstyear]

Thank you <3