Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python3Packages.skein: fix broken #203726

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

illustris
Copy link
Contributor

@illustris illustris commented Nov 30, 2022

Description of changes

dask-yarn has been failing to build since https://hydra.nixos.org/build/196296193
The underlying cause is a runtime issue in skien introduced somewhere between 4f8287f and 31acb60

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@illustris
Copy link
Contributor Author

@ofborg build python3Packages.skein

@illustris
Copy link
Contributor Author

@ofborg build python3Packages.skein.passthru.tests.smokeTest

@illustris
Copy link
Contributor Author

Seems to be an upstream issue. Also tested on arch to rule out any nixos quirks.

[root@734a72a0bcf4 yay-git]# python -c 'import skein; skein.Client()'
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
23/03/05 19:18:15 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /0.0.0.0:8032
23/03/05 19:18:15 ERROR skein.Driver: Error running Driver
java.lang.IllegalArgumentException: Java Security Provider unsupported for SslProvider: OPENSSL
        at com.anaconda.skein.shaded.io.netty.handler.ssl.SslContext.verifyNullSslContextProvider(SslContext.java:439)
        at com.anaconda.skein.shaded.io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:421)
        at com.anaconda.skein.shaded.io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:447)
        at com.anaconda.skein.Driver.startServer(Driver.java:128)
        at com.anaconda.skein.Driver.run(Driver.java:287)
        at com.anaconda.skein.Driver.main(Driver.java:175)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.10/site-packages/skein/core.py", line 339, in __init__
    address, proc = _start_driver(security=security,
  File "/usr/lib/python3.10/site-packages/skein/core.py", line 257, in _start_driver
    raise DriverError("Failed to start java process")
skein.exceptions.DriverError: Failed to start java process
Exception ignored in: <function Client.__del__ at 0x7f4332b14af0>
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/skein/core.py", line 492, in __del__
    self.close()
  File "/usr/lib/python3.10/site-packages/skein/core.py", line 479, in close
    if self._proc is not None:
AttributeError: 'Client' object has no attribute '_proc'
[root@734a72a0bcf4 yay-git]# cat /etc/os-release
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
VERSION_ID=TEMPLATE_VERSION_ID
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo

The fix is to replace the sslprovider in skein's java code

diff --git a/java/src/main/java/com/anaconda/skein/ApplicationMaster.java b/java/src/main/java/com/anaconda/skein/ApplicationMaster.java
index 136c27e..56a09d6 100644
--- a/java/src/main/java/com/anaconda/skein/ApplicationMaster.java
+++ b/java/src/main/java/com/anaconda/skein/ApplicationMaster.java
@@ -299,7 +299,7 @@ public class ApplicationMaster {
         .forServer(new File(".skein.crt"), new File(".skein.pem"))
         .trustManager(new File(".skein.crt"))
         .clientAuth(ClientAuth.REQUIRE)
-        .sslProvider(SslProvider.OPENSSL)
+        .sslProvider(SslProvider.JDK)
         .build();

     NioEventLoopGroup eg = new NioEventLoopGroup(NUM_EVENT_LOOP_GROUP_THREADS);
diff --git a/java/src/main/java/com/anaconda/skein/Driver.java b/java/src/main/java/com/anaconda/skein/Driver.java
index 9010d9b..b5a7e31 100644
--- a/java/src/main/java/com/anaconda/skein/Driver.java
+++ b/java/src/main/java/com/anaconda/skein/Driver.java
@@ -124,7 +124,7 @@ public class Driver {
         .forServer(certBytes.newInput(), keyBytes.newInput())
         .trustManager(certBytes.newInput())
         .clientAuth(ClientAuth.REQUIRE)
-        .sslProvider(SslProvider.OPENSSL)
+        .sslProvider(SslProvider.JDK)
         .build();

     NioEventLoopGroup eg = new NioEventLoopGroup(NUM_EVENT_LOOP_GROUP_THREADS);

Patching this in nixos won't be easy, because of issues with the reproducibility of maven builds. This one in particular extracts a protoc binary from a tarball during the build process, making patchelf useless. I'll raise the issue upstream to update the binaries on pypi.

@illustris
Copy link
Contributor Author

Upstream issue:
jcrist/skein#249

@illustris illustris added the 2.status: wait-for-upstream Waiting for upstream fix (or their other action). label Oct 19, 2023
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 19, 2024
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label May 22, 2024
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.status: merge conflict 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 2.status: wait-for-upstream Waiting for upstream fix (or their other action). 6.topic: python 10.rebuild-darwin: 0 10.rebuild-linux: 0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants