You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Skein does not work on distros with newer versions of packages.
I first encountered this issue on NIxOS, but I've also reproduced the issue on arch. See below for repro steps.
Relevant logs/tracebacks
[root@734a72a0bcf4 yay-git]# python -c 'import skein; skein.Client()'
WARNING: log4j.properties is not found. HADOOP_CONF_DIR may be incomplete.
23/03/05 19:18:15 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /0.0.0.0:8032
23/03/05 19:18:15 ERROR skein.Driver: Error running Driver
java.lang.IllegalArgumentException: Java Security Provider unsupported for SslProvider: OPENSSL
at com.anaconda.skein.shaded.io.netty.handler.ssl.SslContext.verifyNullSslContextProvider(SslContext.java:439)
at com.anaconda.skein.shaded.io.netty.handler.ssl.SslContext.newServerContextInternal(SslContext.java:421)
at com.anaconda.skein.shaded.io.netty.handler.ssl.SslContextBuilder.build(SslContextBuilder.java:447)
at com.anaconda.skein.Driver.startServer(Driver.java:128)
at com.anaconda.skein.Driver.run(Driver.java:287)
at com.anaconda.skein.Driver.main(Driver.java:175)
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.10/site-packages/skein/core.py", line 339, in __init__
address, proc = _start_driver(security=security,
File "/usr/lib/python3.10/site-packages/skein/core.py", line 257, in _start_driver
raise DriverError("Failed to start java process")
skein.exceptions.DriverError: Failed to start java process
Exception ignored in: <function Client.__del__ at 0x7f4332b14af0>
Traceback (most recent call last):
File "/usr/lib/python3.10/site-packages/skein/core.py", line 492, in __del__
self.close()
File "/usr/lib/python3.10/site-packages/skein/core.py", line 479, in close
if self._proc is not None:
AttributeError: 'Client' object has no attribute '_proc'
[root@734a72a0bcf4 yay-git]# cat /etc/os-release
NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
VERSION_ID=TEMPLATE_VERSION_ID
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
PRIVACY_POLICY_URL="https://terms.archlinux.org/docs/privacy-policy/"
LOGO=archlinux-logo
Repro
Build this dockerfile:
FROM archlinux:latest
# install hadoop from AURRUN pacman -Syu --noconfirm && pacman -S --noconfirm git base-devel && \
useradd -m -G wheel -s /bin/bash build && \
echo '%wheel ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers
USER build
WORKDIR /home/build
RUN git clone https://aur.archlinux.org/yay.git && cd yay && makepkg -si --noconfirm
RUN yay -S --noconfirm hadoop
RUN sudo pacman -S --noconfirm python-pip
RUN sudo pip install skein
RUN JAVA_HOME=/usr/lib/jvm/java-11-openjdk python -c 'import skein; skein.Client()'
Version information
Python version: 3.10.9
Hadoop version: 3.3.4
Skein version: 0.8.2
I'm not entirely sure what causes this issue, but I suspect it has something to do with openssl. I found that the following patch fixes the issue:
diff --git a/java/src/main/java/com/anaconda/skein/ApplicationMaster.java b/java/src/main/java/com/anaconda/skein/ApplicationMaster.java
index 136c27e..56a09d6 100644
--- a/java/src/main/java/com/anaconda/skein/ApplicationMaster.java+++ b/java/src/main/java/com/anaconda/skein/ApplicationMaster.java@@ -299,7 +299,7 @@ public class ApplicationMaster {
.forServer(new File(".skein.crt"), new File(".skein.pem"))
.trustManager(new File(".skein.crt"))
.clientAuth(ClientAuth.REQUIRE)
- .sslProvider(SslProvider.OPENSSL)+ .sslProvider(SslProvider.JDK)
.build();
NioEventLoopGroup eg = new NioEventLoopGroup(NUM_EVENT_LOOP_GROUP_THREADS);
diff --git a/java/src/main/java/com/anaconda/skein/Driver.java b/java/src/main/java/com/anaconda/skein/Driver.java
index 9010d9b..b5a7e31 100644
--- a/java/src/main/java/com/anaconda/skein/Driver.java+++ b/java/src/main/java/com/anaconda/skein/Driver.java@@ -124,7 +124,7 @@ public class Driver {
.forServer(certBytes.newInput(), keyBytes.newInput())
.trustManager(certBytes.newInput())
.clientAuth(ClientAuth.REQUIRE)
- .sslProvider(SslProvider.OPENSSL)+ .sslProvider(SslProvider.JDK)
.build();
NioEventLoopGroup eg = new NioEventLoopGroup(NUM_EVENT_LOOP_GROUP_THREADS);
The text was updated successfully, but these errors were encountered:
Description of the bug
Skein does not work on distros with newer versions of packages.
I first encountered this issue on NIxOS, but I've also reproduced the issue on arch. See below for repro steps.
Relevant logs/tracebacks
Repro
Build this dockerfile:
Version information
I'm not entirely sure what causes this issue, but I suspect it has something to do with openssl. I found that the following patch fixes the issue:
The text was updated successfully, but these errors were encountered: