Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jgit apache #121

Open
appiepollo14 opened this issue Mar 28, 2024 · 27 comments · May be fixed by #122
Open

jgit apache #121

appiepollo14 opened this issue Mar 28, 2024 · 27 comments · May be fixed by #122

Comments

@appiepollo14
Copy link

Hi,

I'ld love to be able to use jgit apache with sshd in quarkus. I found the quarkiverse sshd extension. However, I can't use it in my opinion in combination with quarkus-jgit, as that plugin is based in jgit, not on jgit.apache.

What are your views on solving this question. How can I combine the two?

  • Is there any documentation on making an upstream dependency native compilable so I can use the apache version myself.
  • Should we create a new quarkiverse extension for jgit-apache?
  • Is there a plan on upgrading to the apache version as upstream for this extension quarkus.jgit?

Would love to talk.

@gastaldi
Copy link
Member

Can you elaborate on what you need specifically from org.eclipse.jgit.ssh.apache that isn't provided by org.eclipse.jgit.ssh.jsch?

In our tests we found the jsch implementation to be less painful to integrate with and rather sufficient for the use cases we encountered so far.

@appiepollo14
Copy link
Author

The fact that it contains a sshdfactory to use sshd as the ssh client in jgit instead of jsch

@gastaldi
Copy link
Member

gastaldi commented Mar 28, 2024

There is https://github.com/quarkiverse/quarkus-sshd which aims to provide support for Apache Mina. Can you give it a try by depending on that extension instead of quarkus-jsch?
Don't forget to also exclude org.eclipse.jgit:org.eclipse.jgit.ssh.jsch

@appiepollo14
Copy link
Author

I understand. But the factory is part of jgit itself. So there is no way to use sshd as the client with the quarkus jgit as the upstream doesn't contain that class. That's what in the classes here from the upstream. https://git.eclipse.org/r/plugins/gitiles/jgit/jgit/+/master/org.eclipse.jgit.ssh.apache/src/org/eclipse/jgit/transport/sshd

If there is another way, please let me know.

@gastaldi
Copy link
Member

gastaldi commented Mar 28, 2024

Yes, in addition to excluding the above dependency, you need to add a dependency to org.eclipse.jgit:org.eclipse.jgit.ssh.apache

@appiepollo14
Copy link
Author

That is exactly my point. org.eclipse.jgit:org.eclipse.jgit.ssh.apache is a replacement for org.eclipse.jgit:org.eclipse.jgit.ssh. So when introducing the apache one, the quarkus jgit extension has no longer use in the project. When adding the apache dependency it will not natively compile anymore. Hence my initial question. Should this be a successor of the current jgit quarkus upstream dependency, should I make it natively compilable myself, than please are there any docs, of a completely new extension maybe?

@gastaldi
Copy link
Member

As I said in #121 (comment), my hope is that quarkus-sshd would fix the native compilation issues, since org.eclipse.jgit:org.eclipse.jgit.ssh.apache depends on the same library.

We could introduce separate extensions here to support the different SSH providers, so the jsch would depend on quarkus-jsch and the other quarkus-sshd if that makes sense.

@gastaldi gastaldi linked a pull request Mar 29, 2024 that will close this issue
@gastaldi
Copy link
Member

gastaldi commented Mar 29, 2024

@appiepollo14 I refactored the extension by separating the JSCH SSH provider into another extension. Same for SSHD SSH provider: #122

Please give it a try and let me know how it goes. This still needs some polishing before being merged.

Your project will look like this:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>io.quarkiverse.jgit</groupId>
            <artifactId>quarkus-jgit-bom</artifactId>
            <version>999-SNAPSHOT</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
	 </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>io.quarkiverse.jgit</groupId>
        <artifactId>quarkus-jgit</artifactId>
    </dependency>
    <dependency>
        <groupId>io.quarkiverse.jgit</groupId>
        <artifactId>quarkus-jgit-sshd</artifactId>
    </dependency>
</dependencies>

@appiepollo14
Copy link
Author

Will do and let you know! Thanks for picking up so fast on this!!

@appiepollo14
Copy link
Author

I've tried to checkout the ssh_providers branch. When doing a clean install, there are multiple errors about versions missing in pom files. Hence why I cannot use this code for tests in another project.

@gastaldi
Copy link
Member

gastaldi commented Apr 1, 2024

@appiepollo14 try again please, I forgot to reference the relativePath from the new parent poms

@appiepollo14
Copy link
Author

@gastaldi it now does a clean install! Great! In my project I've setup the dependencies as instructed by you above. The functions I need, do work. Great!

However, when compiling my project natively, it fails. There are some issues with deps??

`14:58:17,123 INFO [org.apa.ssh.com.uti.sec.bou.BouncyCastleSecurityProviderRegistrar] getOrCreateProvider(BC) created instance of org.bouncycastle.jce.provider.BouncyCastleProvider
14:58:17,136 INFO [org.apa.ssh.com.uti.sec.edd.EdDSASecurityProviderRegistrar] getOrCreateProvider(EdDSA) created instance of net.i2p.crypto.eddsa.EdDSASecurityProvider
14:58:17,183 INFO [org.apa.ssh.com.io.DefaultIoServiceFactoryFactory] No detected/configured IoServiceFactoryFactory; using Nio2ServiceFactoryFactory

....

Error: Discovered unresolved type during parsing: sun.security.x509.X509Key. This error is reported at image build time because class net.i2p.crypto.eddsa.EdDSAEngine is registered for linking at image build time by command line and command line.
Error encountered while parsing net.i2p.crypto.eddsa.EdDSAEngine.engineInitVerify(EdDSAEngine.java:147)
Parsing context:
at java.security.Signature.initVerify(Signature.java:505)
at root method.(Unknown Source)
`

Adding:
<dependency> <groupId>net.i2p.crypto</groupId> <artifactId>eddsa</artifactId> <version>0.3.0</version> </dependency>

to the project doesn't make a difference. Any pointers?

@gastaldi
Copy link
Member

gastaldi commented Apr 1, 2024

@appiepollo14 I've pushed a1cf821 (#122) which may fix this issue, can you pull and try again?

@gastaldi
Copy link
Member

gastaldi commented Apr 1, 2024

I've also created quarkiverse/quarkus-sshd#59 asking to resolve that in the quarkus-sshd extension

@appiepollo14
Copy link
Author

appiepollo14 commented Apr 1, 2024

Great! New errors occur: believe also should be fixed in quarkus-sshd. Errors:

`
Error: Unsupported features in 2 methods
Detailed message:
Error: Discovered unresolved method during parsing: org.apache.sshd.common.util.security.eddsa.Ed25519PEMResourceKeyParser.decodeEd25519KeyPair(byte[]). This error is reported at image build time because class org.apache.sshd.common.config.keys.loader.pem.PKCS8PEMResourceKeyPairParser is registered for linking at image build time by command line and command line.
Error encountered while parsing org.apache.sshd.common.config.keys.loader.pem.PKCS8PEMResourceKeyPairParser.extractKeyPairs(PKCS8PEMResourceKeyPairParser.java:129)
Parsing context:
at org.apache.sshd.common.config.keys.loader.pem.PKCS8PEMResourceKeyPairParser.extractKeyPairs(PKCS8PEMResourceKeyPairParser.java:90)
at org.apache.sshd.common.config.keys.loader.AbstractKeyPairResourceParser.extractKeyPairs(AbstractKeyPairResourceParser.java:198)
at org.apache.sshd.common.config.keys.loader.AbstractKeyPairResourceParser.extractKeyPairs(AbstractKeyPairResourceParser.java:167)
at org.apache.sshd.common.config.keys.loader.AbstractKeyPairResourceParser.loadKeyPairs(AbstractKeyPairResourceParser.java:117)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceParser$2.loadKeyPairs(KeyPairResourceParser.java:166)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:157)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:148)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:139)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:130)
at org.apache.sshd.common.util.security.SecurityUtils.loadKeyPairIdentities(SecurityUtils.java:523)
at org.eclipse.jgit.internal.transport.sshd.CachingKeyPairProvider.loadKey(CachingKeyPairProvider.java:156)
at org.eclipse.jgit.internal.transport.sshd.CachingKeyPairProvider.loadKey(CachingKeyPairProvider.java:131)
at org.eclipse.jgit.internal.transport.sshd.CachingKeyPairProvider$CancellingKeyPairIterator.hasNext(CachingKeyPairProvider.java:221)
at jdk.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:513)
at com.fasterxml.jackson.databind.util.ClassUtil.closeOnFailAndThrowAsIOE(ClassUtil.java:527)
at com.fasterxml.jackson.databind.ObjectWriter._writeCloseable(ObjectWriter.java:1295)
at com.fasterxml.jackson.databind.ObjectWriter._writeValueAndClose(ObjectWriter.java:1269)
at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsString(ObjectWriter.java:1140)
at com.fasterxml.jackson.databind.node.InternalNodeMapper.nodeToString(InternalNodeMapper.java:34)
at com.fasterxml.jackson.databind.node.BaseJsonNode.toString(BaseJsonNode.java:242)
at root method.(Unknown Source)

Error: Discovered unresolved type during parsing: net.i2p.crypto.eddsa.EdDSAPublicKey. This error is reported at image build time because class org.apache.sshd.common.util.buffer.keys.SkED25519BufferPublicKeyParser is registered for linking at image build time by command line and command line.
Error encountered while parsing org.apache.sshd.common.util.buffer.keys.SkED25519BufferPublicKeyParser.getRawPublicKey(SkED25519BufferPublicKeyParser.java:45)
Parsing context:
at org.apache.sshd.common.util.buffer.keys.SkED25519BufferPublicKeyParser.getRawPublicKey(SkED25519BufferPublicKeyParser.java:34)
at org.apache.sshd.common.util.buffer.keys.BufferPublicKeyParser$2.getRawPublicKey(BufferPublicKeyParser.java:102)
at org.apache.sshd.common.util.buffer.Buffer.getRawPublicKey(Buffer.java:568)
at org.apache.sshd.common.util.buffer.Buffer.getPublicKey(Buffer.java:543)
at org.apache.sshd.common.util.buffer.Buffer.getPublicKey(Buffer.java:528)
at org.apache.sshd.common.util.buffer.keys.OpenSSHCertPublicKeyParser.getRawPublicKey(OpenSSHCertPublicKeyParser.java:82)
at org.apache.sshd.common.config.keys.impl.OpenSSHCertificateDecoder.decodePublicKey(OpenSSHCertificateDecoder.java:75)
at org.apache.sshd.common.config.keys.impl.OpenSSHCertificateDecoder.decodePublicKey(OpenSSHCertificateDecoder.java:46)
at org.apache.sshd.common.config.keys.loader.openssh.OpenSSHKeyPairResourceParser.readPublicKey(OpenSSHKeyPairResourceParser.java:221)
at org.apache.sshd.common.config.keys.loader.openssh.OpenSSHKeyPairResourceParser.extractKeyPairs(OpenSSHKeyPairResourceParser.java:133)
at org.apache.sshd.common.config.keys.loader.AbstractKeyPairResourceParser.extractKeyPairs(AbstractKeyPairResourceParser.java:198)
at org.apache.sshd.common.config.keys.loader.AbstractKeyPairResourceParser.extractKeyPairs(AbstractKeyPairResourceParser.java:167)
at org.apache.sshd.common.config.keys.loader.AbstractKeyPairResourceParser.loadKeyPairs(AbstractKeyPairResourceParser.java:117)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceParser$2.loadKeyPairs(KeyPairResourceParser.java:166)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:157)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:148)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:139)
at org.apache.sshd.common.config.keys.loader.KeyPairResourceLoader.loadKeyPairs(KeyPairResourceLoader.java:130)
at org.apache.sshd.common.util.security.SecurityUtils.loadKeyPairIdentities(SecurityUtils.java:523)
at org.eclipse.jgit.internal.transport.sshd.CachingKeyPairProvider.loadKey(CachingKeyPairProvider.java:156)
at org.eclipse.jgit.internal.transport.sshd.CachingKeyPairProvider.loadKey(CachingKeyPairProvider.java:131)
at org.eclipse.jgit.internal.transport.sshd.CachingKeyPairProvider$CancellingKeyPairIterator.hasNext(CachingKeyPairProvider.java:221)
at jdk.nio.zipfs.ZipFileSystem.close(ZipFileSystem.java:513)
at com.fasterxml.jackson.databind.util.ClassUtil.closeOnFailAndThrowAsIOE(ClassUtil.java:527)
at com.fasterxml.jackson.databind.ObjectWriter._writeCloseable(ObjectWriter.java:1295)
at com.fasterxml.jackson.databind.ObjectWriter._writeValueAndClose(ObjectWriter.java:1269)
at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsString(ObjectWriter.java:1140)
at com.fasterxml.jackson.databind.node.InternalNodeMapper.nodeToString(InternalNodeMapper.java:34)
at com.fasterxml.jackson.databind.node.BaseJsonNode.toString(BaseJsonNode.java:242)
at root method.(Unknown Source)
`

@gastaldi
Copy link
Member

gastaldi commented Apr 1, 2024

Can you please add this stacktrace to the issue I created earlier along with a reproducer?

@appiepollo14
Copy link
Author

Will do

appiepollo14 added a commit to appiepollo14/quarkus-reproducer that referenced this issue Apr 4, 2024
@appiepollo14
Copy link
Author

@gastaldi make a small reproducer.

However, it won't even natively compile now due to:
Caused by: java.lang.SecurityException: class "org.eclipse.jgit.transport.sshd.SshdSessionFactory"'s signer information does not match signer information of other classes in the same package Looks to be similar to: spring-cloud/spring-cloud-contract#1822

How to continue?

@gastaldi
Copy link
Member

gastaldi commented Apr 5, 2024

@zakkak could this be a GraalVM issue? Here is the complete stacktrace:

com.oracle.svm.core.util.VMError$HostedError: InternalFeature defined by com.oracle.svm.hosted.reflect.ReflectionFeature unexpectedly failed with a(n) java.lang.SecurityException
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.util.VMError.shouldNotReachHere(VMError.java:86)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.handleFeatureError(FeatureHandler.java:287)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:92)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.setupNativeImage(NativeImageGenerator.java:964)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.doRun(NativeImageGenerator.java:590)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.run(NativeImageGenerator.java:550)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.buildImage(NativeImageGeneratorRunner.java:539)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.build(NativeImageGeneratorRunner.java:721)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.start(NativeImageGeneratorRunner.java:143)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGeneratorRunner.main(NativeImageGeneratorRunner.java:98)
Caused by: java.lang.SecurityException: class "org.eclipse.jgit.transport.sshd.SshdSessionFactory"'s signer information does not match signer information of other classes in the same package
	at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1173)
	at java.base/java.lang.ClassLoader.preDefineClass(ClassLoader.java:917)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1025)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoader.defineClass(NativeImageClassLoader.java:500)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoader.findClassViaClassPath(NativeImageClassLoader.java:452)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoader.loadClass(NativeImageClassLoader.java:640)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:534)
	at java.base/java.lang.Class.forName(Class.java:513)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageClassLoader.forName(ImageClassLoader.java:307)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageClassLoader.forName(ImageClassLoader.java:303)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.ImageClassLoader.findClass(ImageClassLoader.java:296)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.config.RegistryAdapter.resolveType(RegistryAdapter.java:77)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.config.ReflectionRegistryAdapter.resolveType(ReflectionRegistryAdapter.java:48)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ReflectionConfigurationParser.parseClass(ReflectionConfigurationParser.java:94)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ReflectionConfigurationParser.parseClassArray(ReflectionConfigurationParser.java:74)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ReflectionConfigurationParser.parseAndRegister(ReflectionConfigurationParser.java:69)
	at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.parseAndRegister(ConfigurationParser.java:73)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.config.ConfigurationParserUtils.doParseAndRegister(ConfigurationParserUtils.java:130)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.config.ConfigurationParserUtils.lambda$parseAndRegisterConfigurations$2(ConfigurationParserUtils.java:116)
	at java.base/java.util.stream.ReferencePipeline$4$1.accept(ReferencePipeline.java:214)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.config.ConfigurationParserUtils$1.tryAdvance(ConfigurationParserUtils.java:109)
	at java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:332)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
	at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:276)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1708)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)
	at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.IntPipeline.reduce(IntPipeline.java:515)
	at java.base/java.util.stream.IntPipeline.sum(IntPipeline.java:473)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.config.ConfigurationParserUtils.parseAndRegisterConfigurations(ConfigurationParserUtils.java:118)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.config.ConfigurationParserUtils.parseAndRegisterConfigurations(ConfigurationParserUtils.java:75)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.reflect.ReflectionFeature.duringSetup(ReflectionFeature.java:265)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageGenerator.lambda$setupNativeImage$16(NativeImageGenerator.java:964)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.FeatureHandler.forEachFeature(FeatureHandler.java:90)
	... 7 more

@appiepollo14
Copy link
Author

@gastaldi Or possible because jgit might be referenced twice in the dependency tree based on the separated providers?

@zakkak
Copy link
Contributor

zakkak commented Apr 8, 2024

@zakkak could this be a GraalVM issue?

I don't think so and I haven't seen this again.

@gastaldi Or possible because jgit might be referenced twice in the dependency tree based on the separated providers?

That sounds more plausible.

@appiepollo14
Copy link
Author

@gastaldi how can we proceed with this topic? Can we steer this branch to include the jgit only once per ssh provider or should we separate this extension into two extensions (jsch/sshd)?

@gastaldi
Copy link
Member

It seems that Quarkus is modifying that artifact (I can see a modified-org.eclipse.jgit.org.eclipse.jgit.ssh.apache-6.9.0.202403050737-r.jar in the quarkus-reproducer-1.0.0-SNAPSHOT-native-image-source-jar\lib directory.

I'm investigating what's causing this

@gastaldi
Copy link
Member

@appiepollo14 I created quarkusio/quarkus#40001 to fix the signed JAR issue

@appiepollo14
Copy link
Author

@gastaldi can we test the result quarkusio/quarkus#40001 for the quarkus-jgit issue before a new Quarkus version is released? If yes, please tell me how.

@gastaldi
Copy link
Member

Sure, you just need to build Quarkus (using mvn -Dquickly) and use io.quarkus:quarkus-bom:999-SNAPSHOT in your sample project instead

@appiepollo14
Copy link
Author

appiepollo14 commented Apr 17, 2024

Did it, thanks. Native compilation fails again due to quarkiverse/quarkus-sshd#59, will focus on that one again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants