TLS handshake cert error

[ianopolous]

I'm implementing a protocol using this and get this error in the handshake to an external server.

javax.net.ssl.SSLHandshakeException: QUICHE_ERR_TLS_FAIL: error:1000007e:SSL routines:OPENSSL_internal:CERT_CB_ERROR

The certificates are self signed, with an extension, and I have a custom trustmanager which is correctly verifying the certs and extension.

The same cert code works fine with TLS over TCP and netty. I've found this doc on the openssl error: https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_set_cert_cb.html which suggest some kind of error in the cert callback.

Any advice would be much appreciated, thank you for the awesome project.

[ianopolous]

Upon digging deeper, that link above says the callback should return 1 on success and 0 on error when verifying certs. But stepping through in the debugger I see BoringSSLCertificateVerifyCallback.java is returning BoringSSL.X509_V_OK, which is 0.

Am I barking up the wrong tree?

[ianopolous]

I've tried modifying the return value of that callback in BoringSSLCertificateVerifyCallback from 0 to 1 and then it gives a different error:
QUICHE_ERR_TLS_FAIL: error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
So seems like that's not the problem.

[normanmaurer]

can you share your impl ? Also are you sure its not throwing an exception etc ?

[ianopolous]

Sure, this is the relevant bit of code (Kotlin):
https://github.com/Peergos/jvm-libp2p/blob/feat/quic/src/main/kotlin/io/libp2p/transport/quic/QuicTransport.kt#L158
The TrustManager implementation is here (I checked it's returning fine without error):
https://github.com/Peergos/jvm-libp2p/blob/feat/quic/src/main/kotlin/io/libp2p/security/tls/TLSSecureChannel.kt#L193

[ianopolous]

There are examples of the kind of certs it's receiving here: It is always a single self signed cert with an extension:
https://github.com/Peergos/jvm-libp2p/blob/feat/quic/src/test/kotlin/io/libp2p/security/tls/CertificatesTest.kt

[ianopolous]

I've double checked that no exceptions are being thrown in any of my code. I also tried breaking on "any exception" and that didn't hit anything either, so it seems nothing in the JDK classes is throwing either.

[ianopolous]

I've pushed a minimalish test that reproduces it here:
https://github.com/Peergos/jvm-libp2p/blob/feat/quic/src/test/java/io/libp2p/core/QuicPingTestJava.java#L17
This test assumes the install-run-ipfs.sh in root has been run first to install and run the (Golang) server.

[ianopolous]

I've implemented the server side as well now, and got a pure java test that shows the same error, so no need for the external Golang server. I still might be doing something dumb in the client setup. Thank you for any suggestions.

This is the java client -> java server test:
https://github.com/Peergos/jvm-libp2p/blob/feat/quic/src/test/java/io/libp2p/core/QuicServerTestJava.java#L15

[ianopolous]

I've tracked this down to us using Ed25519 certificates, so it was Boringssl that was throwing the exception because of an unsupported key type (BoringSSLCertificateCallback line 161). It seems like they don't support Ed25519 certificates:
cloudflare/boring#113
cloudflare/quiche#1482

[normanmaurer]

So I guess can close this one ?

[ianopolous]

The 2nd linked issue suggests it's possible to add ed25519 to the boringssl config in the signature algorithms list. It would be great to enable it by default if possible.

[normanmaurer]

@ianopolous I wonder if its a good idea to enable it by default. I guess we could allow to enable it tho .

[ianopolous]

It's more modern and secure than all the other algorithms. So I don't see a reason not to enable it.

[normanmaurer]

@ianopolous I am mostly "caution" here as BoringSSL doesn't do it by default. Usually these people have a good reason for doing so :)

[ianopolous]

Sure, it's a good question. Just having the option at all would be great.

[normanmaurer]

/cc @davidben ... Any thoughts ?

[davidben]

Ed25519 isn't enabled by default for TLS in BoringSSL. At this layer of the stack, introducing key types has a very long runway since it goes through your CAs, etc. Ed25519 is nicer[*] than ECDSA P-256, but it's not enough of an improvement to justify the cost of having too many options here.

Also, it's 2023. Putting a lot of effort into a new classical algorithm is kinda silly, when we'll need to replace it all with postquantum algorithms anyway.

If Netty really wants to enable it, the feature does exist in the library. But doing so will diverge from our defaults and what web browsers do. (In hindsight, I probably shouldn't have wasted time wiring it up to the TLS library, but, ah well, it's there now. 😄)

[*] Broadly. The cofactors are a nuisance, and some corners of verification were hideously underspecified, which left quite a mess.

[ianopolous]

Thanks for your thoughts @davidben Our use case is the ipfs and libp2p ecosystem, which thankfully doesn't depend on CAs or web browsers.

@normanmaurer Having the option at all to enable it would be great.

[normanmaurer]

@ianopolous let me look into it