-
-
Notifications
You must be signed in to change notification settings - Fork 79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
modern lx images have broken IPv6 due missing fallback #1219
Comments
I created my zones like this: ---
autoboot: 'false'
bootargs: ''
brand: lx
cpu-shares: '1'
dns-domain: acheron.be
fs-allowed: ''
hostid: ''
ip-type: exclusive
ipv6: 'true'
kernel-version: '4.4'
limitpriv: default
net:
- allowed-address: 10.23.10.159/20
defrouter: 10.23.10.1
global-nic: ixgbe0
mac-addr: 2:8:20:e5:1:33
physical: lxtest0
vlan-id: '100'
- allowed-address: 2a02:XXX:XXX:10::159/64
global-nic: ixgbe0
mac-addr: 2:8:20:e5:1:33
physical: lxtest0
vlan-id: '100'
pool: ''
resolvers:
- 2a02:XXX:XXX:10::153
- 10.23.11.53
- 2a02:XXX:XXX:10::253
- 10.23.12.53
scheduling-class: ''
zonename: lxtest
zonepath: /zones/lxtest zadm create -b lx -i 834bbc6d lxtest < test.sjon I ran in.ndpd manually to get the IPv6 default route as lxinit currently does not start it. The following images where used:
|
So ... yeah past me was here already, probably why I had a note to retest this down the line and hit this again and forgot why I left the note to self: TritonDataCenter/smartos-live#998 |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
Unstale |
Ran into this again while testing some things in an lx zone, IPv6 is broken in some subtle ways. Edit: there might also be some related nfs issues, when mounting over IPv4 it works but over IPv6 it fails with |
Here is the same but using dtrace-dtruss.d curl ifconfig.cl instead of strace |
I dug out the old lxunsup.d script form way back in the day:
Not sure what the 2 out of bounds are, but it looks like setsockopt is missing some stuff :( Looks like it's always that one that shows up for every util that hits the resolveer failure:
Seems that lxsockopts.d agrees
|
I vaguely remember 41 being SOL_IPV6, not sure on SOCKOP 25 though. |
NOTE: The following analysis applies to all available LX implementations. I looked at SmartOS for my answers below. Socket option 25 is IPV6_RECVERR, which has an IPv4 counterpart IP_RECVERR of 11:
We, and even FreeBSD, don't have this in the native TCP stacks:
Building this would be a challenge. Might be worth it, but at first glance (new recvmsg(3socket) and queuing of networking errors) it would be difficult. |
I spend some more time digging around, as it didn't make sense to me why only the IPv6 one would be failing since we support neither natively. @danmcd already pointed out FreeBSD lies about supporting it to make things 'work'. But after spending my lunch break in usr/src/uts/common/brand/lx ... illumos-omnios/usr/src/uts/common/brand/lx/syscall/lx_socket.c Lines 3263 to 3277 in 1ad7f63
We lie about it too for IP_RECVERR! So that is probably why glibc has a working resolver with just IPv4 ones. I think we can just add do the same for it here:
And that will probbaly make glibc resolver work with IPv6 nameservers, if I understand correctly the it would just mean they could timeout on communication failures instead of immediately bailing. I would say that is preferable to it just bailing with an error and not functioning at all. Given both we and freebsd already lie about it for IP_RECVERR I think it should be acceptable to do so for IPV6_RECVERR. |
diff --git usr/src/uts/common/brand/lx/syscall/lx_socket.c usr/src/uts/common/brand/lx/syscall/lx_socket.c
index a433020f90..15118fe8aa 100644
--- usr/src/uts/common/brand/lx/syscall/lx_socket.c
+++ usr/src/uts/common/brand/lx/syscall/lx_socket.c
@@ -3351,6 +3351,11 @@ lx_setsockopt_ipv6(sonode_t *so, int optname, void *optval, socklen_t optlen)
lx_proto_opts_t sockopts_tbl = PROTO_SOCKOPTS(ltos_ipv6_sockopts);
switch (optname) {
+ case LX_IPV6_RECVERR:
+ /*
+ * Ping and glibc's resolver set this, see lx_setsockopt_ip
+ */
+ return (0);
case LX_IPV6_MTU:
/*
* There isn't a good translation for IPV6_MTU and certain apps Gonna try and build this 🤞 |
I might have an editorial comment on the comment itself, but good digging on what we'd done and what we need to do. Honestly you should file a PR against one of here or illumos-joyent, and make sure you indicate me on the review. |
This is very promising |
The last know working lx image with IPv6 support is ubuntu 16.04, anything newer (18.04-22.04, CentOS Stream 9, Debian Bullseye) exibit weird issues. Especially if a an IPv6 resolver is used.
When an IPv6 resolver is used name resolving seems completely broken, probably because we're hitting the not implemented bits.
Without IPv6 resolvers present in /etc/resolv.conf at least name resolving works and 'some' IPv6 stuff too, e.g curl seems to work fine. But stuff like ping is still broken.
On 16.04 ping prints a warning about running an older kernel:
On newer images ping just barfs about protocol stuff
The text was updated successfully, but these errors were encountered: