Fix so_original_dst retrieval in dual-stack mode

[alpeb]

In 9ee109a we generalized the so_original_dst function to first retrieve the socket's SO_DOMAIN in order to determine what constants to use to retrieve the original destination, that would work for both IP families.

This works fine when the socket is bound to 0.0.0.0 and receives a connection to the pod's IPv4 address, or when bound to :: and receives a connection to the pod's IPv6 address. When the cluster is in dual-stack mode and we bind the socket to ::, it properly receives connections targeted at both the pod's IPv4 and IPv6 addresses, but in the former case the getsockopt call to retrieve the original destination fails with:

WARN ThreadId(01) inbound: linkerd_app_core::serve: Server failed to accept connection error=No such file or directory (os error 2)

The problem is that in this case we're dealing with an IPv4-mapped IPv6 address so SO_DOMAIN returns AF_INET6 but apparently in this case we require AF_INET when interrogating getsockopt for the original destination.

To fix, instead of asking for SO_DOMAIN we determine the peer's IP family and use that to set the appropriate constants for getsocketopt.

[olix0r]

With a mind towards (1) keeping unsafe code as minimal as possible and (2) avoiding needless per connection work, I'm curious if we can rework this slightly:

• Can we keep our unsafe code in distinct so_original_dst_v4 and so_original_dst_v6 functions so that the determination logic is handled in safe code?
• We should not need to invoke peer_addr() again. This incurs another syscall. In BindWithOrigDst, we should be able to add the type constraint B::Addrs: Param<Remote<ClientAddr>> and then get the already-read peer address from the socket metadata. This would be something like

        let incoming = incoming.map(|res| {
            let (inner, tcp) = res?;
            let is_ipv4 = match inner.param() {
                Remote(ClientAddr(SocketAddr::V4(_)))) => true,
                Remote(ClientAddr(SocketAddr::V6(a))) => a.ip().to_ipv4_mapped().is_some(),
            };
            let orig_dst = if is_ipv4 {
                orig_dst_addr_v4(&tcp)?;
            } else {
                orig_dst_addr_v6(&tcp)?;
            };
            let addrs = Addrs { inner, orig_dst };
            Ok((addrs, tcp))
        })

[alpeb]

Ah that makes a lot of sense. I've pushed the suggested changes 👍