IPv6 Next-hop address not being advertised correctly to peer

[alungd]

Hi,

Hoping someone can help me with a little issue I'm having upgrading a system from CentOS7/FRR7.2 to RHEL9 FRR9.1.1.

We've had the FRR7.2 install in place for 4 or 5 years with no issues, but we need to upgrade the hosts to RHEL9 and at the same time I thought I'd take the opportunity to upgrade FRR to the latest 9.1.1. We use it to advertise some local routes out to the network via BGP, and on this new version (same config) v4 is working identically to before, but the next-hop we set for v6 appears to be getting overwritten. Here's a snippet from the debug log:

2024/06/28 13:42:56 BGP: [TN0HX-6G1RR] u2:s2 send UPDATE w/ attr: , origin ?, mp_nexthop fd00::a02:202, metric 0, community 5089:4802, path 65000 65000 65000
2024/06/28 13:42:56 BGP: [H06SA-0JAPR] u2:s2 send MP_REACH for afi/safi IPv6/unicast
2024/06/28 13:42:56 BGP: [HVRWP-5R9NQ] u2:s2 send UPDATE 2a04:3541:1000:500:1405:37ff:fe8a:6be9/128 IPv6 unicast
2024/06/28 13:42:56 BGP: [WEV7K-2GAQ5] u2:s2 send UPDATE len 105 (max message len: 4096) numpfx 1
2024/06/28 13:42:56 BGP: [NNG4V-WFG7M] u2:s2 10.178.224.63 send UPDATE w/ mp_nexthop ::ffff:ab2:dd61

As you can see the next-hop is correctly set to fd00::a02:202 in the first instance, but then in the final line gets set to ::ffff:ab2:dd61, the address we have configured on the mgmt interface. The route-map where this gets set looks like this:

route-map tagmap_blocklist6 permit 30
 match tag 9999
 set as-path prepend 65000 65000 65000
 set community 5089:4802 additive
 set ipv6 next-hop global fd00::a02:202
exit

I have confirmed its matching against this route-map correctly and as I said this config is the same one that's been working perfectly for years.

I also checked this behaviour against FRR8, and get the same issue. So it looks like something has changed between FRR7.2 and later versions.

If anyone has any clues about this change in behaviour and how I work around it, it would be really appreciated!

Thanks,

Al

[ton31337]

Could you show the full configuration?

[alungd]

Sure. here's the full frr.conf:

frr version 9.1.1
frr defaults traditional
hostname XXXXX
log file /var/log/frr/frr.log
no ip forwarding
no ipv6 forwarding
hostname router-XXXXX
domainname XXXXX
service integrated-vtysh-config
!
debug zebra events
debug zebra rib
debug zebra nexthop detail
debug bgp neighbor-events
debug bgp updates in
debug bgp updates out
debug bgp zebra
debug bgp conditional-advertisement
!
debug route-map
!
password zebra
enable password zebra
!
ip route 92.242.132.24/32 82.8.211.254 tag 9999
ipv6 route 2a04:3541:1000:500:1405:37ff:fe8a:6be9/128 2a02:88fe:a::1 tag 9999
!
router bgp 65000
bgp router-id 10.178.221.97
neighbor 10.178.224.63 remote-as 5089
neighbor 10.178.224.63 ebgp-multihop
!
address-family ipv4 unicast
redistribute static
neighbor 10.178.224.63 route-map tagmap_blocklist out
exit-address-family
!
address-family ipv6 unicast
redistribute static
neighbor 10.178.224.63 activate
neighbor 10.178.224.63 route-map tagmap_blocklist6 out
exit-address-family
exit
!
bgp community-list 60 seq 5 permit 5089:4801
bgp community-list 90 seq 5 permit 5089:4802
bgp community-list 95 seq 5 permit 5089:4803
!
route-map tagmap_blocklist permit 10
match tag 6666
set as-path prepend 65000 65000 65000
set community 5089:4801 additive
set ip next-hop 10.1.1.1
exit
!
route-map tagmap_blocklist permit 20
match ip next-hop type blackhole
match tag 9999
set as-path prepend 65000 65000 65000
set community 5089:4802 5089:4803 additive
set ip next-hop 10.178.221.97
exit
!
route-map tagmap_blocklist permit 30
match tag 9999
set as-path prepend 65000 65000 65000
set community 5089:4802 additive
set ip next-hop 10.2.2.2
exit
!
route-map tagmap_blocklist6 permit 10
match tag 6666
set as-path prepend 65000 65000 65000
set community 5089:4801 additive
set ipv6 next-hop global fd00::a01:101
exit
!
route-map tagmap_blocklist6 permit 20
match ipv6 next-hop type blackhole
match tag 9999
set as-path prepend 65000 65000 65000
set community 5089:4802 5089:4803 additive
set ipv6 next-hop global ::ffff:ab2:dd61
exit
!
route-map tagmap_blocklist6 permit 30
match tag 9999
set as-path prepend 65000 65000 65000
set community 5089:4802 additive
set ipv6 next-hop global fd00::a02:202
exit
!

The static IPs in this config get updated by a script that adds and removes them as required.

Thanks!

[alungd]

Update: I found #15611 earlier, which appears to be a bug thats very similar to what I'm seeing that's already being (been?) addressed. Unless I'm reading it wrongly?

[alungd]

Struggling to find a work around for this. I've tried releases back to 8.2.2, which appears to be the first release to support RHEL9, and the issue is there. I've also tried FRR9 on my old CentOS7 server which FRR7.2 is working correctly, and it has the same issue, so it certainly seems to be a change that happened between FRR versions 7.2 and 8.2.2 somewhere.

Does anyone have any thoughts on whether this is something we can fix or work around?

Thanks,

Al