Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BGP peering with BFD is established twice #17337

Open
2 tasks done
karampok opened this issue Nov 4, 2024 · 14 comments · May be fixed by #17344
Open
2 tasks done

BGP peering with BFD is established twice #17337

karampok opened this issue Nov 4, 2024 · 14 comments · May be fixed by #17344
Assignees
Labels

Comments

@karampok
Copy link

karampok commented Nov 4, 2024

Description

When doing a BGP Peering with BFD between two FRR peers (peer A and peer B), the BGP is established but the peering takes place twice. This can be observed by the

show bgp neigh
...
 Connections established 2; dropped 1
...

Version

quay.io/frrouting/frr:9.1.0

in both peers

FRRouting 9.1_git (frr-k8s-control-plane) on Linux(6.11.5-200.fc40.x86_64).
Copyright 1996-2005 Kunihiro Ishiguro, et al.
configured with:
    '--prefix=/usr' '--sbindir=/usr/lib/frr' '--sysconfdir=/etc/frr' '--libdir=/usr/lib' '--localstatedir=/var/run/frr' '--enable-rpki' '--enable-vtysh' '--enable-multipath=64' '--enable-vty-group=frrvty' '--enable-user=frr' '--enable-group=frr' '--enable-pcre2posix' '--enable-scripting' 'CC=gcc' 'CXX=g++'

How to reproduce

Peer A, the frr config was added using the reloader script

 /usr/lib/frr/frr-reload.py --debug --reload --overwrite --stdout "$FILE_TO_RELOAD" 2>&1 

and the frrconfig

frr defaults traditional
hostname frr-k8s-control-plane
log file /etc/frr/frr.log
log timestamp precision 3
no ipv6 forwarding
service integrated-vtysh-config
!
debug bgp keepalives
debug bgp neighbor-events
debug bgp nht
debug bgp updates in
debug bgp updates out
debug bgp zebra
debug bgp bfd
debug bfd distributed
debug bfd peer
debug bfd zebra
debug bfd network
!
router bgp 64512
 no bgp ebgp-requires-policy
 no bgp hard-administrative-reset
 no bgp default ipv4-unicast
 no bgp graceful-restart notification
 no bgp network import-check
 neighbor 172.18.0.3 remote-as 4200000000
 neighbor 172.18.0.3 bfd
 !
 address-family ipv4 unicast
  network 5.5.5.5/32
  neighbor 172.18.0.3 activate
 exit-address-family
 !
 address-family ipv6 unicast
  neighbor 172.18.0.3 activate
 exit-address-family
exit

Peer B (config on the disk, ready before peer A starts)

router bgp 4200000000
 bgp router-id 172.18.0.3
 no bgp ebgp-requires-policy
 no bgp default ipv4-unicast
 no bgp network import-check
 neighbor 172.18.0.2 remote-as 64512
 neighbor 172.18.0.2 bfd
 neighbor 172.18.0.2 passive
 !
 address-family ipv4 unicast
  neighbor 172.18.0.2 activate
  neighbor 172.18.0.2 next-hop-self
 exit-address-family
exit
!
route-map RMAP permit 10
 set ipv6 next-hop prefer-global
exit
!
end

Expected behavior

I would expect a peering happening once, no dropped connection

Actual behavior

  Connections established 2; dropped 1
  Last reset 00:00:39,  Notification sent (Cease/BFD Down)
  External BGP neighbor may be up to 1 hops away.
####### Dumping file /etc/frr/frr.log
2024/11/04 12:42:22 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:22 MGMTD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:22 BGP: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:22 BFD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:22 STATIC: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:23 ZEBRA: [V98V0-MTWPF] client 50 says hello and bids fair to announce only bgp routes vrf=0
2024/11/04 12:42:25.332 MGMTD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.332 MGMTD: [G6NKK-8C6DV] end_config: VTY:0x7f9012c022e0, pending SET-CFG: 0
2024/11/04 12:42:25.332 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.332 ZEBRA: [G6NKK-8C6DV] end_config: VTY:0x7f6a60053500, pending SET-CFG: 0
2024/11/04 12:42:25.333 BGP: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.333 BGP: [G6NKK-8C6DV] end_config: VTY:0x7f0bdce5c480, pending SET-CFG: 0
2024/11/04 12:42:25.334 STATIC: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.334 STATIC: [G6NKK-8C6DV] end_config: VTY:0x7f96bdbbc380, pending SET-CFG: 0
2024/11/04 12:42:25.334 BFD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.334 BFD: [G6NKK-8C6DV] end_config: VTY:0x7f7830cb8050, pending SET-CFG: 0
2024/11/04 12:42:25.437 MGMTD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.437 MGMTD: [G6NKK-8C6DV] end_config: VTY:0x7f9012c02320, pending SET-CFG: 0
2024/11/04 12:42:25.438 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.438 ZEBRA: [G6NKK-8C6DV] end_config: VTY:0x7f6a60053520, pending SET-CFG: 0
2024/11/04 12:42:25.439 BGP: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.439 BGP: [G6NKK-8C6DV] end_config: VTY:0x7f0bdce5c4c0, pending SET-CFG: 0
2024/11/04 12:42:25.440 STATIC: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.440 STATIC: [G6NKK-8C6DV] end_config: VTY:0x7f96bdbbc3c0, pending SET-CFG: 0
2024/11/04 12:42:25.441 BFD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:25.441 BFD: [G6NKK-8C6DV] end_config: VTY:0x7f7830cb8150, pending SET-CFG: 0
2024/11/04 12:42:38.655 BFD: [YA0Q5-C0BPV] control-packet: no session found [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:39.605 BFD: [YA0Q5-C0BPV] control-packet: no session found [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:40.455 BFD: [YA0Q5-C0BPV] control-packet: no session found [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:41.435 BFD: [YA0Q5-C0BPV] control-packet: no session found [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:41.855 BGP: [Z1RP6-0X3QJ] Creating Default VRF, AS 64512
2024/11/04 12:42:41.862 BGP: [TTAN7-16X9N] dup addr detect enable max_moves 5 time 180 freeze disable freeze_time 0
2024/11/04 12:42:41.862 BGP: [ZZKY3-FX5JH] bgp_get: Registering BGP instance VRF default to zebra
2024/11/04 12:42:41.862 BGP: [TNK7N-FJF7K] Registering VRF 0
2024/11/04 12:42:41.863 BGP: [KGTKH-FVHEW] Rx Router Id update VRF 0 Id 172.18.0.2/32
2024/11/04 12:42:41.863 BGP: [WMCA1-27995] RID change : vrf VRF default(0), RTR ID 172.18.0.2
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 0 IF eth0
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF eth0 addr 172.18.0.2/16
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF eth0 addr fc00:f853:ccd:e793::2/64
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF eth0 addr fe80::42:acff:fe12:2/64
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 0 IF lo
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 0 IF veth472a42ad
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF veth472a42ad addr 10.244.0.1/32
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF veth472a42ad addr fe80::a881:68ff:fe3a:f15b/64
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 0 IF veth5034657b
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF veth5034657b addr 10.244.0.1/32
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF veth5034657b addr fe80::e870:a7ff:fef1:8d63/64
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 0 IF vetha4234df1
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF vetha4234df1 addr 10.244.0.1/32
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF vetha4234df1 addr fe80::7026:48ff:fea5:187a/64
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 0 IF vethb22a0778
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF vethb22a0778 addr 10.244.0.1/32
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 0 IF vethb22a0778 addr fe80::6404:e6ff:fef4:4dad/64
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 6 IF eth1
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 6 IF eth1 addr 172.31.0.2/16
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 6 IF eth1 addr fc00:f853:ccd:e799::2/64
2024/11/04 12:42:41.863 BGP: [GYPW0-GVZQ8] Rx Intf address add VRF 6 IF eth1 addr fe80::42:acff:fe1f:2/64
2024/11/04 12:42:41.863 BGP: [HKBB3-YX6A9] Rx Intf add VRF 6 IF red
2024/11/04 12:42:41.869 MGMTD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:41.869 MGMTD: [G6NKK-8C6DV] end_config: VTY:0x7f9012c02510, pending SET-CFG: 0
2024/11/04 12:42:41.870 BFD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:41.870 BFD: [G6NKK-8C6DV] end_config: VTY:0x7f7830cb8460, pending SET-CFG: 0
2024/11/04 12:42:41.870 STATIC: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:41.870 STATIC: [G6NKK-8C6DV] end_config: VTY:0x7f96bdbbc5a0, pending SET-CFG: 0
2024/11/04 12:42:41.871 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:41.871 ZEBRA: [G6NKK-8C6DV] end_config: VTY:0x7f6a60053620, pending SET-CFG: 0
2024/11/04 12:42:41.871 BFD: [MSVDW-Y8Z5Q] ptm-add-dest: register peer [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/04 12:42:41.871 BFD: [PSB4R-8T1TJ] session-new: mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default
2024/11/04 12:42:41.871 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 0.0.0.0/32 -> 172.18.0.3/32 VRF default(0) (CPI bit no): Down
2024/11/04 12:42:41.871 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Unknown -> Down
2024/11/04 12:42:41.871 BGP: [QFMSE-NPSNN] zclient_bfd_session_update:   sessions updated: 1
2024/11/04 12:42:41.877 BGP: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:41.877 BGP: [G6NKK-8C6DV] end_config: VTY:0x7f0bdce5c6a0, pending SET-CFG: 0
2024/11/04 12:42:42.001 BFD: [MSVDW-Y8Z5Q] ptm-add-dest: register peer [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/04 12:42:42.002 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 0.0.0.0/32 -> 172.18.0.3/32 VRF default(0) (CPI bit no): Down
2024/11/04 12:42:42.002 BGP: [QFMSE-NPSNN] zclient_bfd_session_update:   sessions updated: 0
2024/11/04 12:42:42.006 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:42.006 ZEBRA: [G6NKK-8C6DV] end_config: VTY:0x7f6a60053750, pending SET-CFG: 0
2024/11/04 12:42:42.006 MGMTD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:42.006 MGMTD: [G6NKK-8C6DV] end_config: VTY:0x7f9012c02550, pending SET-CFG: 0
2024/11/04 12:42:42.009 STATIC: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:42.009 STATIC: [G6NKK-8C6DV] end_config: VTY:0x7f96bdbbc5e0, pending SET-CFG: 0
2024/11/04 12:42:42.009 BFD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:42.009 BFD: [G6NKK-8C6DV] end_config: VTY:0x7f7830cb8690, pending SET-CFG: 0
2024/11/04 12:42:42.009 BGP: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/11/04 12:42:42.009 BGP: [G6NKK-8C6DV] end_config: VTY:0x7f0bdce5c750, pending SET-CFG: 0
2024/11/04 12:42:42.871 BFD: [J1C6V-VMRW5] state-change: [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default] init -> up
2024/11/04 12:42:42.872 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 0.0.0.0/32 -> 172.18.0.3/32 (interface eth0) VRF default(0) (CPI bit no): Up
2024/11/04 12:42:42.872 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Down -> Up
2024/11/04 12:42:42.872 BGP: [QFMSE-NPSNN] zclient_bfd_session_update:   sessions updated: 1
2024/11/04 12:42:43.009 BGP: [GYE1R-WA4DN] 172.18.0.3: released from config-pending hold
2024/11/04 12:42:43.009 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] BGP_Start (Idle->Connect), fd -1
2024/11/04 12:42:43.009 BGP: [W3X54-30BTE] Allocated bnc 172.18.0.3/32(0)(0)(VRF default) peer 0x7f0bdcda3710
2024/11/04 12:42:43.009 BGP: [HXW3G-K1M2A] sendmsg_zebra_rnh: sending cmd ZEBRA_NEXTHOP_REGISTER for 172.18.0.3/32 (vrf VRF default)
2024/11/04 12:42:43.009 BGP: [S7AHN-X0695] 172.18.0.3 [FSM] Waiting for NHT, no path to neighbor present
2024/11/04 12:42:43.009 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
2024/11/04 12:42:43.009 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd -1 went from Idle to Connect
2024/11/04 12:42:43.009 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] TCP_connection_open_failed (Connect->Active), fd -1
2024/11/04 12:42:43.009 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Active established_peers 0
2024/11/04 12:42:43.009 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd -1 went from Connect to Active
2024/11/04 12:42:43.009 BGP: [ZJYCS-KQYD6] VRF default(0): Rcvd NH update 172.18.0.0/16(0)(0) - metric 0/0 #nhops 1/0 flags Reg Conn 
2024/11/04 12:42:43.009 BGP: [HJ3XJ-8EHF0]     nhop via if 11 (0 labels)
2024/11/04 12:42:43.009 BGP: [K7JBS-J1T3G] NH update for 172.18.0.3/32(0)(0)(VRF default) - flags Valid Reg Conn  chgflags Changed - evaluate paths
2024/11/04 12:42:43.009 BGP: [MZNH7-XJV7F] evaluate_paths: Updating peer (172.18.0.3(VRF default)) status with NHT nexthops 1
2024/11/04 12:42:43.009 BGP: [W3632-R2FGS] parse nexthop update 172.18.0.0/16(0)(VRF default): bnc info not found for import check
2024/11/04 12:42:43.009 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] ConnectRetry_timer_expired (Active->Connect), fd -1
2024/11/04 12:42:43.009 BGP: [WNKP5-SN018] Found existing bnc 172.18.0.3/32(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x7f0bdcda3710
2024/11/04 12:42:43.009 BGP: [Z195V-FNKRK] 172.18.0.3 [Event] Connect start to 172.18.0.3 fd 22
2024/11/04 12:42:43.010 BGP: [G0837-S7QES] 172.18.0.3 [FSM] Non blocking connect waiting result, fd 22
2024/11/04 12:42:43.010 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
2024/11/04 12:42:43.010 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from Active to Connect
2024/11/04 12:42:43.010 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] TCP_connection_open (Connect->OpenSent), fd 22
2024/11/04 12:42:43.010 BGP: [RWZTG-AA74G] 172.18.0.3 open active, local address 172.18.0.2
2024/11/04 12:42:43.010 BGP: [XKJ09-9VTZ7] 172.18.0.3 Sending hostname cap with hn = frr-k8s-control-plane, dn = (null)
2024/11/04 12:42:43.010 BGP: [JFFAN-DEGED] 172.18.0.3 sending OPEN, version 4, my as 64512, holdtime 180, id 172.18.0.2
2024/11/04 12:42:43.010 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenSent established_peers 0
2024/11/04 12:42:43.010 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from Connect to OpenSent
2024/11/04 12:42:43.010 BGP: [WNM1E-D314G] 172.18.0.3 rcv OPEN, version 4, remote-as (in open) 23456, holdtime 180, id 172.18.0.3
2024/11/04 12:42:43.010 BGP: [QG29C-5TSVS] 172.18.0.3 rcv OPEN w/ OPTION parameter len: 71
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has MultiProtocol Extensions capability (1), length 4
2024/11/04 12:42:43.010 BGP: [VXVH9-3MXR0] 172.18.0.3 OPEN has MultiProtocol Extensions capability for afi/safi: IPv4/unicast
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Route Refresh capability (2), length 0
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Enhanced Route Refresh capability (70), length 0
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has 4-octet AS number capability (65), length 4
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has BGP Extended Message capability (6), length 0
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has AddPath capability (69), length 4
2024/11/04 12:42:43.010 BGP: [SBSKM-G6QBW] 172.18.0.3 OPEN has AddPath capability for afi/safi: IPv4/unicast, receive
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 16
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has FQDN capability (73), length 14
2024/11/04 12:42:43.010 BGP: [MR6EG-NS6N7] 172.18.0.3 received hostname cbe0c9228b55, domainname (null)
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 4
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Graceful Restart capability (64), length 2
2024/11/04 12:42:43.010 BGP: [KDY57-R1CN0] 172.18.0.3 Peer has restarted. Restart Time: 120, N-bit set: yes
2024/11/04 12:42:43.010 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 9
2024/11/04 12:42:43.010 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Long-lived BGP Graceful Restart capability (71), length 7
2024/11/04 12:42:43.010 BGP: [ZVHDJ-HW40B] 172.18.0.3 Addr-family IPv4/unicast(afi/safi) not enabled. Ignore the Long-lived Graceful Restart capability
2024/11/04 12:42:43.010 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm), fd 22
2024/11/04 12:42:43.010 BGP: [HRDT0-0DPQ7] 172.18.0.3 sending KEEPALIVE
2024/11/04 12:42:43.010 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenConfirm established_peers 0
2024/11/04 12:42:43.010 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from OpenSent to OpenConfirm
2024/11/04 12:42:43.010 BGP: [X61A3-E95TJ] 172.18.0.3 KEEPALIVE rcvd
2024/11/04 12:42:43.010 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established), fd 22
2024/11/04 12:42:43.010 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Established established_peers 1
2024/11/04 12:42:43.010 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from OpenConfirm to Established
2024/11/04 12:42:43.010 BGP: [M452J-37M96] 172.18.0.3(Unknown) Update Group Hash: sort: 2 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2024/11/04 12:42:43.010 BGP: [S4FCZ-AFJ7Z] 172.18.0.3(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2024/11/04 12:42:43.010 BGP: [Z8Q37-65KK3] 172.18.0.3(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: (NONE) rmap out: (NONE)
2024/11/04 12:42:43.010 BGP: [SM2F3-HRYKP] 172.18.0.3(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2024/11/04 12:42:43.010 BGP: [V8B3M-T6VFC] 172.18.0.3(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2024/11/04 12:42:43.010 BGP: [Y5EX3-GHT5V] 172.18.0.3(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2024/11/04 12:42:43.010 BGP: [X19K7-9V4K2] 172.18.0.3(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2024/11/04 12:42:43.010 BGP: [SQ314-QBJCR] 172.18.0.3(Unknown) Update Group Hash key: 3043862662
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 1 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 2 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 3 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 4 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 5 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 6 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 7 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 1 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 2 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 3 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 4 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 5 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 6 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 7 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 1 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 2 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 3 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 4 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 5 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 6 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 7 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:43.010 BGP: [VQY9X-CQZKG] bgp_peer_bfd_update_source: address [0.0.0.0->172.18.0.3] to [172.18.0.2->172.18.0.3]
2024/11/04 12:42:43.010 BGP: [X8BD9-8RKN4] bgp_peer_bfd_update_source: interface none to eth0
2024/11/04 12:42:43.010 BFD: [MSVDW-Y8Z5Q] ptm-del-dest: deregister peer [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/04 12:42:43.010 BFD: [NYF5K-SE3NS] ptm-del-session: [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default] refcount=0
2024/11/04 12:42:43.010 BFD: [NW21R-MRYNT] session-delete: mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default
2024/11/04 12:42:43.010 BGP: [P3D3N-3277A] 172.18.0.3 [FSM] Timer (routeadv timer expire)
2024/11/04 12:42:43.010 BFD: [YA0Q5-C0BPV] control-packet: no session found [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:43.010 BFD: [MSVDW-Y8Z5Q] ptm-add-dest: register peer [mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/04 12:42:43.011 BFD: [PSB4R-8T1TJ] session-new: mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default ifname:eth0
2024/11/04 12:42:43.011 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 172.18.0.2/32 -> 172.18.0.3/32 (interface eth0) VRF default(0) (CPI bit no): Down
2024/11/04 12:42:43.011 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Up -> Down
2024/11/04 12:42:43.011 BGP: [HZN6M-XRM1G] %NOTIFICATION: sent to neighbor 172.18.0.3 6/10 (Cease/BFD Down) 0 bytes 
2024/11/04 12:42:43.011 BGP: [QFMSE-NPSNN] zclient_bfd_session_update:   sessions updated: 1
2024/11/04 12:42:43.011 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] BGP_Stop (Established->Clearing), fd 22
2024/11/04 12:42:43.011 BGP: [GYHDK-S2BXG] peer 172.18.0.3, EOR_required 0
2024/11/04 12:42:43.011 BGP: [M452J-37M96] 172.18.0.3(Unknown) Update Group Hash: sort: 2 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2024/11/04 12:42:43.011 BGP: [S4FCZ-AFJ7Z] 172.18.0.3(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2024/11/04 12:42:43.011 BGP: [Z8Q37-65KK3] 172.18.0.3(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: (NONE) rmap out: (NONE)
2024/11/04 12:42:43.011 BGP: [SM2F3-HRYKP] 172.18.0.3(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2024/11/04 12:42:43.011 BGP: [V8B3M-T6VFC] 172.18.0.3(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2024/11/04 12:42:43.011 BGP: [Y5EX3-GHT5V] 172.18.0.3(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2024/11/04 12:42:43.011 BGP: [X19K7-9V4K2] 172.18.0.3(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2024/11/04 12:42:43.011 BGP: [SQ314-QBJCR] 172.18.0.3(Unknown) Update Group Hash key: 3043862662
2024/11/04 12:42:43.011 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Clearing established_peers 0
2024/11/04 12:42:43.011 BGP: [WMCA1-27995] RID change : vrf VRF default(0), RTR ID 172.18.0.2
2024/11/04 12:42:43.011 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd -1 went from Established to Clearing
2024/11/04 12:42:43.011 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] Clearing_Completed (Clearing->Idle), fd -1
2024/11/04 12:42:43.011 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Idle established_peers 0
2024/11/04 12:42:43.011 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd -1 went from Clearing to Idle
2024/11/04 12:42:44.011 BFD: [YA0Q5-C0BPV] control-packet: 'remote discriminator' is zero, not overridden [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:44.781 BFD: [YA0Q5-C0BPV] control-packet: 'remote discriminator' is zero, not overridden [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:45.011 BGP: [ZQTB5-H8522] 172.18.0.3 [FSM] Timer (start timer expire).
2024/11/04 12:42:45.011 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] BGP_Start (Idle->Connect), fd -1
2024/11/04 12:42:45.011 BGP: [WNKP5-SN018] Found existing bnc 172.18.0.3/32(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x7f0bdcda3710
2024/11/04 12:42:45.011 BGP: [Z195V-FNKRK] 172.18.0.3 [Event] Connect start to 172.18.0.3 fd 22
2024/11/04 12:42:45.011 BGP: [G0837-S7QES] 172.18.0.3 [FSM] Non blocking connect waiting result, fd 22
2024/11/04 12:42:45.011 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
2024/11/04 12:42:45.011 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from Idle to Connect
2024/11/04 12:42:45.012 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] TCP_connection_open (Connect->OpenSent), fd 22
2024/11/04 12:42:45.012 BGP: [RWZTG-AA74G] 172.18.0.3 open active, local address 172.18.0.2
2024/11/04 12:42:45.012 BGP: [XKJ09-9VTZ7] 172.18.0.3 Sending hostname cap with hn = frr-k8s-control-plane, dn = (null)
2024/11/04 12:42:45.012 BGP: [JFFAN-DEGED] 172.18.0.3 sending OPEN, version 4, my as 64512, holdtime 180, id 172.18.0.2
2024/11/04 12:42:45.012 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenSent established_peers 0
2024/11/04 12:42:45.012 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from Connect to OpenSent
2024/11/04 12:42:45.013 BGP: [WNM1E-D314G] 172.18.0.3 rcv OPEN, version 4, remote-as (in open) 23456, holdtime 180, id 172.18.0.3
2024/11/04 12:42:45.013 BGP: [QG29C-5TSVS] 172.18.0.3 rcv OPEN w/ OPTION parameter len: 71
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has MultiProtocol Extensions capability (1), length 4
2024/11/04 12:42:45.013 BGP: [VXVH9-3MXR0] 172.18.0.3 OPEN has MultiProtocol Extensions capability for afi/safi: IPv4/unicast
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Route Refresh capability (2), length 0
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Enhanced Route Refresh capability (70), length 0
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has 4-octet AS number capability (65), length 4
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has BGP Extended Message capability (6), length 0
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has AddPath capability (69), length 4
2024/11/04 12:42:45.013 BGP: [SBSKM-G6QBW] 172.18.0.3 OPEN has AddPath capability for afi/safi: IPv4/unicast, receive
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 16
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has FQDN capability (73), length 14
2024/11/04 12:42:45.013 BGP: [MR6EG-NS6N7] 172.18.0.3 received hostname cbe0c9228b55, domainname (null)
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 4
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Graceful Restart capability (64), length 2
2024/11/04 12:42:45.013 BGP: [KDY57-R1CN0] 172.18.0.3 Peer has restarted. Restart Time: 120, N-bit set: yes
2024/11/04 12:42:45.013 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 9
2024/11/04 12:42:45.013 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Long-lived BGP Graceful Restart capability (71), length 7
2024/11/04 12:42:45.013 BGP: [ZVHDJ-HW40B] 172.18.0.3 Addr-family IPv4/unicast(afi/safi) not enabled. Ignore the Long-lived Graceful Restart capability
2024/11/04 12:42:45.013 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm), fd 22
2024/11/04 12:42:45.013 BGP: [HRDT0-0DPQ7] 172.18.0.3 sending KEEPALIVE
2024/11/04 12:42:45.013 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenConfirm established_peers 0
2024/11/04 12:42:45.013 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from OpenSent to OpenConfirm
2024/11/04 12:42:45.014 BGP: [X61A3-E95TJ] 172.18.0.3 KEEPALIVE rcvd
2024/11/04 12:42:45.014 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established), fd 22
2024/11/04 12:42:45.014 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Established established_peers 1
2024/11/04 12:42:45.014 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from OpenConfirm to Established
2024/11/04 12:42:45.014 BGP: [M452J-37M96] 172.18.0.3(Unknown) Update Group Hash: sort: 2 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2024/11/04 12:42:45.014 BGP: [S4FCZ-AFJ7Z] 172.18.0.3(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2024/11/04 12:42:45.014 BGP: [Z8Q37-65KK3] 172.18.0.3(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: (NONE) rmap out: (NONE)
2024/11/04 12:42:45.014 BGP: [SM2F3-HRYKP] 172.18.0.3(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2024/11/04 12:42:45.014 BGP: [V8B3M-T6VFC] 172.18.0.3(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2024/11/04 12:42:45.014 BGP: [Y5EX3-GHT5V] 172.18.0.3(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2024/11/04 12:42:45.014 BGP: [X19K7-9V4K2] 172.18.0.3(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2024/11/04 12:42:45.014 BGP: [SQ314-QBJCR] 172.18.0.3(Unknown) Update Group Hash key: 3043862662
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 1 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 2 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 3 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 4 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 5 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 6 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 7 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 1 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 2 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 3 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 4 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 5 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 6 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 2 safi: 7 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 1 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 2 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 3 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 4 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 5 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 6 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.014 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 3 safi: 7 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:45.015 BGP: [P3D3N-3277A] 172.18.0.3 [FSM] Timer (routeadv timer expire)
2024/11/04 12:42:45.532 BFD: [YA0Q5-C0BPV] control-packet: 'remote discriminator' is zero, not overridden [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:46.011 BFD: [SEY1D-NT8EQ] state-change: [mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default ifname:eth0] init -> down reason:control-expired
2024/11/04 12:42:46.064 BGP: [K9G3A-CJ5QQ] u2:s2 announcing routes upon coalesce timer expiry(1050 ms)
2024/11/04 12:42:46.064 BGP: [T5JFA-13199] subgroup_process_announce_selected: p=5.5.5.5/32, selected=0x7f0bdcf09ea0
2024/11/04 12:42:46.064 BGP: [P3D3N-3277A] 172.18.0.3 [FSM] Timer (routeadv timer expire)
2024/11/04 12:42:46.064 BGP: [TN0HX-6G1RR] u2:s2 send UPDATE w/ attr: nexthop 0.0.0.0, metric 0, path 
2024/11/04 12:42:46.064 BGP: [HVRWP-5R9NQ] u2:s2 send UPDATE 5.5.5.5/32 IPv4 unicast
2024/11/04 12:42:46.064 BGP: [WEV7K-2GAQ5] u2:s2 send UPDATE len 56 (max message len: 65535) numpfx 1
2024/11/04 12:42:46.064 BGP: [MBFVT-8GSC6] u2:s2 172.18.0.3 send UPDATE w/ nexthop 172.18.0.2
2024/11/04 12:42:46.064 BGP: [ZP3RE-J4Q8C] send End-of-RIB for IPv4 Unicast to 172.18.0.3
2024/11/04 12:42:46.116 BGP: [T5AAP-5GA85] 172.18.0.3(cbe0c9228b55) rcvd UPDATE w/ attr: nexthop 172.18.0.3, origin i, path 4200000000 64512
2024/11/04 12:42:46.116 BGP: [PCFFM-WMARW] 172.18.0.3(cbe0c9228b55) rcvd UPDATE wlen 0 attrlen 25 alen 5
2024/11/04 12:42:46.116 BGP: [RZMGQ-A03CG] 172.18.0.3(cbe0c9228b55) rcvd UPDATE about 5.5.5.5/32 IPv4 unicast -- DENIED due to: as-path contains our own AS;
2024/11/04 12:42:46.116 BGP: [PCFFM-WMARW] 172.18.0.3(cbe0c9228b55) rcvd UPDATE wlen 0 attrlen 0 alen 0
2024/11/04 12:42:46.116 BGP: [WT375-N4KPV] EOR REQ 0, EOR RCV 0
2024/11/04 12:42:46.116 BGP: [G3V0W-FSWRC] bgp_best_path_select_defer: processing route for IPv4 Unicast : cnt 0
2024/11/04 12:42:46.116 BGP: [RF7DJ-R0V7Z] bgp_zebra_update: VRF default afi: 1 safi: 1 Command ZEBRA_CLIENT_ROUTE_UPDATE_COMPLETE
2024/11/04 12:42:46.116 BGP: [M59KS-A3ZXZ] bgp_update_receive: rcvd End-of-RIB for IPv4 Unicast from 172.18.0.3 in vrf default
2024/11/04 12:42:46.502 BFD: [J1C6V-VMRW5] state-change: [mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default ifname:eth0] down -> up
2024/11/04 12:42:46.502 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 172.18.0.2/32 -> 172.18.0.3/32 (interface eth0) VRF default(0) (CPI bit no): Up
2024/11/04 12:42:46.502 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Down -> Up
2024/11/04 12:42:46.502 BGP: [QFMSE-NPSNN] zclient_bfd_session_update:   sessions updated: 1
####### BGP Neighbors
BGP neighbor is 172.18.0.3, remote AS 4200000000, local AS 64512, external link
  Local Role: undefined
  Remote Role: undefined
Hostname: cbe0c9228b55
  BGP version 4, remote router ID 172.18.0.3, local router ID 172.18.0.2
  BGP state = Established, up for 00:00:37
  Last read 00:00:36, Last write 00:00:36
  Hold time is 180 seconds, keepalive interval is 60 seconds
  Configured hold time is 180 seconds, keepalive interval is 60 seconds
  Configured tcp-mss is 0, synced tcp-mss is 1448
  Configured conditional advertisements interval is 60 seconds
  Neighbor capabilities:
    4 Byte AS: advertised and received
    Extended Message: advertised and received
    AddPath:
      IPv4 Unicast: RX advertised and received
      IPv6 Unicast: RX advertised
    Long-lived Graceful Restart: advertised and received
      Address families by peer:
    Route refresh: advertised and received
    Enhanced Route Refresh: advertised and received
    Address Family IPv4 Unicast: advertised and received
    Address Family IPv6 Unicast: advertised
    Hostname Capability: advertised (name: frr-k8s-control-plane,domain name: n/a) received (name: cbe0c9228b55,domain name: n/a)
    Version Capability: not advertised not received
    Graceful Restart Capability: advertised and received
      Remote Restart timer is 120 seconds
      Address families by peer:
        none
  Graceful restart information:
    End-of-RIB send: IPv4 Unicast
    End-of-RIB received: IPv4 Unicast
    Local GR Mode: Helper*

    Remote GR Mode: Helper

    R bit: True
    N bit: True
    Timers:
      Configured Restart Time(sec): 120
      Received Restart Time(sec): 120
      Configured LLGR Stale Path Time(sec): 0
    IPv4 Unicast:
      F bit: False
      End-of-RIB sent: Yes
      End-of-RIB sent after update: Yes
      End-of-RIB received: Yes
      Timers:
        Configured Stale Path Time(sec): 360
        LLGR Stale Path Time(sec): 0
    IPv6 Unicast:
      F bit: False
      End-of-RIB sent: No
      End-of-RIB sent after update: No
      End-of-RIB received: No
      Timers:
        Configured Stale Path Time(sec): 360
        LLGR Stale Path Time(sec): 0
  Message statistics:
    Inq depth is 0
    Outq depth is 0
                         Sent       Rcvd
    Opens:                  2          2
    Notifications:          1          0
    Updates:                2          2
    Keepalives:             2          2
    Route Refresh:          0          0
    Capability:             0          0
    Total:                  7          6
  Minimum time between advertisement runs is 0 seconds

 For address family: IPv4 Unicast
  Update group 2, subgroup 2
  Packet Queue length 0
  Community attribute sent to this neighbor(all)
  0 accepted prefixes

 For address family: IPv6 Unicast
  Not part of any update group
  Community attribute sent to this neighbor(all)
  0 accepted prefixes

  Connections established 2; dropped 1
  Last reset 00:00:39,  Notification sent (Cease/BFD Down)
  External BGP neighbor may be up to 1 hops away.
Local host: 172.18.0.2, Local port: 49492
Foreign host: 172.18.0.3, Foreign port: 179
Nexthop: 172.18.0.2
Nexthop global: fc00:f853:ccd:e793::2
Nexthop local: fe80::42:acff:fe12:2
BGP connection: shared network
BGP Connect Retry Timer in Seconds: 120
Estimated round trip time: 0 ms
Read thread: on  Write thread: on  FD used: 22

  BFD: Type: single hop
  Detect Multiplier: 3, Min Rx interval: 300, Min Tx interval: 300
  Status: Up, Last update: 0:00:00:35


####### BFD Peers
BFD Peers:
	peer 172.18.0.3 local-address 172.18.0.2 vrf default interface eth0
		ID: 3135967594
		Remote ID: 286069549
		Active mode
		Status: up
		Uptime: 35 second(s)
		Diagnostics: ok
		Remote diagnostics: ok
		Peer Type: dynamic
		RTT min/avg/max: 0/0/0 usec
		Local timers:
			Detect-multiplier: 3
			Receive interval: 300ms
			Transmission interval: 300ms
			Echo receive interval: 50ms
			Echo transmission interval: disabled
		Remote timers:
			Detect-multiplier: 3
			Receive interval: 300ms
			Transmission interval: 300ms
			Echo receive interval: 50ms

Additional context

The logs indicate that once first BGP peering takes place, the bfd_session must be updated which result into terminating peering and re-establishe.

2024/11/04 12:42:43.010 BGP: [VQY9X-CQZKG] bgp_peer_bfd_update_source: address [0.0.0.0->172.18.0.3] to [172.18.0.2->172.18.0.3]
2024/11/04 12:42:43.010 BGP: [X8BD9-8RKN4] bgp_peer_bfd_update_source: interface none to eth0
2024/11/04 12:42:43.010 BFD: [MSVDW-Y8Z5Q] ptm-del-dest: deregister peer [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/04 12:42:43.010 BFD: [NYF5K-SE3NS] ptm-del-session: [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default] refcount=0
2024/11/04 12:42:43.010 BFD: [NW21R-MRYNT] session-delete: mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default
2024/11/04 12:42:43.010 BGP: [P3D3N-3277A] 172.18.0.3 [FSM] Timer (routeadv timer expire)
2024/11/04 12:42:43.010 BFD: [YA0Q5-C0BPV] control-packet: no session found [mhop:no peer:172.18.0.3 local:172.18.0.2 port:11]
2024/11/04 12:42:43.010 BFD: [MSVDW-Y8Z5Q] ptm-add-dest: register peer [mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/04 12:42:43.011 BFD: [PSB4R-8T1TJ] session-new: mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default ifname:eth0
2024/11/04 12:42:43.011 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 172.18.0.2/32 -> 172.18.0.3/32 (interface eth0) VRF default(0) (CPI bit no): Down
2024/11/04 12:42:43.011 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Up -> Down
2024/11/04 12:42:43.011 BGP: [HZN6M-XRM1G] %NOTIFICATION: sent to neighbor 172.18.0.3 6/10 (Cease/BFD Down) 0 bytes 
2024/11/04 12:42:43.011 BGP: [QFMSE-NPSNN] zclient_bfd_session_update:   sessions updated: 1
2024/11/04 12:42:43.011 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] BGP_Stop (Established->Clearing), fd 22

Checklist

  • I have searched the open issues for this bug.
  • I have not included sensitive information in this report.

Do you see any wrong-doing when comes to configuration? Is there a way to configure it differently to avoid that from happening? Thank you

@karampok karampok added the triage Needs further investigation label Nov 4, 2024
@ton31337
Copy link
Member

ton31337 commented Nov 4, 2024

This is because most likely BFD was not established and BGP sent a notification: %NOTIFICATION: sent to neighbor 172.18.0.3 6/10 (Cease/BFD Down), that triggered session reset.

@ton31337 ton31337 added the bgp label Nov 4, 2024
@ton31337 ton31337 self-assigned this Nov 4, 2024
@karampok
Copy link
Author

karampok commented Nov 4, 2024

Hi thank you for the reply. I have been observing the pcap, and I can see that the BFD was up before the OPEN messages.

image

Do you see a way to configure the FRRs to avoid that session reset? Or is that expected?

@ton31337
Copy link
Member

ton31337 commented Nov 4, 2024

According to this:

zclient_bfd_session_update: 172.18.0.2/32 -> 172.18.0.3/32 (interface eth0) VRF default(0) (CPI bit no): Down

This is expected.

@karampok
Copy link
Author

karampok commented Nov 4, 2024

So, should I conclude that there is no configuration that will set up BGP with BFD peering and will avoid this behavior? Is that accurate?

(Context: I am trying graceful restart, and that behavior at the end removes the routes that GR stales)

@ton31337
Copy link
Member

ton31337 commented Nov 4, 2024

Either way, the BFD notification event triggers a notification (With GR notification support which is disabled in your case too), where the behavior is the same (the RFC (https://datatracker.ietf.org/doc/html/rfc9384) defines it as a HARD RESET).

@karampok
Copy link
Author

karampok commented Nov 5, 2024

Can you elaborate? I am no sure I understand what you suggest.

BFD notification triggers BGP Cease notification is ok. I try to understand why there is a BFD down event happens in the first place and how to avoid it.

In other words, now what is happening : (I think)

  1. bfd session established
  2. bgp session established
  3. bgp updates the bgp_session (local-address,interface)
  4. due to bfd update session there is a BFD down event
  5. bgp session is cleared
  6. bgp session is up

Is there a way to delay bfd (after BGP) or configure it in way that no update is needed?

Thanks

@ton31337
Copy link
Member

ton31337 commented Nov 5, 2024

Can you try with delay OPEN? E.g. neighbor 127.0.0.1 timers delayopen 10.

@karampok
Copy link
Author

karampok commented Nov 5, 2024

I have alreay tried that option, I see not difference. I can re-try to be sure.

Why do you suggest that? I mean delayopen delays the OPEN message so when you have both active BGP peers will avoid trying open at the same time the peering or is it related to bfd?

@ton31337
Copy link
Member

ton31337 commented Nov 5, 2024

I meant that with delay OPEN we give some time for BFD to be UP before establishing the BGP session.

@karampok
Copy link
Author

karampok commented Nov 5, 2024

I will re-try ok and I will come back.

But given those logs

2024/11/04 12:42:43.010 BGP: [VQY9X-CQZKG] bgp_peer_bfd_update_source: address [0.0.0.0->172.18.0.3] to [172.18.0.2->172.18.0.3]
2024/11/04 12:42:43.010 BGP: [X8BD9-8RKN4] bgp_peer_bfd_update_source: interface none to eth0

I think the problem is that bfd is up with not the right local-address/interface and BGP is correcting it (source)

@ton31337
Copy link
Member

ton31337 commented Nov 5, 2024

@karampok could you check this fix #17344? At least with this I can't reproduce the issue.

@ton31337 ton31337 removed the triage Needs further investigation label Nov 5, 2024
@karampok
Copy link
Author

karampok commented Nov 5, 2024

thanks I have to check how to pack a commit into an image first so I can try it.

Until then here are the logs with delayopen to 240 seconds

2024/11/05 13:56:17.257 BFD: [J1C6V-VMRW5] state-change: [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default] down -> up
2024/11/05 13:56:17.257 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 0.0.0.0/32 -> 172.18.0.3/32 (interface eth0) VRF default(0) (CPI bit no): Up
2024/11/05 13:56:17.257 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Down -> Up
2024/11/05 13:56:17.257 BGP: [QFMSE-NPSNN] zclient_bfd_session_update:   sessions updated: 1
2024/11/05 13:56:17.399 BGP: [GYE1R-WA4DN] 172.18.0.3: released from config-pending hold
2024/11/05 13:56:17.399 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] BGP_Start (Idle->Connect), fd -1
2024/11/05 13:56:17.399 BGP: [W3X54-30BTE] Allocated bnc 172.18.0.3/32(0)(0)(VRF default) peer 0x7f626e816710
2024/11/05 13:56:17.399 BGP: [HXW3G-K1M2A] sendmsg_zebra_rnh: sending cmd ZEBRA_NEXTHOP_REGISTER for 172.18.0.3/32 (vrf VRF default)
2024/11/05 13:56:17.399 BGP: [S7AHN-X0695] 172.18.0.3 [FSM] Waiting for NHT, no path to neighbor present
2024/11/05 13:56:17.399 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
2024/11/05 13:56:17.399 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd -1 went from Idle to Connect
2024/11/05 13:56:17.400 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] TCP_connection_open_failed (Connect->Active), fd -1
2024/11/05 13:56:17.400 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Active established_peers 0
2024/11/05 13:56:17.400 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd -1 went from Connect to Active
2024/11/05 13:56:17.400 BGP: [ZJYCS-KQYD6] VRF default(0): Rcvd NH update 172.18.0.0/16(0)(0) - metric 0/0 #nhops 1/0 flags Reg Conn 
2024/11/05 13:56:17.400 BGP: [HJ3XJ-8EHF0]     nhop via if 467 (0 labels)
2024/11/05 13:56:17.400 BGP: [K7JBS-J1T3G] NH update for 172.18.0.3/32(0)(0)(VRF default) - flags Valid Reg Conn  chgflags Changed - evaluate paths
2024/11/05 13:56:17.400 BGP: [MZNH7-XJV7F] evaluate_paths: Updating peer (172.18.0.3(VRF default)) status with NHT nexthops 1
2024/11/05 13:56:17.400 BGP: [W3632-R2FGS] parse nexthop update 172.18.0.0/16(0)(VRF default): bnc info not found for import check
2024/11/05 13:56:17.400 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] ConnectRetry_timer_expired (Active->Connect), fd -1
2024/11/05 13:56:17.400 BGP: [WNKP5-SN018] Found existing bnc 172.18.0.3/32(0)(VRF default) flags 0xf ifindex 0 #paths 0 peer 0x7f626e816710
2024/11/05 13:56:17.400 BGP: [Z195V-FNKRK] 172.18.0.3 [Event] Connect start to 172.18.0.3 fd 22
2024/11/05 13:56:17.400 BGP: [G0837-S7QES] 172.18.0.3 [FSM] Non blocking connect waiting result, fd 22
2024/11/05 13:56:17.400 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Connect established_peers 0
2024/11/05 13:56:17.400 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from Active to Connect
2024/11/05 13:56:17.400 BGP: [RWZTG-AA74G] 172.18.0.3 open active, local address 172.18.0.2
2024/11/05 13:56:17.400 BGP: [Z7X1S-E98T0] 172.18.0.3 [FSM] BGP OPEN message delayed for 240 seconds
2024/11/05 14:00:17.400 BGP: [XC901-RSZMG] 172.18.0.3 [FSM] Timer (DelayOpentimer expire)
2024/11/05 14:00:17.400 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] DelayOpen_timer_expired (Connect->OpenSent), fd 22
2024/11/05 14:00:17.400 BGP: [XKJ09-9VTZ7] 172.18.0.3 Sending hostname cap with hn = frr-k8s-control-plane, dn = (null)
2024/11/05 14:00:17.400 BGP: [JFFAN-DEGED] 172.18.0.3 sending OPEN, version 4, my as 64512, holdtime 180, id 172.18.0.2
2024/11/05 14:00:17.400 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenSent established_peers 0
2024/11/05 14:00:17.400 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from Connect to OpenSent
2024/11/05 14:00:17.401 BGP: [WNM1E-D314G] 172.18.0.3 rcv OPEN, version 4, remote-as (in open) 23456, holdtime 180, id 172.18.0.3
2024/11/05 14:00:17.401 BGP: [QG29C-5TSVS] 172.18.0.3 rcv OPEN w/ OPTION parameter len: 71
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has MultiProtocol Extensions capability (1), length 4
2024/11/05 14:00:17.401 BGP: [VXVH9-3MXR0] 172.18.0.3 OPEN has MultiProtocol Extensions capability for afi/safi: IPv4/unicast
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Route Refresh capability (2), length 0
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Enhanced Route Refresh capability (70), length 0
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has 4-octet AS number capability (65), length 4
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has BGP Extended Message capability (6), length 0
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has AddPath capability (69), length 4
2024/11/05 14:00:17.401 BGP: [SBSKM-G6QBW] 172.18.0.3 OPEN has AddPath capability for afi/safi: IPv4/unicast, receive
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 16
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has FQDN capability (73), length 14
2024/11/05 14:00:17.401 BGP: [MR6EG-NS6N7] 172.18.0.3 received hostname ce793246300a, domainname (null)
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 4
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Graceful Restart capability (64), length 2
2024/11/05 14:00:17.401 BGP: [KDY57-R1CN0] 172.18.0.3 Peer has not restarted. Restart Time: 120, N-bit set: yes
2024/11/05 14:00:17.401 BGP: [NVZPF-5ST3B] 172.18.0.3 rcvd OPEN w/ optional parameter type 2 (Capability) len 9
2024/11/05 14:00:17.401 BGP: [SCW43-WN4M1] 172.18.0.3 OPEN has Long-lived BGP Graceful Restart capability (71), length 7
2024/11/05 14:00:17.401 BGP: [ZVHDJ-HW40B] 172.18.0.3 Addr-family IPv4/unicast(afi/safi) not enabled. Ignore the Long-lived Graceful Restart capability
2024/11/05 14:00:17.401 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] Receive_OPEN_message (OpenSent->OpenConfirm), fd 22
2024/11/05 14:00:17.401 BGP: [HRDT0-0DPQ7] 172.18.0.3 sending KEEPALIVE
2024/11/05 14:00:17.401 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: OpenConfirm established_peers 0
2024/11/05 14:00:17.401 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from OpenSent to OpenConfirm
2024/11/05 14:00:17.401 BGP: [X61A3-E95TJ] 172.18.0.3 KEEPALIVE rcvd
2024/11/05 14:00:17.401 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] Receive_KEEPALIVE_message (OpenConfirm->Established), fd 22
2024/11/05 14:00:17.401 BGP: [T91AW-FGMHW] bgp_fsm_change_status : vrf default(0), Status: Established established_peers 1
2024/11/05 14:00:17.401 BGP: [HKWM3-ZC5QP] 172.18.0.3 fd 22 went from OpenConfirm to Established
2024/11/05 14:00:17.401 BGP: [M452J-37M96] 172.18.0.3(Unknown) Update Group Hash: sort: 2 UpdGrpFlags: 0 UpdGrpAFFlags: 16777219
2024/11/05 14:00:17.401 BGP: [S4FCZ-AFJ7Z] 172.18.0.3(Unknown) Update Group Hash: addpath: 4 UpdGrpCapFlag: 256 UpdGrpCapAFFlag: 2048 route_adv: 0 change local as: 0, as_path_loop_detection: 0
2024/11/05 14:00:17.401 BGP: [Z8Q37-65KK3] 172.18.0.3(Unknown) Update Group Hash: max packet size: 65535 pmax_out: 0 Peer Group: (NONE) rmap out: (NONE)
2024/11/05 14:00:17.401 BGP: [SM2F3-HRYKP] 172.18.0.3(Unknown) Update Group Hash: dlist out: (NONE) plist out: (NONE) aslist out: (NONE) usmap out: (NONE) advmap: (NONE) 0
2024/11/05 14:00:17.401 BGP: [V8B3M-T6VFC] 172.18.0.3(Unknown) Update Group Hash: default rmap: (NONE) shared network and afi active network: 0
2024/11/05 14:00:17.401 BGP: [Y5EX3-GHT5V] 172.18.0.3(Unknown) Update Group Hash: Lonesoul: 0 ORF prefix: 0 max prefix out: 0
2024/11/05 14:00:17.401 BGP: [X19K7-9V4K2] 172.18.0.3(Unknown) Update Group Hash: local role: 255 AIGP: 0 SOO: (NONE)
2024/11/05 14:00:17.401 BGP: [SQ314-QBJCR] 172.18.0.3(Unknown) Update Group Hash key: 3043862662
2024/11/05 14:00:17.401 BGP: [VQY9X-CQZKG] bgp_peer_bfd_update_source: address [0.0.0.0->172.18.0.3] to [172.18.0.2->172.18.0.3]
2024/11/05 14:00:17.401 BGP: [X8BD9-8RKN4] bgp_peer_bfd_update_source: interface none to eth0
2024/11/05 14:00:17.401 BGP: [P3D3N-3277A] 172.18.0.3 [FSM] Timer (routeadv timer expire)
2024/11/05 14:00:17.401 BFD: [MSVDW-Y8Z5Q] ptm-del-dest: deregister peer [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/05 14:00:17.401 BFD: [NYF5K-SE3NS] ptm-del-session: [mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default] refcount=0
2024/11/05 14:00:17.401 BFD: [NW21R-MRYNT] session-delete: mhop:no peer:172.18.0.3 local:0.0.0.0 vrf:default
2024/11/05 14:00:17.401 BFD: [YA0Q5-C0BPV] control-packet: no session found [mhop:no peer:172.18.0.3 local:172.18.0.2 port:467]
2024/11/05 14:00:17.401 BFD: [MSVDW-Y8Z5Q] ptm-add-dest: register peer [mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default cbit:0x00 minimum-ttl:255]
2024/11/05 14:00:17.401 BFD: [PSB4R-8T1TJ] session-new: mhop:no peer:172.18.0.3 local:172.18.0.2 vrf:default ifname:eth0
2024/11/05 14:00:17.401 BGP: [Q4BCV-6FHZ5] zclient_bfd_session_update: 172.18.0.2/32 -> 172.18.0.3/32 (interface eth0) VRF default(0) (CPI bit no): Down
2024/11/05 14:00:17.401 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Up -> Down
2024/11/05 14:00:17.401 BGP: [HZN6M-XRM1G] %NOTIFICATION: sent to neighbor 172.18.0.3 6/10 (Cease/BFD Down) 0 bytes 

What is unclear

// BFD is up, and initiate from the that host because remote has passive-mode bfd
// BFD is attached to a BGP session, should the BFD only happen after BGP established?
179 2024/11/05 13:56:17.257 BGP: [MKVHZ-7MS3V] bfd_session_status_update: neighbor 172.18.0.3 vrf default(0) bfd state Down -> Up


203 2024/11/05 13:56:17.400 BGP: [Z7X1S-E98T0] 172.18.0.3 [FSM] BGP OPEN message delayed for 240 seconds
204 2024/11/05 14:00:17.400 BGP: [XC901-RSZMG] 172.18.0.3 [FSM] Timer (DelayOpentimer expire)
205 2024/11/05 14:00:17.400 BGP: [ZWCSR-M7FG9] 172.18.0.3 [FSM] DelayOpen_timer_expired (Connect->OpenSent), fd 22

// Why we cannot really have that well configured since first effort?
51 2024/11/05 14:00:17.401 BGP: [VQY9X-CQZKG] bgp_peer_bfd_update_source: address [0.0.0.0->172.18.0.3] to [172.18.0.2->172.18.0.3]
252 2024/11/05 14:00:17.401 BGP: [X8BD9-8RKN4] bgp_peer_bfd_update_source: interface none to eth0


@ton31337
Copy link
Member

ton31337 commented Nov 5, 2024

You can take .rpm/.deb from here by the OS: https://ci1.netdef.org/browse/FRR-PULLREQ3-5951/artifact.

@karampok
Copy link
Author

karampok commented Nov 6, 2024

Hi, I tried and once I have both ends updated to that commit and I do not see any drop anymore. Need to run some more tests but it looks ok. Thanks

karampok added a commit to karampok/frr-k8s that referenced this issue Nov 7, 2024
- check the bgp peering is in place at all times
- use custom image with fix [0] on top of stable 9.1

[0] FRRouting/frr#17344
[1] FRRouting/frr#17337

Signed-off-by: karampok <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants