From jameslist72 at gmail.com Mon May 6 08:52:29 2024 From: jameslist72 at gmail.com (james list) Date: Mon, 6 May 2024 14:52:29 +0200 Subject: [c-nsp] STP over Port-channel issue Message-ID: dear experts a customer of mine has a legacy environment with 4 x Cisco 9500 (IOS XE 17.09.03) connected in a square mode with 2 links (2 per each connection) and each couple of links is considered a single virtual port (port-channel). Loops are managed with PVSTP. Two x C9500 are in DC1 while the other two x 9500 are in DC2. 9500-1 == 9500-3 || || 9500-2 == 9500-4 The issue is: when there is a provider flapping, ie one out of the two link connecting C9500 in the two DC is flapping, we see one port flaps, port-channel remains UP but PVSTP reconverge and there is a communication failure. The question: since the PO remains up, why we see this behaviour ? are BDPU sent just over one link (ie the higher interfac e) ? how can we solve this issue keeping this scenario ? moving to RSTP could solve ? Thanks in advance Cheers James here some logs: May 6 11:48:35.590: %LINEPROTO-5-UPDOWN: Line protocol on Interface TwentyFiveGigE1/0/48, changed state to down May 6 11:48:36.593: %LINK-3-UPDOWN: Interface TwentyFiveGigE1/0/48, changed state to down May 6 11:48:40.343: %LINK-3-UPDOWN: Interface TwentyFiveGigE1/0/48, changed state to up May 6 11:48:44.171: %LINEPROTO-5-UPDOWN: Line protocol on Interface TwentyFiveGigE1/0/48, changed state to up #sh etherchannel summary ------+-------------+-----------+----------------------------------------------- 1 Po1(SU) LACP Twe1/0/1(P) Twe1/0/2(P) 12 Po12(SU) LACP Twe1/0/47(P) Twe1/0/48(P) Port-channel: Po12 (Primary Aggregator) ------------ Age of the Port-channel = 187d:15h:09m:01s Logical slot/port = 9/12 Number of ports = 2 HotStandBy port = null Port state = Port-channel Ag-Inuse Protocol = LACP Port security = Disabled Fast-switchover = disabled Fast-switchover Dampening = disabled Ports in the Port-channel: Index Load Port EC state No of bits ------+------+------+------------------+----------- 0 00 Twe1/0/47 Active 0 0 00 Twe1/0/48 Active 0 Time since last port bundled: 0d:01h:50m:00s Twe1/0/48 Time since last port Un-bundled: 0d:01h:50m:09s Twe1/0/48 # sh spanning-tree interface port-channel 12 state VLAN0015 blocking VLAN0016 forwarding # sh spanning-tree vlan 15 detail VLAN0015 is executing the rstp compatible Spanning Tree protocol Bridge Identifier has priority 20480, sysid 15, address 8c84.4236.5fe0 Configured hello time 2, max age 20, forward delay 15, transmit hold-count 6 Current root has priority 8207, address 444c.a830.ff71 Root port is 2089 (Port-channel1), cost of root path is 2000 Topology change flag not set, detected flag not set Number of topology changes 73 last change occurred 01:52:13 ago from Port-channel1 Times: hold 1, topology change 35, notification 2 hello 2, max age 20, forward delay 15 Timers: hello 0, topology change 0, notification 0, aging 300 Port 2089 (Port-channel1) of VLAN0015 is root forwarding Port path cost 1000, Port priority 128, Port Identifier 128.2089. Designated root has priority 8207, address 444c.a830.ff71 Designated bridge has priority 16399, address cc79.d760.73e0 Designated port id is 128.2089, designated path cost 1000 Timers: message age 15, forward delay 0, hold 0 Number of transitions to forwarding state: 18 Link type is point-to-point by default BPDU: sent 289, received 4527220 Port 2100 (Port-channel12) of VLAN0015 is alternate blocking Port path cost 1000, Port priority 128, Port Identifier 128.2100. Designated root has priority 8207, address 444c.a830.ff71 Designated bridge has priority 12303, address 444c.a830.fc01 Designated port id is 128.107, designated path cost 1999 Timers: message age 15, forward delay 0, hold 0 Number of transitions to forwarding state: 20 Link type is point-to-point by default BPDU: sent 340, received 4525304 From saku at ytti.fi Mon May 6 09:21:49 2024 From: saku at ytti.fi (Saku Ytti) Date: Mon, 6 May 2024 16:21:49 +0300 Subject: [c-nsp] STP over Port-channel issue In-Reply-To: References: Message-ID: On Mon, 6 May 2024 at 15:53, james list via cisco-nsp wrote: > The question: since the PO remains up, why we see this behaviour ? > are BDPU sent just over one link (ie the higher interfac e) ? Correct. > how can we solve this issue keeping this scenario ? > moving to RSTP could solve ? No. I understand you want topology to remain intact, as long as there is at least 1 member up, but I'm not sure we can guarantee that. I think if you set LACP to fast, it'll fail in max 3s, and you ensure STP fails slower (i.e. don't use rapid pvst), you probably will just see BPDU switch physical interface, instead of STP convergence. You'd need something similar to microBFD on LAGs, where BFD PDU is sent on each member, instead of an unspecified single member, but afaik this does not exist. So what you really need is L3/MPLS topology :/. -- ++ytti From jameslist72 at gmail.com Mon May 6 12:04:07 2024 From: jameslist72 at gmail.com (james list) Date: Mon, 6 May 2024 18:04:07 +0200 Subject: [c-nsp] STP over Port-channel issue In-Reply-To: References: Message-ID: Thanks good point on LACP Fast, we'll test it. RSTP should be in any case slower than 3 seconds with LACP FAST. Cheers James Il giorno lun 6 mag 2024 alle ore 15:22 Saku Ytti ha scritto: > On Mon, 6 May 2024 at 15:53, james list via cisco-nsp > wrote: > > > The question: since the PO remains up, why we see this behaviour ? > > are BDPU sent just over one link (ie the higher interfac e) ? > > Correct. > > > how can we solve this issue keeping this scenario ? > > moving to RSTP could solve ? > > No. > > I understand you want topology to remain intact, as long as there is > at least 1 member up, but I'm not sure we can guarantee that. I think > if you set LACP to fast, it'll fail in max 3s, and you ensure STP > fails slower (i.e. don't use rapid pvst), you probably will just see > BPDU switch physical interface, instead of STP convergence. > > You'd need something similar to microBFD on LAGs, where BFD PDU is > sent on each member, instead of an unspecified single member, but > afaik this does not exist. > > So what you really need is L3/MPLS topology :/. > > -- > ++ytti >