[c-nsp] TCN's - Causing brief outages on ASR1K

Will Tardy will.tardy at vocus.com.au
Fri Dec 19 01:57:00 EST 2014 

Switches can have rate-limiters (aka storm-control measures) to control broadcast, multicast and unknown-unicast. BPDU's are layer-2 multicast. Your issues sounds like you're losing BPDUs.

Try "debug spanning-tree root" to see if it's causing root changes.

If you are running a lot of VLANs worth of BPDUs, you might be bumping into a limit somewhere on your network or a peer's. 

It's because of problems like this, that STP shouldn't cross administrative-domains.


On 18 Dec 2014, at 4:09 pm, CiscoNSP List <cisconsp_list at hotmail.com> wrote:

>> 
>> This seems like..."interesting" advice. At that point, you might as
>> well just turn spanning-tree off. This is somewhere around cutting off
>> your foot to stop your toe bleeding.
>> 
>> That said: This seems like "design problem" not so much "gear
>> problem". Why are you running spanning tree with devices you don't
>> administratively control? And if you do control them, why the hell are
>> you seeing TCNs so often if your network is stable?
> 
> 
> We dont control the device this port is connected to, and when the port was configured, bpdufilter was not enabled(12months ago)....TCN's have only recently been "arriving" on this port.
> 
> 
>> 
>> -Blake
>> 
>> On Wed, Dec 17, 2014 at 8:35 PM, CiscoNSP List
>> <cisconsp_list at hotmail.com> wrote:
>>> Another update on this....TAC are recommending that we enable bpdufilter on "all" ports, as any port that is root and receives a TCN will cause an "outage"....we have bpdufilter enabled on customer facing ports(to other switches), but some of our legacy equipment/connections would be missing this command Im sure....I find it incredibly difficult to believe that we have not been hit by this in the past, if this is "expected" behaviour on any switch.
>>> 
>>> Would love to hear from any switching experts on TAC's recommendation, and have we just been "lucky" not to be impacted by this in the past?
>>> 
>>> 
>>> Cheers.
>>> 
>>> From: cisconsp_list at hotmail.com
>>> To: petelists at templin.org; mrantoinemonnier at gmail.com; luky-37 at hotmail.com; cisco-nsp at puck.nether.net
>>> Subject: RE: [c-nsp] TCN's - Causing brief outages on ASR1K
>>> Date: Tue, 16 Dec 2014 10:51:07 +1100
>>> 
>>> 
>>> 
>>> 
>>> Thanks for all the replies.
>>> 
>>> Just an update to this - No issues for 4 days (with  spanning-tree portfast trunk enabled on trunk port from 4948 -> ASR), then this morning, another TCN was received on an AGG port(On the 4948) to a carrier (The TCN's (Since we are now looking for them)) seem to only arrive on 2 ports...both being carrier AGG ports, with multiple vlans, and to the same carrier.....we do not have any visibility into the carriers network
>>> 
>>> This TCN did also cause OSPF+LDP flaps on the ASR....again, only for a ~5seconds
>>> 
>>> It's "RRR"(So highest priority) with TAC, but we are still in the same place we were over a week ago.....as you can imagine, customers are not impressed!
>>> 
>>> 
>>> 
>>>> Date: Mon, 15 Dec 2014 09:04:43 -0800
>>>> From: petelists at templin.org
>>>> To: mrantoinemonnier at gmail.com; luky-37 at hotmail.com; cisconsp_list at hotmail.com; cisco-nsp at puck.nether.net
>>>> Subject: Re: [c-nsp] TCN's - Causing brief outages on ASR1K
>>>> 
>>>> You can run RSTP or MST all day long on a switch to get rapid STP
>>>> convergence, but you'll only gain the rapidness of RSTP/MST on ports
>>>> where they neighbor is actually participating in the correct STP
>>>> variant. Routers don't participate in STP, so the 4948 has to treat
>>>> those ports as legacy STP. Whenever there's a root placement event, the
>>>> 4948 has to block the port until the STP process/timers can confirm that
>>>> there's no superior root bridge hiding inside or behind the router.
>>>> 
>>>> Now, if there's a small enough event going on that SHOULDN'T be causing
>>>> a root placement event but IS, that could be a bug in the 4948 code.
>>>> 
>>>> However, I'd say very strongly that you SHOULD have portfast [trunk]
>>>> towards any devices that aren't participating in the STP process, unless
>>>> those devices are capable of creating an L2 loop.
>>>> 
>>>>> On 12/15/2014 1:18 AM, Antoine Monnier wrote:
>>>>> A TCN will cause all the learned MAC addresses to be flushed by the
>>>>> switiches, but it will not "block" traffic. So the TCN on its own should
>>>>> not be the cause of OSPF and LDP flaps.
>>>>> 
>>>>> Is your switch running out of space for all the learned MAC addresses?
>>>>> 
>>>>>  I dont see how enabling "portfast trunk" would help in that scenario (it
>>>>> should only change the behavior if an interface flaps).
>>>>> Has the source of TCN being identified? Configuring ports as "portfast"
>>>>> will lower the probability of generating TCN, that may be why they advised
>>>>> you to do this. However applying to a port that is stable (no interface
>>>>> flap) is not really going to help for this specific problem.
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Mon, Dec 15, 2014 at 9:06 AM, Lukas Tribus <luky-37 at hotmail.com> wrote:
>>>>>> 
>>>>>>> Is it expected behaviour for a TCN to cause a flap on an ASR...We have
>>>>>> many other POP's with switches
>>>>>>> 4948's/4500's etc(trunk)->ASR+7200's and do not have spanning-tree
>>>>>> portfast trunk enabled, and
>>>>>>> they do not experience any flaps?
>>>>>> This has nothing todo with the ASR1k at all. Its expected behavior that
>>>>>> STP on the switch will block traffic
>>>>>> when there's a reconvergence, especially when malconfigured (like not
>>>>>> using portfast on
>>>>>> router or host connected links).
>>>>>> 
>>>>>> Why this doesn't happen on your 7k2 we can't tell, there are a lot of
>>>>>> moving parts that only you know
>>>>>> (for example whether you are using pvst, rapid-pvst or mst and where
>>>>>> exactly the root of those particular
>>>>>> vlans is).
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Regards,
>>>>>> 
>>>>>> Lukas
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>>> _______________________________________________
>>>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>> 
>>> _______________________________________________
>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>> 
>> _______________________________________________
>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>                         
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

More information about the cisco-nsp mailing list