[c-nsp] loop guard still useful?

[Lukas Tribus]

Hello list,



tl;dr rapid-pvst and autonegotiation will fail you, use UDLD and loop guard.



> So I'm wondering if there's any reason to keep loop guard configured
> on a switch?
> Any current hardware that doesn't support rapidSTP? Some other reason?

Whatever is documented on CCO, it will *NOT* prevent layer 2 loops
in redundant layer 2 networks in all situations, at least with rapid-pvst.

I don't think this is actually implemented in Cisco rapid-pvst, but it may be
in MST.



> Ethernet with autonegotiation on should detect unidirectional links
> automatically and go down on both ends at RTT/2 delay.

Ethernet autonegotiation will let you down if the HW or SW fails, you
need control plane features to detect unidirectional links and that is
also the case for directly connected switches on point-to-point
dark-fibers with autonegotiation.



> On A-B link, where A=>B works but A<=B does not, A will go down and A
> will assert RFI or remote fault indicator on the line. B will receive
> this, and go down as well.

This assumption breaks when you have some kind of RX or TX stall, which
I saw first-hand on a 7600 linecard that suddenly became faulty (and
caused a major layer 2 loop because the links had neither UDLD nor loop
guard, just plain old rapid-pvst and autonegotiation).

Also, all of this is not true if you have EoMPLS, EoSDH, radio-links and
other nasty stuff that can suddenly break a single direction and/or both
directions without RFI'ing the failure end-to-end.

Unless you operate a network exclusively on dark-fibers or passive WDM,
you can't assume that everything will go physically down nice and easy.

You carriers layer 2 circuit signaled RFI once? That doesn't mean it
will continue to do so after (or within) the next maintenance window or
if the carrier has a different kind of failure within his network.

Would you deploy a routed or MPLS switched network without BFD and
without IGP hello timeout, just relying on the interfaces operational
state?

I can't accept a layer 2 loop because of a single hardware defect. I need
the control plane to protect me from that and this is what UDLD and loop
guard can do.


Thats why:
> In fact after what Saku said I would consider trusting the layer 1

You can trust layer 1 so long that you trust that you will never ever
have a single hardware defect, a single NP stall or ASIC freeze.


Yes, both HW and SW defects can always cause layer 2 problems even with
UDLD and loop-guard, but those cases are unrelated to STP and much less
common (you can only migrate to layer 3 to eliminate that factor, because
neither FabricPath, nor TRILL nor VPLS can fix fundamental layer 2 issues).



> No, I don't like UDLD at all - too many bad experiences with it. It
> was a necessary evil with cat5500s and 100Mb fiber connections, but
> you don't need UDLD on 1Gb fiber links.

UDLD had bugs, so did STP, RSTP, MST and even autonegotiation, but if
your IOS image is younger than 6 years, those bugs have probably been
fixed.



> So it seems like loop guard isn't needed if rstp is enabled.

I strongly disagree.

Another example where loop guard will help you and where both UDLD
and RSTP fail is this (no unidirectional link needed for this):

Suppose two switches are interconnected with a primary and a backup
EoWHATEVER link, and the backup link doesn't signal RFI:

- backup links goes down (both directions, not unidirectional)
- switches don't see each others BPDUs on the backup link, so they go
  into forwarding
- backup links comes back up, and formes a loop right away (STP status is
  forwarding on all ports of all links)
- when the BPDU is retransmitted (once every 2 seconds in rapid-pvst),
  there's already a loop on the circuit overloading the links capacity,
  so that the BPDU that are supposed to fix the problem are lost
- because the primary link is also overloaded, BPDU will be lost on that
  link as well
- you have to manually interrupt the loop to restore the service


Loop guard will prevent the switch from making the stupid decision to
move the port from blocking to forwarding, because it doesn't receive any
BPDUs from the other switch anymore (Discarding -> Learning -> Forwarding
in rapid-pvst). Instead, it will block the traffic indefinitely until
BPDUs arrive again on that Vlan (this is per stp instance, so per vlan in
rapid-pvst).


One could argue that BPDUs need to be prioritized on all links, including
Layer 2 Ethernet poing-to-point links, but thats besides the point, because
you would still get a loop in the example mentioned above (until STP
converges); also this is often difficult/not doable on layer 2 circuits.



>> so apparently 100Mb fiber doesn't have that problem any more. I don't
>> think 1 or 10Gb fiber ever did..
> I believe if you implement autonego, you have to implement RFI. But
> I'm not 100% sure about that.

I can tell that at least single-fiber 100BASE-BX10 (such as these [1])
don't do autonegotiation (in fact unidirectional links in 100BASE-BX10
FTTX networks are an everyday occurrence, they just don't meltdown your
network because you don't do spanning-tree with your customers anyway).

All 1000BASE standards (including single-fiber) expect autonegotiation
fortunately (but some platforms disable it by default - like ASR9k).

I have no experience with dual-fiber 100BaseFX though.


I don't know if MSTP does it any better, but Cisco's rapid-pvst doesn't.

Of course, it less likely to fail when you have a small network with
only dark-fibers and no layer 2 circuits, no radio or EoSDH links, but
your own box may fail nonetheless (like 7k6 linecard mentioned earlier).


Loop guard is your friend, use loop guard in STP networks!

UDLD is your friend as well, use UDLD in layer 2 networks!



>> Thus, if your Joe Average while
>> troubleshooting does a shut/no shut, he actually gets the loop.
> I'm not sure about shut/no shut but a reboot after the link goes
> unidirectional -- yes, you get a loop.

Which is why you need UDLD as well as loop guard.



> If you are operating an all-cisco net you might take a look at bridge
> assurance.

I don't know bridge assurance, but I do know that UDLD and loop guard
protects from all the of the remaining RSTP's flaws.




cheers,
Lukas


[1] http://www.cisco.com/c/en/us/products/collateral/interfaces-modules/fast-ethernet-sfp-modules/product_data_sheet0900aecd801f931c.html