[c-nsp] STP to L2 FTTx gear

[Justin Shore]

Hey, Phil.  Thanks for the reply.

I forgot to include this log excerpt in my first message.  Maybe it will 
help.

6545845: Jan  6 00:59:16 CST: %SPANTREE-SP-2-RECV_PVID_ERR: Received 
BPDU with inconsistent peer vlan id 801 on GigabitEthernet1/9 VLAN811.
6545846: Jan  6 00:59:16 CST: %SPANTREE-SP-2-BLOCK_PVID_PEER: Blocking 
GigabitEthernet1/9 on VLAN0801. Inconsistent peer vlan.
6545847: Jan  6 00:59:16 CST: %SPANTREE-SP-2-BLOCK_PVID_LOCAL: Blocking 
GigabitEthernet1/9 on VLAN0811. Inconsistent local vlan.
6545848: Jan  6 00:59:57 CST: %SPANTREE-SP-2-UNBLOCK_CONSIST_PORT: 
Unblocking GigabitEthernet1/9 on VLAN0801. Port consistency restored.
6545849: Jan  6 00:59:57 CST: %SPANTREE-SP-2-UNBLOCK_CONSIST_PORT: 
Unblocking GigabitEthernet1/9 on VLAN0811. Port consistency restored.

Phil Mayers wrote:
> Just one minor correction; You didn't setup the trunks with a native 
> VLAN, so it defaults to 1. Given the somewhat special nature of vlan 1 
> on Ciscos, and the fact these ports face a 3rd party, this might be 
> unwise. I would recommend using a dummy vlan as the native vlan on the 
> port.

I hadn't thought about that.  That could be a problem.  I could either 
go the route of a dummy VLAN or just configure the L2 infrastructure to 
accept a native VLAN.  I'll work on both options.

> Dear Cisco - please provide a way to put the entire port into "no native 
> vlan" mode, as opposed to the entire chassis. Sincerely yours etc.

Agreed.  Other vendors seem to embrace that level of configurability. 
It would be nice to have it if for no other reason than for 
interoperability to other vendors though we probably won't use it that 
often.

> I think not:
> 
> http://tinyurl.com/yak4ls
> 
> """Port VLAN ID (PVID) inconsistency—A per-VLAN spanning tree (PVST+) 
> Bridge Protocol Data Unit (BPDU) is received on a different VLAN than it 
> was originated: (Port VLAN ID Mismatch or *PVID_Inc)."""
> 
> I'll bet your L2 provider has gotten vlan 801 & 811 swapped around on 
> one of the links to the 7600 somehow. Or, has somehow left PVST running 
> inside their own network, and is doing vlan remapping, but only on the 
> carrier vlans for 801/811.

I'll dig into that one.  That could be the issue.  I can't recall the IP 
of the second switch we connect to or I'd check it now.  It is possible 
that one was flipped.  I'll check out angle in the morning.

We're also the L2 provider, though it's not on our usual Cisco gear.  We 
chose Occam as our FTTx vendor.  The GigE ring and access layer is made 
up of Occam switches.  Occam has taken what I'd consider to be an 
unusual approach for VLAN numbering.  They retag (not QinQ but VLAN ID 
retagging) our 8xx VLANs based on the VLAN's purpose to what I believe 
are hardcoded VLAN IDs on their gear.  For example management uses VLAN 
2.  Voice uses VLAN 5.  Public Internet data uses 4.  Etc.  This 
translation should only occur on the upstream edge touching my L3 core. 
  One thing that I do not know is if my BPDUs are traversing the entire 
ring before popping out on the other GigE linecard and into my second 
7600, or if the BPDUs are passing directly between the 2 GigE linecards 
in our CO and aren't traversing the ring.  I need to inquire with the 
vendor about this.

>> So, my question is which form of STP would be best for this particular 
>> scenario?  1d, 1w, PVST, Rapid-PVST or should I be looking at MST?  I 
> 
> None are likely to work with 100% reliability if you can't get a 
> completely clean L2 path.
> 
> My personal view is that PVST still wins in most cases. MST is a bad 
> joke, because you can only run one MST "process" on a cisco; this can be 
> crippling in some topologies e.g. ours.

I wasn't aware of that limitation.  I could see how that would be a 
minor problem (minor meaning crippling).

I'm thinking that it wouldn't be a bad idea if the Occam gear 
participated in PVST with us.  Then I'd lower the port priority on the 
link between the first pair of Occam switches (the ones I touch) so that 
link will be chosen to be blocked.  By having the Occam gear participate 
I would be able to overcome failure situations where the device fails 
but the link is still up.  I don't suppose UDLD would be an option here, 
would it?

>> don't care about load-balancing at the present.  It would be nice is 
>> the physical link mimicked HSRP but I'm not that picky.  I'm assuming 
>> that 
> 
> I am assuming you have a link between the 7600s with the vlans on; 
> simply set the STP cost lower on that, and the 7600 which is the STP 
> secondary root will block the ports facing the L2 provider, and favour 
> the port(s) facing the other 7600.
 >
> Making the HSRP master always be the same as the STP master is tricky, 
> but can be achieved with "standby preempt" and appropriate "standby 
> track" statements, especially if you're on 12.2(33) with the much better 
> object tracking,

I hadn't altered the costs yet.  This was the basic plan though once I 
had STP working.  I'm currently tracking the physical interface, set the 
HSRP priorities and enabled preemption.  This won't account for a 
failure in either Occam switch that doesn't take down the physical 
interface.  UDLD would take care of this but I don't think I can use it 
here.  IIRC UDLD is proprietary so Occam support is unlikely.  I don't 
believe you can carry UDLD between UDLD devices through devices that 
don't support UDLD.

> Our experience procuring L2 links from providers (not FTTx it must be 
> said) has been distinctly mixed; we've had MTU issues, vlan tagging 
> problems, unidirectional forwarding losses, and failure of unicast 
> flooding.
> 
> I can imagine letting your provider participate in your STP being very, 
> very painful!

I probably didn't word it well earlier.  We aren't using a 3rd-party L2 
provider.  We are the provider.  We're just using a non-Cisco FTTx 
solution (Cisco doesn't have a viable FTTx solution for this market). 
I'm not technically responsible for the Occams but I can get in and try 
to find the problem anyway.

Thanks for the pointers, Phil.  I'll report back what I find.

Justin