[c-nsp] Nexus 2232PP FEX Switch Question

[Douglas C. Stephens]

Joeseph,

The other responders are correct that the Nexus 2000 series fabric extenders are
not switches.  Nevertheless, I was forced to deal with a situation recently where
a customer had installed a managed switch downstream of one of my 2248 fabric
extenders, so even though it doesn't involve VPC, maybe it helps you or somebody.

I had the ports on this 2248 fabric extender all configured with "spanning-tree port
type edge", since that is what I expected all the downstream equipment to be.
One day several links began flapping all the time because STP BPDUs would be
generated by the customer's switch, cause the port on the 2248 to err-disable,
the 5 minutes later err-recover would kick in and re-enble the port -- rinse and
repeat.  This generated a lot of log noise that looked like this (note the lines
in the middle referencing BPDUGuard):

%ETHPORT-5-SPEED: Interface Ethernet103/1/13, operational speed changed to 1 Gbps
%ETHPORT-5-IF_DUPLEX: Interface Ethernet103/1/13, operational duplex mode changed to Full
%ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet103/1/13, operational Receive Flow Control state changed to off
%ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet103/1/13, operational Transmit Flow Control state changed to on
%ETHPORT-5-IF_UP: Interface Ethernet103/1/13 is up in mode access
%ETHPORT-5-IF_DOWN_NONE: Interface Ethernet103/1/13 is down (None)
%ETHPORT-5-IF_DOWN_ERROR_DISABLED: Interface Ethernet103/1/13 is down (Error disabled. Reason:BPDUGuard)
%ETHPORT-5-IF_ERRDIS_RECOVERY: Interface Ethernet103/1/13 is being recovered from error disabled state (Last Reason:BPDUGuard)
%ETHPORT-5-SPEED: Interface Ethernet103/1/13, operational speed changed to 1 Gbps
%ETHPORT-5-IF_DUPLEX: Interface Ethernet103/1/13, operational duplex mode changed to Full
%ETHPORT-5-IF_RX_FLOW_CONTROL: Interface Ethernet103/1/13, operational Receive Flow Control state changed to off
%ETHPORT-5-IF_TX_FLOW_CONTROL: Interface Ethernet103/1/13, operational Transmit Flow Control state changed to on
%ETHPORT-5-IF_UP: Interface Ethernet103/1/13 is up in mode access

A solution that worked for me for this situation I found referenced at Cisco here
described as enabling BPDUfilter:

http://www.cisco.com/en/US/docs/switches/datacenter/nexus5000/sw/troubleshooting/guide/n5K_ts_l2.html#wp1026440

I implemented this on the affected 2248 ports like this:

config t
  interface Ethernet103/1/13
  no spanning-tree port type edge
  spanning-tree bpdufilter enable
exit

Once changed, as soon as err-recovery re-enabled the ports, they stayed enabled.
Once I became aware that the customer had used a managed switch with STP running,
and then had to implement this solution, I "educated" the customer to make sure
nothing was connected to that switch except physical servers.


At 06:56 PM 12/10/2013, Joseph Hardeman wrote:
>Hi Everyone,
>
>I knew I should have come here first but I went with the word of a CCXX
>something or another (Director of IT) from a vendor and a couple links he
>sent me.  After I explained the setup I was putting together and how
>everything needed to work together, he told me that the Nexus 2232PP Switch
>could do what I wanted and needed.
>
>I have a pair of 2232PP FEX switches that we just got to extend our 5000
>series switches from one cage to another (I have never worked with the FEX
>Switches before, so I should have done better homework).  We were going to
>use the pair of FEX switches to provide redundant links to additional
>switches from which I was going to connect to customer switches or
>firewalls.  Which going through the initial config today on setting up the
>VPC port on say Eth100/1/1 for my first test switch, I got an error saying
>that the VPC could not be added that there was one already applied.
>
>I reached out to the TAC Contact I have that has helped me out this week,
>and he told me that the FEX Switches were never meant to connect to other
>switches and the BPDUGuard would shut down the ports to the switch shortly
>after the ports come up.  And it was not just this model but any FEX Switch.
>
>So my question to you guys is, (drum roll please):
>
>Does anyone have a Nexus 2000 FEX Switch Pair doing VPC Port Channel to
>another switch instead of a host/server?  And if you do how did you make it
>work.  I am considering returning these switches as I can't use them right
>now and I really need a usable pair of switches for the 10G+ cage to cage
>connectivity and then 1 or 10G to either my switches or customer switches.
> Which my switches then step it down to either 100M or 10M if needed.
>
>Is there a FEX switch that will do this? (imagine the full mesh setup)
>
>routers -> 5000Switch -> FEX Switch -> 2960G or 3560G (for example) switch
>-> Possible other switching/firewall gear -> end system
>
>Can you recommend any other switch that can do what I want, or should I
>just get another pair of 5000 series switches?
>
>Thanks,
>
>Any thoughts or suggestions would be helpful.
>
>Joe
>_______________________________________________
>cisco-nsp mailing list  cisco-nsp at puck.nether.net
>https://puck.nether.net/mailman/listinfo/cisco-nsp
>archive at http://puck.nether.net/pipermail/cisco-nsp/

--
Douglas C. Stephens             | Network/DNS/Unix/Windows Admin
System Support Specialist       | Email Postmaster
Information Technology          | Phone: (515) 294-6102
Ames Laboratory, US DOE         | Email: stephens at ameslab.gov
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 170 bytes
Desc: not available
URL: <https://puck.nether.net/pipermail/cisco-nsp/attachments/20131211/63f25e0f/attachment-0001.sig>