[c-nsp] Nexus 5K optimisation for iSCSI traffic

Wed Aug 17 02:55:44 EDT 2011

That is why my ISSU migration scenario is like:

1) check on the blade switch that all ports except the uplink are in EDGE
mode and forwarding (= no loop)
2) then disable STP on uplink (bpdufilter on uplink)
3) then put "spanning-tree port type edge trunk" on N5K (port becomes edge
:-)
4) and do ISSU
5) reverse :-)

Or if you are really confident in the nic teaming configuration of the
server guys, you can also just shutdown the link :-)

regards,
Geert

2011/8/13 Tóth András <diosbejgli at gmail.com>

> Let me add that my point is more about the rare situation when a
> downstream port on the blade switch gets connected (looped) with one
> of the upstream FEX switches, then you might face a loop.
>
> FlexLink is indeed better when you have 2 uplinks and you don't want
> them both to be active at the same time, I tried to emphasize that
> precautions should be taken with FL as well because the active link in
> FL is not running STP.
>
> Andras
>
>
> 2011/8/12 Tóth András <diosbejgli at gmail.com>:
> > Hi Brad,
> >
> > I am not sure that FlexLink is safer than BPDU filter in any way. As
> > the Configuration Guide points out, FlexLink will disable STP on the
> > interface and it's required to ensure a loop-free topology.
> > A port with BPDU filter is behaving in the same way as one with
> > FlexLink because neither of them is sending or processing BPDU
> > packets, therefore a loop might form. In other words, by connecting 2
> > interfaces with FlexLink on them will sustain a loop in the same way
> > as 2 interfaces with BPDU filter.
> >
> > STP is disabled on Flex Link ports. A Flex Link port does not
> > participate in STP, even if the VLANs present on the port are
> > configured for STP. When STP is not enabled, be sure that there are no
> > loops in the configured topology.
> >
> http://www.cisco.com/en/US/docs/switches/blades/3020/software/release/12.2_55_se/configuration/guide/swflink.html#wp1088284
> >
> > FlexLink can be used with the backup interface being down, it should work
> fine.
> >
> > Best regards,
> > Andras
> >
> >
> > On Mon, Aug 8, 2011 at 12:41 AM, Brad Hedlund (brhedlun)
> > <brhedlun at cisco.com> wrote:
> >> Geert,
> >>
> >>
> >>
> >> Allow me clarify #2 below, as the way I wrote that was a bit misleading.
> >> Thanks for having me look at this again.
> >>
> >>
> >>
> >> When you enable BPDU filter on an interface, that interface will not
> >> receive or send BPDUs.  It's the equivalent of disabling STP on that
> >> interface.
> >>
> >> This, as you can imagine, may lead to spanning-tree loops between your
> >> blade switch and upstream switch(es).  Because, unlike with FlexLinks,
> >> additional uplinks added with STP disabled have no state relationship
> >> with the other links.
> >>
> >> At least STP is still protecting the server facing links, so the _risky_
> >> rating still remains, as opposed to _dangerous_ in option #3.
> >>
> >>
> >>
> >> I'm not sure if you can successfully configure Flex Links with no
> >> available backup interface.
> >>
> >> Perhaps you can define an empty port as the backup?  I don't know.
> >>
> >>
> >>
> >> Maybe someone can try it in a lab and let us know?
> >>
> >>
> >>
> >> Cheers,
> >>
> >> Brad
> >>
> >> http://bradhedlund.com
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> From: Geert Nijs [mailto:geert.nijs at gmail.com]
> >> Sent: Sunday, August 07, 2011 2:00 PM
> >> To: Brad Hedlund (brhedlun)
> >> Cc: Matthew Melbourne; cisco-nsp at puck.nether.net
> >> Subject: Re: [c-nsp] Nexus 5K optimisation for iSCSI traffic
> >>
> >>
> >>
> >> Thanks Brad for this tip.
> >> I was experiencing a similar problem where a downstream cisco
> >> bladeswitch was preventing ISSU on my N5K (it was not considered an STP
> >> edge port).
> >> I myself came up with the outbound BPDUfilter solution on the blade
> >> switch, but flexlinks might be a safer solution.
> >> 1) Could you explain why flexlink is safer than bpdufilter for example ?
> >> 2) Can i use flexlinks if my blade switch only has one uplink (ie. there
> >> is no backup port) (there is of course another blade switch in the
> >> chassis that has an uplink to the alternative switch and blade switches
> >> are running
> >> link state tracking, i am sure you are familiar with this setup)
> >>
> >>
> >> regards,
> >> Geert
> >>
> >> 2011/8/5 Brad Hedlund (brhedlun) <brhedlun at cisco.com>
> >>
> >> No. The FEX has BPDU Guard logic running in hardware. The moment a BPDU
> >> is received on the port it will be disabled.
> >>
> >> On the blade switches you can implement:
> >> 1) Flex Links (safe)
> >> 2) Egress BPDU filter (risky)
> >> 3) Disable STP (dangerous)
> >>
> >> For #2 and #3, a misconfigured or missing LACP config can cause a loop.
> >> For #3, a misconfigured server NIC teaming can cause a loop.
> >>
> >> Cheers,
> >> Brad
> >> http://bradhedlund.com
> >>
> >> Sent from my iPad
> >> (please excuse brevity, typos)
> >>
> >> On Aug 5, 2011, at 9:49 AM, "Matthew Melbourne" <matt at melbourne.org.uk>
> >> wrote:
> >>
> >>> Can P prevent a FEX port being disabled by implementing bpdufilter, or
> >>> do we need to ensure that BPDUs aren't receiving on FEX ports?
> >>>
> >>> We were hoping to use LACP between the downstream switch and the FEXes
> >>> as a poor-man's loop prevention mechanism.
> >>>
> >>> Cheers,
> >>>
> >>> Matt
> >>>
> >>> On 5 August 2011 15:17, John Gill <johgill at cisco.com> wrote:
> >>>> It would be filter toward the FEX ports on your blade switches, but
> >> not on
> >>>> the FEX ports themselves. Whether you turn STP off or not on the
> >> blades, the
> >>>> FEX doesn't know. Just remembering if you create a loop, you no
> >> longer have
> >>>> the protection of STP; you are intentionally tricking the FEX into
> >> not
> >>>> knowing there is a switch downstream.
> >>>>
> >>>> Regards,
> >>>> John Gill
> >>>> cisco
> >>>>
> >>>> On 8/5/11 9:59 AM, Matthew Melbourne wrote:
> >>>>>
> >>>>> Thanks for that - that's another issue we've encountered. I am
> >> hoping
> >>>>> we can implement bpdufilter on the FEX ports (as well as disabling
> >> STP
> >>>>> on downstream switches).
> >>>>>
> >>>>> On 5 August 2011 14:12, Brad Hedlund (brhedlun)<brhedlun at cisco.com>
> >>>>>  wrote:
> >>>>>>
> >>>>>> Note that the FEX will disable any port that receives a BPDU, by
> >> design
> >>>>>> in hardware.  You will need to disable STP on the
> >> blade-switch-to-FEX links
> >>>>>> for this to work. If it's Cisco blade switches you can use Flex
> >> Links.
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Brad
> >>>>>> http://bradhedlund.com
> >>>>>>
> >>>>>> Sent from my iPad
> >>>>>> (please excuse brevity, typos)
> >>>>>>
> >>>>>> On Aug 5, 2011, at 6:08 AM, "Matthew
> >> Melbourne"<matt at melbourne.org.uk>
> >>>>>>  wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> We're implementing two pairs of N5Ks (and downstream N2k FEXes) to
> >> act
> >>>>>>> as separate iSCSI SAN fabrics, with SAN heads attached directly to
> >>>>>>> N5Ks and host ports (and downstream integrated blade switches)
> >>>>>>> connecting to the FEXes. Does anyone have any real-world
> >> experience of
> >>>>>>> using N5Ks for a large iSCSI deployment. I have enabled jumbo
> >> frames
> >>>>>>> through a network-qos policy-map as an obvious first-step, but
> >> wonder
> >>>>>>> whether anything can be optimised by tuning buffer sizes to
> >>>>>>> accommodate the bursty nature of iSCSI (etc)? This switches will
> >> only
> >>>>>>> be switching iSCSI traffic.
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Matt
> >>>>>>>
> >>>>>>> --
> >>>>>>> Matthew Melbourne
> >>>>>>> _______________________________________________
> >>>>>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> >>>>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >>>>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Matthew Melbourne
> >>
> >> _______________________________________________
> >> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >>
> >>
> >>
> >> _______________________________________________
> >> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >>
> >
>