[c-nsp] Nexus 5K optimisation for iSCSI traffic

Fri Aug 12 13:23:31 EDT 2011

Hi Brad,

I am not sure that FlexLink is safer than BPDU filter in any way. As
the Configuration Guide points out, FlexLink will disable STP on the
interface and it's required to ensure a loop-free topology.
A port with BPDU filter is behaving in the same way as one with
FlexLink because neither of them is sending or processing BPDU
packets, therefore a loop might form. In other words, by connecting 2
interfaces with FlexLink on them will sustain a loop in the same way
as 2 interfaces with BPDU filter.

STP is disabled on Flex Link ports. A Flex Link port does not
participate in STP, even if the VLANs present on the port are
configured for STP. When STP is not enabled, be sure that there are no
loops in the configured topology.
http://www.cisco.com/en/US/docs/switches/blades/3020/software/release/12.2_55_se/configuration/guide/swflink.html#wp1088284

FlexLink can be used with the backup interface being down, it should work fine.

Best regards,
Andras

On Mon, Aug 8, 2011 at 12:41 AM, Brad Hedlund (brhedlun)
<brhedlun at cisco.com> wrote:
> Geert,
>
>
>
> Allow me clarify #2 below, as the way I wrote that was a bit misleading.
> Thanks for having me look at this again.
>
>
>
> When you enable BPDU filter on an interface, that interface will not
> receive or send BPDUs.  It's the equivalent of disabling STP on that
> interface.
>
> This, as you can imagine, may lead to spanning-tree loops between your
> blade switch and upstream switch(es).  Because, unlike with FlexLinks,
> additional uplinks added with STP disabled have no state relationship
> with the other links.
>
> At least STP is still protecting the server facing links, so the _risky_
> rating still remains, as opposed to _dangerous_ in option #3.
>
>
>
> I'm not sure if you can successfully configure Flex Links with no
> available backup interface.
>
> Perhaps you can define an empty port as the backup?  I don't know.
>
>
>
> Maybe someone can try it in a lab and let us know?
>
>
>
> Cheers,
>
> Brad
>
> http://bradhedlund.com
>
>
>
>
>
>
>
> From: Geert Nijs [mailto:geert.nijs at gmail.com]
> Sent: Sunday, August 07, 2011 2:00 PM
> To: Brad Hedlund (brhedlun)
> Cc: Matthew Melbourne; cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] Nexus 5K optimisation for iSCSI traffic
>
>
>
> Thanks Brad for this tip.
> I was experiencing a similar problem where a downstream cisco
> bladeswitch was preventing ISSU on my N5K (it was not considered an STP
> edge port).
> I myself came up with the outbound BPDUfilter solution on the blade
> switch, but flexlinks might be a safer solution.
> 1) Could you explain why flexlink is safer than bpdufilter for example ?
> 2) Can i use flexlinks if my blade switch only has one uplink (ie. there
> is no backup port) (there is of course another blade switch in the
> chassis that has an uplink to the alternative switch and blade switches
> are running
> link state tracking, i am sure you are familiar with this setup)
>
>
> regards,
> Geert
>
> 2011/8/5 Brad Hedlund (brhedlun) <brhedlun at cisco.com>
>
> No. The FEX has BPDU Guard logic running in hardware. The moment a BPDU
> is received on the port it will be disabled.
>
> On the blade switches you can implement:
> 1) Flex Links (safe)
> 2) Egress BPDU filter (risky)
> 3) Disable STP (dangerous)
>
> For #2 and #3, a misconfigured or missing LACP config can cause a loop.
> For #3, a misconfigured server NIC teaming can cause a loop.
>
> Cheers,
> Brad
> http://bradhedlund.com
>
> Sent from my iPad
> (please excuse brevity, typos)
>
> On Aug 5, 2011, at 9:49 AM, "Matthew Melbourne" <matt at melbourne.org.uk>
> wrote:
>
>> Can P prevent a FEX port being disabled by implementing bpdufilter, or
>> do we need to ensure that BPDUs aren't receiving on FEX ports?
>>
>> We were hoping to use LACP between the downstream switch and the FEXes
>> as a poor-man's loop prevention mechanism.
>>
>> Cheers,
>>
>> Matt
>>
>> On 5 August 2011 15:17, John Gill <johgill at cisco.com> wrote:
>>> It would be filter toward the FEX ports on your blade switches, but
> not on
>>> the FEX ports themselves. Whether you turn STP off or not on the
> blades, the
>>> FEX doesn't know. Just remembering if you create a loop, you no
> longer have
>>> the protection of STP; you are intentionally tricking the FEX into
> not
>>> knowing there is a switch downstream.
>>>
>>> Regards,
>>> John Gill
>>> cisco
>>>
>>> On 8/5/11 9:59 AM, Matthew Melbourne wrote:
>>>>
>>>> Thanks for that - that's another issue we've encountered. I am
> hoping
>>>> we can implement bpdufilter on the FEX ports (as well as disabling
> STP
>>>> on downstream switches).
>>>>
>>>> On 5 August 2011 14:12, Brad Hedlund (brhedlun)<brhedlun at cisco.com>
>>>>  wrote:
>>>>>
>>>>> Note that the FEX will disable any port that receives a BPDU, by
> design
>>>>> in hardware.  You will need to disable STP on the
> blade-switch-to-FEX links
>>>>> for this to work. If it's Cisco blade switches you can use Flex
> Links.
>>>>>
>>>>> Cheers,
>>>>> Brad
>>>>> http://bradhedlund.com
>>>>>
>>>>> Sent from my iPad
>>>>> (please excuse brevity, typos)
>>>>>
>>>>> On Aug 5, 2011, at 6:08 AM, "Matthew
> Melbourne"<matt at melbourne.org.uk>
>>>>>  wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We're implementing two pairs of N5Ks (and downstream N2k FEXes) to
> act
>>>>>> as separate iSCSI SAN fabrics, with SAN heads attached directly to
>>>>>> N5Ks and host ports (and downstream integrated blade switches)
>>>>>> connecting to the FEXes. Does anyone have any real-world
> experience of
>>>>>> using N5Ks for a large iSCSI deployment. I have enabled jumbo
> frames
>>>>>> through a network-qos policy-map as an obvious first-step, but
> wonder
>>>>>> whether anything can be optimised by tuning buffer sizes to
>>>>>> accommodate the bursty nature of iSCSI (etc)? This switches will
> only
>>>>>> be switching iSCSI traffic.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Matt
>>>>>>
>>>>>> --
>>>>>> Matthew Melbourne
>>>>>> _______________________________________________
>>>>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>
>>
>>
>>
>> --
>> Matthew Melbourne
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>