[c-nsp] Nexus 5K optimisation for iSCSI traffic

Fri Aug 12 18:40:48 EDT 2011

Let me add that my point is more about the rare situation when a
downstream port on the blade switch gets connected (looped) with one
of the upstream FEX switches, then you might face a loop.

FlexLink is indeed better when you have 2 uplinks and you don't want
them both to be active at the same time, I tried to emphasize that
precautions should be taken with FL as well because the active link in
FL is not running STP.

Andras

2011/8/12 Tóth András <diosbejgli at gmail.com>:
> Hi Brad,
>
> I am not sure that FlexLink is safer than BPDU filter in any way. As
> the Configuration Guide points out, FlexLink will disable STP on the
> interface and it's required to ensure a loop-free topology.
> A port with BPDU filter is behaving in the same way as one with
> FlexLink because neither of them is sending or processing BPDU
> packets, therefore a loop might form. In other words, by connecting 2
> interfaces with FlexLink on them will sustain a loop in the same way
> as 2 interfaces with BPDU filter.
>
> STP is disabled on Flex Link ports. A Flex Link port does not
> participate in STP, even if the VLANs present on the port are
> configured for STP. When STP is not enabled, be sure that there are no
> loops in the configured topology.
> http://www.cisco.com/en/US/docs/switches/blades/3020/software/release/12.2_55_se/configuration/guide/swflink.html#wp1088284
>
> FlexLink can be used with the backup interface being down, it should work fine.
>
> Best regards,
> Andras
>
>
> On Mon, Aug 8, 2011 at 12:41 AM, Brad Hedlund (brhedlun)
> <brhedlun at cisco.com> wrote:
>> Geert,
>>
>>
>>
>> Allow me clarify #2 below, as the way I wrote that was a bit misleading.
>> Thanks for having me look at this again.
>>
>>
>>
>> When you enable BPDU filter on an interface, that interface will not
>> receive or send BPDUs.  It's the equivalent of disabling STP on that
>> interface.
>>
>> This, as you can imagine, may lead to spanning-tree loops between your
>> blade switch and upstream switch(es).  Because, unlike with FlexLinks,
>> additional uplinks added with STP disabled have no state relationship
>> with the other links.
>>
>> At least STP is still protecting the server facing links, so the _risky_
>> rating still remains, as opposed to _dangerous_ in option #3.
>>
>>
>>
>> I'm not sure if you can successfully configure Flex Links with no
>> available backup interface.
>>
>> Perhaps you can define an empty port as the backup?  I don't know.
>>
>>
>>
>> Maybe someone can try it in a lab and let us know?
>>
>>
>>
>> Cheers,
>>
>> Brad
>>
>> http://bradhedlund.com
>>
>>
>>
>>
>>
>>
>>
>> From: Geert Nijs [mailto:geert.nijs at gmail.com]
>> Sent: Sunday, August 07, 2011 2:00 PM
>> To: Brad Hedlund (brhedlun)
>> Cc: Matthew Melbourne; cisco-nsp at puck.nether.net
>> Subject: Re: [c-nsp] Nexus 5K optimisation for iSCSI traffic
>>
>>
>>
>> Thanks Brad for this tip.
>> I was experiencing a similar problem where a downstream cisco
>> bladeswitch was preventing ISSU on my N5K (it was not considered an STP
>> edge port).
>> I myself came up with the outbound BPDUfilter solution on the blade
>> switch, but flexlinks might be a safer solution.
>> 1) Could you explain why flexlink is safer than bpdufilter for example ?
>> 2) Can i use flexlinks if my blade switch only has one uplink (ie. there
>> is no backup port) (there is of course another blade switch in the
>> chassis that has an uplink to the alternative switch and blade switches
>> are running
>> link state tracking, i am sure you are familiar with this setup)
>>
>>
>> regards,
>> Geert
>>
>> 2011/8/5 Brad Hedlund (brhedlun) <brhedlun at cisco.com>
>>
>> No. The FEX has BPDU Guard logic running in hardware. The moment a BPDU
>> is received on the port it will be disabled.
>>
>> On the blade switches you can implement:
>> 1) Flex Links (safe)
>> 2) Egress BPDU filter (risky)
>> 3) Disable STP (dangerous)
>>
>> For #2 and #3, a misconfigured or missing LACP config can cause a loop.
>> For #3, a misconfigured server NIC teaming can cause a loop.
>>
>> Cheers,
>> Brad
>> http://bradhedlund.com
>>
>> Sent from my iPad
>> (please excuse brevity, typos)
>>
>> On Aug 5, 2011, at 9:49 AM, "Matthew Melbourne" <matt at melbourne.org.uk>
>> wrote:
>>
>>> Can P prevent a FEX port being disabled by implementing bpdufilter, or
>>> do we need to ensure that BPDUs aren't receiving on FEX ports?
>>>
>>> We were hoping to use LACP between the downstream switch and the FEXes
>>> as a poor-man's loop prevention mechanism.
>>>
>>> Cheers,
>>>
>>> Matt
>>>
>>> On 5 August 2011 15:17, John Gill <johgill at cisco.com> wrote:
>>>> It would be filter toward the FEX ports on your blade switches, but
>> not on
>>>> the FEX ports themselves. Whether you turn STP off or not on the
>> blades, the
>>>> FEX doesn't know. Just remembering if you create a loop, you no
>> longer have
>>>> the protection of STP; you are intentionally tricking the FEX into
>> not
>>>> knowing there is a switch downstream.
>>>>
>>>> Regards,
>>>> John Gill
>>>> cisco
>>>>
>>>> On 8/5/11 9:59 AM, Matthew Melbourne wrote:
>>>>>
>>>>> Thanks for that - that's another issue we've encountered. I am
>> hoping
>>>>> we can implement bpdufilter on the FEX ports (as well as disabling
>> STP
>>>>> on downstream switches).
>>>>>
>>>>> On 5 August 2011 14:12, Brad Hedlund (brhedlun)<brhedlun at cisco.com>
>>>>>  wrote:
>>>>>>
>>>>>> Note that the FEX will disable any port that receives a BPDU, by
>> design
>>>>>> in hardware.  You will need to disable STP on the
>> blade-switch-to-FEX links
>>>>>> for this to work. If it's Cisco blade switches you can use Flex
>> Links.
>>>>>>
>>>>>> Cheers,
>>>>>> Brad
>>>>>> http://bradhedlund.com
>>>>>>
>>>>>> Sent from my iPad
>>>>>> (please excuse brevity, typos)
>>>>>>
>>>>>> On Aug 5, 2011, at 6:08 AM, "Matthew
>> Melbourne"<matt at melbourne.org.uk>
>>>>>>  wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> We're implementing two pairs of N5Ks (and downstream N2k FEXes) to
>> act
>>>>>>> as separate iSCSI SAN fabrics, with SAN heads attached directly to
>>>>>>> N5Ks and host ports (and downstream integrated blade switches)
>>>>>>> connecting to the FEXes. Does anyone have any real-world
>> experience of
>>>>>>> using N5Ks for a large iSCSI deployment. I have enabled jumbo
>> frames
>>>>>>> through a network-qos policy-map as an obvious first-step, but
>> wonder
>>>>>>> whether anything can be optimised by tuning buffer sizes to
>>>>>>> accommodate the bursty nature of iSCSI (etc)? This switches will
>> only
>>>>>>> be switching iSCSI traffic.
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Matt
>>>>>>>
>>>>>>> --
>>>>>>> Matthew Melbourne
>>>>>>> _______________________________________________
>>>>>>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>>>>>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>>>>>>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>>>
>>>
>>>
>>>
>>> --
>>> Matthew Melbourne
>>
>> _______________________________________________
>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>>
>>
>> _______________________________________________
>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
>