[c-nsp] CEF Tuning with ECMP?

[John Neiberger]

Tim,

One of our engineers changed all the relevant devices to s-g-hash
next-hop-based last night and now all links are being used. It's not a
perfect balance, of course, but that's not to be expected. I'm just
glad we don't have any unused links, particularly on the two switches
that only had 1 out of 4 links passing traffic.

Thanks for your help!
John

On Sun, Oct 3, 2010 at 11:08 AM, Tim Stevenson <tstevens at cisco.com> wrote:
> Hi John,
> Let's get everyone agreed on the terminology first, then we can try to solve
> the problem.
>
> * ECMP = Equal cost multipath, it is most typically a term used around
> unicast routing where for a given IP prefix you have multiple equal cost
> next hops and you load share traffic among them based on a hash (or less
> commonly per packet). The hash can be based on several criteria, ie, IPs &
> L4 ports in various combinations.
>
> * CEF = it's interchangeable with 'ECMP' - CEF-based load sharing & ECMP
> mean the same thing.
>
> * Multicast multipath = Uses a hash to select the RPF interface for a
> particular multicast flow when there are ECMP paths back to the source/RP.
> There are options to determine the input values (G, S,G, S,G+NH). This
> feature is not on by default in IOS. If it is not enabled, then IOS will
> choose ONE of the ECMP paths as the RPF (highest neighbor IP) and ALL
> multicast will be pulled over that link.
>
> * Polarization = In a 'multi-tier' network, using the same hash input on
> each tier results in traffic after the 1st tier polarizing to a subset of
> the available links. It's accomodated for by adding a unique ID at each hop
> to the hash input for unicast; for multicast multipath, by including the
> next hop IP as hash input. Whether this really comes into play depends on
> the depth of the network routing topology.
>
> Ok - so given all of the above, with ECMP routing between the 7600s & the
> 4948s, and with multicast multipath already enabled on the 7600 and using
> S,G basic hashing: if the traffic flow is from the 4k->7600, the only option
> you have to improve things is to use S,G + next-hop. I'm not entirely
> convinced it will have a major impact, it depends on whether you have
> multiple levels of routing, one which is getting RPF hash selection pretty
> evenly but then at this layer, polarization is occurring since only a subset
> of traffic is reaching it and the hash input is the same (so only a subset
> of links is being selected as RPF). Based on your description I can't tell
> if that's a possibility in your setup.
>
> Regardless of all that, changing CEF/ECMP hash input on the 4948 will not
> have any significant impact, since that wouldn't affect multicast traffic at
> all, any particular S,G will still have only ONE of those four interfaces as
> an OIF, and that is driven by where the PIM join came in from the 7600,
> which in turn is driven by whether mcast multipath is enabled, and what hash
> is used to select the RPF interface.
>
> Also, clearly, changing CEF/ECMP hash input on the 7600 would have not any
> impact since you're worried about traffic flowing the other direction
> anyway.
>
> Hope that helps,
> Tim
>
> At 09:09 AM 10/3/2010, John Neiberger remarked:
>
>> I'm starting another thread because the topic is migrating. To
>> simplify, we have a 7600 with SUP720-3BXL connected via four routed
>> links to a 4948. The bulk of the traffic on this network is multicast
>> traffic flowing from the 4948 to the 7600 (and onward from there). In
>> order to get the best load sharing over those four links, what is the
>> recommended CEF tuning and ECMP configuration?
>>
>> I ask because we seem to be running into ECMP polarization and/or CEF
>> polarization. We have already decided that we need to be using
>> s-g-hash next-hop-based for ECMP. We're using s-g-hash basic right
>> now. But what about CEF? Do we need to tune CEF along with tuning ECMP
>> for this to work properly? We want the most even distribution of
>> traffic possible. As it is right now, we're seeing serious unequal
>> load sharing. In some cases all of the traffic is going over one link
>> and not even using the other three.
>>
>> Do any of you know the recommended CEF parameters in a situation like
>> this?
>>
>> Thanks,
>> John
>> _______________________________________________
>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>>
>> <https://puck.nether.net/mailman/listinfo/cisco-nsp>https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at
>> <http://puck.nether.net/pipermail/cisco-nsp/>http://puck.nether.net/pipermail/cisco-nsp/
>
>
>
>
> Tim Stevenson, tstevens at cisco.com
> Routing & Switching CCIE #5561
> Distinguished Technical Marketing Engineer, Cisco Nexus 7000
> Cisco - http://www.cisco.com
> IP Phone: 408-526-6759
> ********************************************************
> The contents of this message may be *Cisco Confidential*
> and are intended for the specified recipients only.
>
>
>