[c-nsp] CEF Tuning with ECMP?

Sun Oct 3 15:17:59 EDT 2010

Tim,

That's a great point. On the two switches that are most greatly affected,
traffic is only going one way so we could switch back to etherchannels. I'll
suggest that.

Thanks again,
John
On Oct 3, 2010 1:11 PM, "Tim Stevenson" <tstevens at cisco.com> wrote:
> Hi John,
>
> For 4948->7600, I don't see why you would have channel replication
> issues. 4K is a shared memory buffer system, so there's no
> ingress/egress replication considerations or suboptimal distribution
> across the fabric of the MD packet for egress replication. There's
> only one copy of the packet in shared memory up to the point where a
> channel member is selected. In this situation I'd think a single 4
> port routed EC would be preferred, with all mroutes having the same
> OIF (the EC) but then allowing the EC hash to select the link based
> on L3+L4. But that's theoretical - if you had issues with that then
> I'm not necessarily suggesting you go back to it.
>
> Of course, if you have heavy mcast going back the other direction as
> well, then egress L3 replication w/EC behavior is a consideration.
>
> 2 cents,
> Tim
>
>
> At 10:16 AM 10/3/2010, John Neiberger remarked:
>
>>Thanks, Tim. I've only heard of ECMP in relation to multicast.
>>Thanks for clearing up the terminology. It sounds like chanting the
>>multicast multipath hash is our only option.
>>
>>We used to use etherchannels but we switched to this to help work
>>around some multicast replication issues. If changing the hash
>>doesn't work, we may have to go back to etherchannels. I really hope
>>we don't have to do that.
>>
>>Thanks again!
>>On Oct 3, 2010 11:08 AM, "Tim Stevenson"
>><<mailto:tstevens at cisco.com>tstevens at cisco.com> wrote:
>> > Hi John,
>> > Let's get everyone agreed on the terminology first, then we can try
>> > to solve the problem.
>> >
>> > * ECMP = Equal cost multipath, it is most typically a term used
>> > around unicast routing where for a given IP prefix you have multiple
>> > equal cost next hops and you load share traffic among them based on a
>> > hash (or less commonly per packet). The hash can be based on several
>> > criteria, ie, IPs & L4 ports in various combinations.
>> >
>> > * CEF = it's interchangeable with 'ECMP' - CEF-based load sharing &
>> > ECMP mean the same thing.
>> >
>> > * Multicast multipath = Uses a hash to select the RPF interface for a
>> > particular multicast flow when there are ECMP paths back to the
>> > source/RP. There are options to determine the input values (G, S,G,
>> > S,G+NH). This feature is not on by default in IOS. If it is not
>> > enabled, then IOS will choose ONE of the ECMP paths as the RPF
>> > (highest neighbor IP) and ALL multicast will be pulled over that link.
>> >
>> > * Polarization = In a 'multi-tier' network, using the same hash input
>> > on each tier results in traffic after the 1st tier polarizing to a
>> > subset of the available links. It's accomodated for by adding a
>> > unique ID at each hop to the hash input for unicast; for multicast
>> > multipath, by including the next hop IP as hash input. Whether this
>> > really comes into play depends on the depth of the network
>> routing topology.
>> >
>> > Ok - so given all of the above, with ECMP routing between the 7600s &
>> > the 4948s, and with multicast multipath already enabled on the 7600
>> > and using S,G basic hashing: if the traffic flow is from the
>> > 4k->7600, the only option you have to improve things is to use S,G +
>> > next-hop. I'm not entirely convinced it will have a major impact, it
>> > depends on whether you have multiple levels of routing, one which is
>> > getting RPF hash selection pretty evenly but then at this layer,
>> > polarization is occurring since only a subset of traffic is reaching
>> > it and the hash input is the same (so only a subset of links is being
>> > selected as RPF). Based on your description I can't tell if that's a
>> > possibility in your setup.
>> >
>> > Regardless of all that, changing CEF/ECMP hash input on the 4948 will
>> > not have any significant impact, since that wouldn't affect multicast
>> > traffic at all, any particular S,G will still have only ONE of those
>> > four interfaces as an OIF, and that is driven by where the PIM join
>> > came in from the 7600, which in turn is driven by whether mcast
>> > multipath is enabled, and what hash is used to select the RPF
interface.
>> >
>> > Also, clearly, changing CEF/ECMP hash input on the 7600 would have
>> > not any impact since you're worried about traffic flowing the other
>> > direction anyway.
>> >
>> > Hope that helps,
>> > Tim
>> >
>> > At 09:09 AM 10/3/2010, John Neiberger remarked:
>> >
>> >>I'm starting another thread because the topic is migrating. To
>> >>simplify, we have a 7600 with SUP720-3BXL connected via four routed
>> >>links to a 4948. The bulk of the traffic on this network is multicast
>> >>traffic flowing from the 4948 to the 7600 (and onward from there). In
>> >>order to get the best load sharing over those four links, what is the
>> >>recommended CEF tuning and ECMP configuration?
>> >>
>> >>I ask because we seem to be running into ECMP polarization and/or CEF
>> >>polarization. We have already decided that we need to be using
>> >>s-g-hash next-hop-based for ECMP. We're using s-g-hash basic right
>> >>now. But what about CEF? Do we need to tune CEF along with tuning ECMP
>> >>for this to work properly? We want the most even distribution of
>> >>traffic possible. As it is right now, we're seeing serious unequal
>> >>load sharing. In some cases all of the traffic is going over one link
>> >>and not even using the other three.
>> >>
>> >>Do any of you know the recommended CEF parameters in a situation like
this?
>> >>
>> >>Thanks,
>> >>John
>> >>_______________________________________________
>> >>cisco-nsp mailing list
>> <mailto:cisco-nsp at puck.nether.net>cisco-nsp at puck.nether.net
>> >><https://puck.nether.net/mailman/listinfo/cisco-nsp><https://puck.
>> nether.net/mailman/listinfo/cisco-nsp>
https://puck.nether.net/mailman/listinfo/cisco-nsp
>> >>archive at
>> >><<http://puck.nether.net/pipermail/cisco-nsp/>http://puck.nether.n
>> et/pipermail/cisco-nsp/>http://puck.nether.net/pipermail/cisco-nsp/
>> >
>> >
>> >
>> >
>> > Tim Stevenson, <mailto:tstevens at cisco.com>tstevens at cisco.com
>> > Routing & Switching CCIE #5561
>> > Distinguished Technical Marketing Engineer, Cisco Nexus 7000
>> > Cisco - <http://www.cisco.com>http://www.cisco.com
>> > IP Phone: 408-526-6759
>> > ********************************************************
>> > The contents of this message may be *Cisco Confidential*
>> > and are intended for the specified recipients only.
>> >
>> >
>
>
>
>
> Tim Stevenson, tstevens at cisco.com
> Routing & Switching CCIE #5561
> Distinguished Technical Marketing Engineer, Cisco Nexus 7000
> Cisco - http://www.cisco.com
> IP Phone: 408-526-6759
> ********************************************************
> The contents of this message may be *Cisco Confidential*
> and are intended for the specified recipients only.
>
>