[c-nsp] etherchannel load-balancing and unpredictability

[John Gill]

Correction:
Now, if your content filter X was using link A and B, and content filter 
Y was using link *C and D*, you might find for these 4 flows that Y 
looks to be failing 100% of the time, and filter X seems to be failing 
50% of the time.

Regards,
John Gill
cisco


On 7/20/11 11:27 AM, John Gill wrote:
> Hi Steve,
> That is correct, each switch decides where it hashes in the *tx*
> direction. The return traffic hashing is decided by the other switch, so
> even if that switch uses the same hash inputs (src-dst-ip for example),
> there is no guarantee it will take the same link when the platforms are
> not the same. Sometimes you will get both directions of the flow to take
> the same link, and other times not.
>
> If Switch 1 happens to send 4 flows equally across 4 links, and Switch 2
> sends the .100/.101 pair back on link A, nothing on link B, .200/201
> back on link C, and both .300/.301 *and* .400/.401 back on link D we
> would have this representation:
>
> Switch 1 Switch 2
> .100 -> 101 ---- Link A ---- .100 <- .101
> .200 -> 201 ---- Link B ---- (no flows)
> .300 -> 301 ---- Link C ---- .200 <- .201
> .400 -> 401 ---- Link D ---- .300 <- .301
> 			       .400 <- .401
>
> In terms of load sharing, we have a potentially off-balance scenario,
> but let's not worry about that since 4 flows will hardly dictate how
> your actual traffic will ultimately be "balanced" in terms of bps.
>
> If you were to look at which flows matched on both sides, in this
> example, only the .100/.101 flow happens to be working properly, but the
> rest of them will see problems when it comes to the content filter -
> luck of the draw.
>
> Now, if your content filter X was using link A and B, and content filter
> Y was using link B and C, you might find for these 4 flows that Y looks
> to be failing 100% of the time, and filter X seems to be failing 50% of
> the time.
>
> With more flows of real network traffic, I would expect both filters to
> show some degree of failure.
>
>
> Regards,
> John Gill
> cisco
>
>
> On 7/19/11 4:49 PM, Steven Pfister wrote:
>> I think I kind of see what you mean. For a given source/destination ip
>> address pair. Switch A might always select path 1 every time going to
>> switch B, but on the return trip, switch B might select path 2 every
>> time going back to switch A. Something like that?
>>
>> But if that were the case, would that happen with all connections? The
>> behavior we were seeing (one content filter has been removed from the
>> network) is that some people were getting blocked correctly and some
>> weren't, which is what I would expect if one filter were working
>> correctly and the other wasn't.
>>
>> Steve Pfister
>> Network Engineer
>> Office of Information Technology
>> Dayton Public Schools
>> 115 S Ludlow St
>> Dayton, OH 45402-1812
>> Phone: 937-542-3149
>> Cell: 937-673-6779
>> spfister at dps.k12.oh.us ( mailto:spfister at dps.k12.oh.us )
>>
>>
>>>>> John Gill<johgill at cisco.com> 7/19/2011 4:30 PM>>>
>> Hello Steve,
>> The port selection function is based on a hash of the inputs, in this
>> case the source and destination IP address, and the output is a value
>> that chooses a member interface.
>>
>> These functions between the input and output vary by platform, but for
>> a
>> given platform you can expect with the same flow, and even the reverse
>>
>> of that flow, you will get the same selection.
>>
>> Because the platforms implement this selection logic in hardware, they
>>
>> can and *do* vary between one model and another. As a matter of fact,
>>
>> the new Nexus 5500 has the ability to choose between various
>> polynomials
>> used in this function. You might find one gives you a better
>> distribution over another. In your case, it sounds like you need some
>>
>> state on both sides of the content filter to be in tact. In this
>> scenario, you would need the same platform on both sides to guarantee
>> this kind of behavior.
>>
>> Regards,
>> John Gill
>> cisco
>>
>>
>> On 7/19/11 4:11 PM, Steven Pfister wrote:
>>> I have a question regarding etherchannel load balancing. I've got a
>>> 4507R switch connected to a 3560 switch by means of two content
>> filters
>>> which are acting as transparent bridges. The two ports on each side
>> that
>>> the content filters are connected to are set up as access ports and
>> are
>>> in an etherchannel. The load balancing method on each switch is set
>> to
>>> src-dst-ip. I was under the impression that each pair of source and
>>> destination ip address would select exactly one content filter no
>> matter
>>> which direction.
>>>
>>> I've been told that this can be 'unpredictable' and may cause
>>> assymetric flows. The algorithm seems fairly straightforward to me.
>> I
>>> don't see where the unpredictability can come in. Can someone explain
>> to
>>> me what I'm missing?
>>>
>>>
>>> Steve Pfister
>>> Network Engineer
>>> Office of Information Technology
>>> Dayton Public Schools
>>> 115 S Ludlow St
>>> Dayton, OH 45402-1812
>>> Phone: 937-542-3149
>>> Cell: 937-673-6779
>>> spfister at dps.k12.oh.us ( mailto:spfister at dps.k12.oh.us )