[c-nsp] etherchannel load-balancing and unpredictability

[John Gill]

Hi Steve,
That is correct, each switch decides where it hashes in the *tx* 
direction.  The return traffic hashing is decided by the other switch, 
so even if that switch uses the same hash inputs (src-dst-ip for 
example), there is no guarantee it will take the same link when the 
platforms are not the same.  Sometimes you will get both directions of 
the flow to take the same link, and other times not.

If Switch 1 happens to send 4 flows equally across 4 links, and Switch 2 
sends the .100/.101 pair back on link A, nothing on link B, .200/201 
back on link C, and both .300/.301 *and* .400/.401 back on link D we 
would have this representation:

   Switch 1		       Switch 2
.100 -> 101 ---- Link A ---- .100 <- .101
.200 -> 201 ---- Link B ----   (no flows)
.300 -> 301 ---- Link C ---- .200 <- .201
.400 -> 401 ---- Link D ---- .300 <- .301
			     .400 <- .401

In terms of load sharing, we have a potentially off-balance scenario, 
but let's not worry about that since 4 flows will hardly dictate how 
your actual traffic will ultimately be "balanced" in terms of bps.

If you were to look at which flows matched on both sides, in this 
example, only the .100/.101 flow happens to be working properly, but the 
rest of them will see problems when it comes to the content filter - 
luck of the draw.

Now, if your content filter X was using link A and B, and content filter 
Y was using link B and C, you might find for these 4 flows that Y looks 
to be failing 100% of the time, and filter X seems to be failing 50% of 
the time.

With more flows of real network traffic, I would expect both filters to 
show some degree of failure.


Regards,
John Gill
cisco


On 7/19/11 4:49 PM, Steven Pfister wrote:
> I think I kind of see what you mean. For a given source/destination ip
> address pair. Switch A might always select path 1 every time going to
> switch B, but on the return trip, switch B might select path 2 every
> time going back to switch A. Something like that?
>
> But if that were the case, would that happen with all connections? The
> behavior we were seeing (one content filter has been removed from the
> network) is that some people were getting blocked correctly and some
> weren't, which is what I would expect if one filter were working
> correctly and the other wasn't.
>
> Steve Pfister
> Network Engineer
> Office of Information Technology
> Dayton Public Schools
> 115 S Ludlow St
> Dayton, OH 45402-1812
> Phone: 937-542-3149
> Cell: 937-673-6779
> spfister at dps.k12.oh.us ( mailto:spfister at dps.k12.oh.us )
>
>
>>>> John Gill<johgill at cisco.com>  7/19/2011 4:30 PM>>>
> Hello Steve,
> The port selection function is based on a hash of the inputs, in this
> case the source and destination IP address, and the output is a value
> that chooses a member interface.
>
> These functions between the input and output vary by platform, but for
> a
> given platform you can expect with the same flow, and even the reverse
>
> of that flow, you will get the same selection.
>
> Because the platforms implement this selection logic in hardware, they
>
> can and *do* vary between one model and another.  As a matter of fact,
>
> the new Nexus 5500 has the ability to choose between various
> polynomials
> used in this function.  You might find one gives you a better
> distribution over another.  In your case, it sounds like you need some
>
> state on both sides of the content filter to be in tact.  In this
> scenario, you would need the same platform on both sides to guarantee
> this kind of behavior.
>
> Regards,
> John Gill
> cisco
>
>
> On 7/19/11 4:11 PM, Steven Pfister wrote:
>> I have a question regarding etherchannel load balancing. I've got a
>> 4507R switch connected to a 3560 switch by means of two content
> filters
>> which are acting as transparent bridges. The two ports on each side
> that
>> the content filters are connected to are set up as access ports and
> are
>> in an etherchannel. The load balancing method on each switch is set
> to
>> src-dst-ip. I was under the impression that each pair of source and
>> destination ip address would select exactly one content filter no
> matter
>> which direction.
>>
>> I've been told that this can be 'unpredictable' and may cause
>> assymetric flows. The algorithm seems fairly straightforward to me.
> I
>> don't see where the unpredictability can come in. Can someone explain
> to
>> me what I'm missing?
>>
>>
>> Steve Pfister
>> Network Engineer
>> Office of Information Technology
>> Dayton Public Schools
>> 115 S Ludlow St
>> Dayton, OH 45402-1812
>> Phone: 937-542-3149
>> Cell: 937-673-6779
>> spfister at dps.k12.oh.us ( mailto:spfister at dps.k12.oh.us )
>> _______________________________________________
>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>