[c-nsp] Sup720-3BXL load balancing re-visited

Fri Apr 28 06:38:39 EDT 2006

Hi,

This morning, I believe we got something solid to present about load
balancing in the Sup720-3BXL, IOS 12.2(18)SXF being broken.

Scenerio.  All links are POS STM-1's.  Our network is behind the 7600,
the 7500's are edge routers to our upstreams, located in London/New
York.  Same AS on both sides, so equal load balancing (mostly).

7500, RSP8, VIP4-80 (7500 no 1)
                                                                     /
7600, Sup720-3BXL w/ SIP-200, 2xPOS STM1
                                                                    \

7500, RSP16, VIP6-80 (7500 no 2)

Just before 1 o'clock last night (UTC), the link to 7500 no 2 went
down for about 40 secs due to unknown failure (we're investigating
that).  All outbound (from the 7600) traffic moved to the other link
(to 7500 no 1) instantly, but it never came back once the other link's
state was restored.  Inbound traffic behaved normally (as that's of
course dictated by our provider's routers, on the other side of the
7500's).

Okay, now we've got all outbound traffic going to 7500 no. 1, but
inbound traffic load balancing normally.  We tried clearing all BGP
peers on 7500 no. 2.  Then all outbound traffic went to 7500 no. 2 and
nothing to 7500 no. 1.  Still, inbound traffic was normal.  We cleared
all BGP peers on 7500 no. 1, and all traffic shifted to no. 1, nothing
to no. 2.

We cleared all BGP peers on the 7600, now we have almost equal load
balancing (we've found that we really can't get equal load balancing
on the 7600, it's more like 1/3 vs. 2/3, but it's better than all
going one way and nothing the other!).

Now, my opinion is that this is a problem with load balancing in the
7600.  We already have "mls ip cef load-sharing simple", which was
supposed to solve this problem, at least for MPLS tagged routes.  Any
other ideas?

Thanks,
Kristo