[nsp] 6500/7600 with large ARP/MAC tables?

[Charles Spurgeon]

We have a 6509 that ran CatOS sup2/msfc2/pfc2 with approx 166
vlans. 

This box has recently been converted to a 6509 CatIOS
sup720/msfc3/pfc3 

The spanning tree load is:
++++++++++++++++++++++++++++++++++++++++++++++++++
Name                   Blocking Listening Learning Forwarding STP Active
---------------------- -------- --------- -------- ---------- ----------
166 vlans                    1         0        0        307        308
++++++++++++++++++++++++++++++++++++++++++++++++++

The sup720 currently supports 5,823 arp table entries, with
a mac address count of:
++++++++++++++++++++++++++++++++++++++++++++++++++
Ser11-sup720#sh mac-address-table count 
MAC Entries  for all vlans :
Dynamic Address Count:                3945
Static Address (User-defined) Count:  1120
Total MAC Addresses In Use:           5065
Total MAC Addresses Available:        65536
++++++++++++++++++++++++++++++++++++++++++++++++++

Our experience has been that operations were stable on both sup2 and
(so far) on the sup720. However, broadcast/multicast floods sent up
the links from the leaf sites can overload the RP CPU and cause
failures with routing protocols.

In our lab tests Sup2 RP appears to be able to handle bcast ARPs at a
rate of approx 70% of a GigE channel. Sup720 RP doesn't appear to be
much better (running 12.2(14)SX1). We've been enabling bcast rate
limits on leaf connections. 

Multicast rate limits are more difficult, since a number of sites use
Norton Ghost and don't want to see any rate limiting on
multicast. Also, the Sup720 doesn't support multicast rate limits on
all ints. However, multicast floods can overload the RP, so it's a
vulnerability.

Note that the Sup720 netflow implementation is 12.2(14)SX1 is
seriously broken and can cause very high SP CPU load when a scanning
flood transits the box. 

If the scanning flood lasts long enough the SP CPU can become
permanently stuck in 94% overload, and will not drop when the scanning
flood stops and/or netflow is disabled. An SP CPU permanently stuck in
overload can cause all manner of weird problems with the Sup720 due to
IPC failures etc.

-Charles

Charles E. Spurgeon / UTnet
UT Austin ITS / Networking
c.spurgeon at its.utexas.edu / 512.475.9265


 >I personally run a Hybrid 6509/s2/msfc2/pfc2 with about 15,000 arp
 >entries, and about 325 interfaces. It runs well. It's got 12.1.19E1.
 >
 >It moves just around a gigabit/sec. It's got about 4,000 routes.
 >
 >CPU utilization for five seconds: 9%/2%; one minute: 12%; five minutes: 12%
 >
 >
 >
 >
 >I have a customer who has only two interfaces, but is over the 20k range
 >in mac entries.
 >
 >
 >
 >On Fri, 22 Aug 2003, Dmitri Kalintsev wrote:
 >
 >> Hi good people,
 >>
 >> I'm looking for some real-life (if possible) experience terminating around
 >> 400-500 VLANs each having around 100-200 hosts, while being an IP gateway
 >> for all of them (one gateway IP per VLAN), anticipated CAM table size of
 >> around 60-70K entries (all hosts are IP and will be talking via this box to
 >> the rest of the network, so ARP table is expected to be the same size). The
 >> hardware configuration assumed is 6500/7600, native IOS (which version,
 >> btw?), Sup720.
 >>
 >> How likely is this configuration to run into harware limitations of the
 >> platform or any other scalability issues? Assume that numbers above are the
 >> worst-case scenario (no further growth anticipated).
 >>
 >> Thanks,
 >> --
 >> D.K.
 >> _______________________________________________
 >> cisco-nsp mailing list  cisco-nsp at puck.nether.net
 >> https://puck.nether.net/mailman/listinfo/cisco-nsp
 >> archive at http://puck.nether.net/pipermail/cisco-nsp/
 >>
 >
 >-- Alex Rubenstein, AR97, K2AHR, alex at nac.net, latency, Al Reuben --
 >--    Net Access Corporation, 800-NET-ME-36, http://www.nac.net   --
 >
 >_______________________________________________
 >cisco-nsp mailing list  cisco-nsp at puck.nether.net
 >https://puck.nether.net/mailman/listinfo/cisco-nsp
 >archive at http://puck.nether.net/pipermail/cisco-nsp/