[nsp] MSFC2 128,000 route limitation

[Matt Buford]

On Wed May 15 2002 - 07:53:32 EDT, Ian Cox wrote:

> The TCAM that holds the FIB table is capable of holding 256,000 entries.
> Without unicast RPF checking turned on the maximum number of unicast
> entries that can be held in the hardware FIB table is 244,000. The
> remaining 12,000 entries are reserved for multicast routes. If unicast RPF
> checking is enabled then the number of routes that are held in the TCAM is
> halved.

> You can exceed the capacity of the hardware forwarding table, and the
> consequences are that the routes that are not programmed into the TCAM
that
> holds the FIB table will be switched in software by the MSFC2 / RP.

I have apparently ran into this limitation, with much worse consequences
(running Sup2/MSFC2 hybrid).  The supervisor CPU shot up to 100%, and all
updates from the MSFC to the supervisor/PFC stopped.  This happened in both
of a pair of redundant 6500s, bringing both down and leaving me unable to
bring them back up with a full routing table.

Cisco TAC found bug cscdw89942, and said the internal notes recommend using
the "set mls cef per-prefix-stats disable" to reduce the number of entries.

It appears that at this point the limitiation is not something to take
lightly.  Reaching it (at least under Hybrid) apparently brings everything
down.  There is no software yet available that fixes this, and the only
workaround is to take measures to reduce your CEF table size (such as
turning off per-prefix-stats).

For perspective, the routers that failed each see two BGP feeds of full
Internet routes, as well as about 12 OSPF routes (each of which has 2 or 3
paths to get there).  This doesn't seem like a particularly large number of
routes to me, however it certainly passes the limit listed in the bug of
50,000 routes with dual paths.

Is there anywhere I can get a count of the actual current number of entries
and/or space free, or is the only way to tell to show the cef table size and
manually figure out if you need to multiply it if you have unicast RPF on,
then make sure that is less than 244,000?  I want to go through all my 6500s
and make sure I'm not about to hit the limit on any of them (some are hybrid
and some are native).  The thought of  all my 6500s falling over at once and
staying down because I reached the maximum limit on routes scares me
greatly.