[c-nsp] association between TCAM usage and CPU load in L3 switch

[Martin T]

Matthias,

as I described in my previous letter, Cisco ME-3600X-24FS-M was not
doing any routing at the time when TCAM for IPv4 unicast entries was
exhausted, but still the CPU was around 100%. While I fully understand
that if switch TCAM is exhausted and switch is performing route
lookups, then those will be punted and CPU usage will be high.
However, in my case, the switch was performing no routing at the time.
Only L2 forwarding.



regards,
Martin

On 2/6/14, Matthias Müller <cnsp at matthias-mueller.net> wrote:
> Hi Martin,
>
> typically Cisco implements TCAM based routing/forwarding on an adjacency
> level. After an exception occured (too many adjacancies be it routes or
> macs), the corresponding address family normally punts every new adjacency
> to cpu after the exception occured. So even reducing the number of routes
> pushed via BGP won't resolve the problem.
> Looking at your failure description you probably have a configuration that
> relies on L3 forwarding, hit an FIB exception and after that all forwarding
> goes via CPU (biggest problem is resetting the BGP sessions after that
> occured, resulting in all prefixes being punted to the cpu...without that
> only new prefixes would be punted to the cpu).
> One advice: always limit on _your_ device the received prefixes by igp/egp
> to a limit your box can handle and know the platform architecture of your
> hardware to know how it will behave in case of failure.
>
> Cheers,
> Matthias
>
> On Thu, 6 Feb 2014 21:00:02 +0200
> Martin T <m4rtntns at gmail.com> wrote:
>
>> Mark, Dimitris,
>>
>> at the time of the test, this L3 switch was not acting as router. In
>> other words it was performing no route lookups, but still the CPU load
>> was ~100% when all the unicast IPv4 TCAM entries were exhausted.
>>
>>
>>
>> regards,
>> Martin
>>
>> On 2/5/14, Dimitris Befas <dimitris.befas at gmail.com> wrote:
>> > Maybe the router does process switching for the traffic while the
>> > corresponding table is full. Maybe you can check with sh int x/x while
>> > the
>> > cpu is overload.
>> > On Feb 5, 2014 5:56 PM, "Martin T" <m4rtntns at gmail.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >> I have a Cisco ME-3600X-24FS-M switch which supports up to 20000
>> >> unicast IPv4 prefixes with SDM profile in use. If I announce full BGP
>> >> table to this switch and exhaust all the TCAM available for unicast
>> >> IPv4 prefixes:
>> >>
>> >>
>> >> ME3600X#sh platform tcam utilization ucastv4
>> >> Nile Tcam Utilization per Application & Region:
>> >> ES == Entry size == Number of 80 bit TCAM words
>> >> ==================================================================
>> >> App/Region            Start  Num Avail  ES    Used Range  Num Used
>> >> ==================================================================
>> >> UCASTV4                   0     20480   1
>> >>     nile0                                                    20000
>> >>     nile1                                                    20000
>> >>
>> >>
>> >> ..then CPU load will be around 100%:
>> >>
>> >>
>> >>       111 111
>> >>       00090009     1             1  1         1
>> >>       000900099999609899888998999099399999999909788888889888998899
>> >>   100 #######*
>> >>    90 #######*
>> >>    80 #######*
>> >>    70 #######*
>> >>    60 ########
>> >>    50 ########
>> >>    40 ########
>> >>    30 ########
>> >>    20 ########
>> >>    10 ########**************************************************
>> >>      0....5....1....1....2....2....3....3....4....4....5....5....6
>> >>                0    5    0    5    0    5    0    5    0    5    0
>> >>                CPU% per minute (last 60 minutes)
>> >>               * = maximum CPU%   # = average CPU%
>> >>
>> >> According to "sh ip route summary", RAM usage for BGP was around
>> >> 85MiB. What exactly is causing such high CPU usage? I should have run
>> >> "sh processes cpu sorted 5min" during the high CPU usage, but forgot
>> >> to do so.
>> >>
>> >>
>> >>
>> >> regards,
>> >> Martin
>> >> _______________________________________________
>> >> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> >> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> >> archive at http://puck.nether.net/pipermail/cisco-nsp/
>> >
>> _______________________________________________
>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
>> https://puck.nether.net/mailman/listinfo/cisco-nsp
>> archive at http://puck.nether.net/pipermail/cisco-nsp/
>