[f-nsp] Odd CPU spikes on Jetcore BI4K, IP level
Jeroen Wunnink
jeroen at easyhosting.nl
Wed Oct 15 07:06:48 EDT 2008
Second set of ports (port 5-8) on slot 1 looks slightly loaded but
nothing major, if you see the counter with:
L3 4096 - 32767 (0x01000 - 0x07fff), free 13870 (0x0362e)
Getting a really low free value, you might want to consider
repartitioning the cam partitions with a higher l3 usage
I gave some more detailed tips on July 15th, might want to go over the
mailing list archive (subject was 'Bigiron 4k with jetcore')
Bardo Cornelissen wrote:
> Hi Jeroen,
>
> I'm going to watch that ip cache, I see it's little above 100k now so I
> assume it could grow above de default 140k and that would explain and we
> should be able to solve this. Many thanks already!
>
> I'm not sure how to look at this cam-partition, can you tell if anything
> looks like an issue or exhausted here:
>
> #sh cam-part det
>
> ==== SLOT 1 CAM PARTITION ====
>
> IGC: 0 (0x00)
> Number of CAM devices per IGC: 1
> Number of hw entries per CAM: 0x08000
> Total size of CAM = 2Mbits
> complete CAM index range per IGC:
> (sw) 1 - 49151 (1 - 0x0bfff), total entries: 49151 (0x0bfff)
> (hw) 0 - 32767 (0 - 0x07fff), total entries: 32768 (0x08000)
> Percentage of CAM hardware entries for each partition:
> Layer3 = 16384 (1Mbits) (50%)
> Level3 = 1023 (0.062438Mbits) (3.121948%)
> Level2 = 1024 (0.0625Mbits) (3.125%)
> Layer2 = 8192 (0.5Mbits) (25%)
> Layer4 = 8192 (0.5Mbits) (25%)
>
> Layer 3 sw index range:
> L3 L3 1 - 2047 (0x00001 - 0x007ff), free 2021 (0x007e5)
> L3 L2 2048 - 4095 (0x00800 - 0x00fff), free 1803 (0x0070b)
> L3 4096 - 32767 (0x01000 - 0x07fff), free 23357 (0x05b3d)
> layer 3 hw index range (inversely mapped):
> 16383 - 0 (0x03fff - 0x00000)
> L2 index range:
> (sw) 32768 - 40959 (0x08000 - 0x09fff), free 7806 (0x01e7e)
> (hw) 16384 - 24575 (0x04000 - 0x05fff)
> L4 pool 3 index range:
> (sw) 40960 - 43519 (0x0a000 - 0x0a9ff), free 2560 (0x00a00)
> (hw) 24576 - 27135 (0x06000 - 0x069ff)
> L4 pool 2 index range:
> (sw) 43520 - 46079 (0x0aa00 - 0x0b3ff), free 2560 (0x00a00)
> (hw) 27136 - 29695 (0x06a00 - 0x073ff)
> L4 pool 1 index range:
> (sw) 46080 - 48639 (0x0b400 - 0x0bdff), free 2560 (0x00a00)
> (hw) 29696 - 32255 (0x07400 - 0x07dff)
> L4 pool 0 index range:
> (sw) 48640 - 49151 (0x0be00 - 0x0bfff), free 504 (0x001f8)
> (hw) 32256 - 32767 (0x07e00 - 0x07fff)
>
> IGC: 1 (0x01)
> Number of CAM devices per IGC: 1
> Number of hw entries per CAM: 0x08000
> Total size of CAM = 2Mbits
> complete CAM index range per IGC:
> (sw) 1 - 49151 (1 - 0x0bfff), total entries: 49151 (0x0bfff)
> (hw) 0 - 32767 (0 - 0x07fff), total entries: 32768 (0x08000)
> Percentage of CAM hardware entries for each partition:
> Layer3 = 16384 (1Mbits) (50%)
> Level3 = 1023 (0.062438Mbits) (3.121948%)
> Level2 = 1024 (0.0625Mbits) (3.125%)
> Layer2 = 8192 (0.5Mbits) (25%)
> Layer4 = 8192 (0.5Mbits) (25%)
>
> Layer 3 sw index range:
> L3 L3 1 - 2047 (0x00001 - 0x007ff), free 1951 (0x0079f)
> L3 L2 2048 - 4095 (0x00800 - 0x00fff), free 1100 (0x0044c)
> L3 4096 - 32767 (0x01000 - 0x07fff), free 13870 (0x0362e)
> layer 3 hw index range (inversely mapped):
> 16383 - 0 (0x03fff - 0x00000)
> L2 index range:
> (sw) 32768 - 40959 (0x08000 - 0x09fff), free 7878 (0x01ec6)
> (hw) 16384 - 24575 (0x04000 - 0x05fff)
> L4 pool 3 index range:
> (sw) 40960 - 43519 (0x0a000 - 0x0a9ff), free 2560 (0x00a00)
> (hw) 24576 - 27135 (0x06000 - 0x069ff)
> L4 pool 2 index range:
> (sw) 43520 - 46079 (0x0aa00 - 0x0b3ff), free 2560 (0x00a00)
> (hw) 27136 - 29695 (0x06a00 - 0x073ff)
> L4 pool 1 index range:
> (sw) 46080 - 48639 (0x0b400 - 0x0bdff), free 2560 (0x00a00)
> (hw) 29696 - 32255 (0x07400 - 0x07dff)
> L4 pool 0 index range:
> (sw) 48640 - 49151 (0x0be00 - 0x0bfff), free 504 (0x001f8)
> (hw) 32256 - 32767 (0x07e00 - 0x07fff)
>
>
> Kind regards,
>
>
> Bardo Cornelissen.
> Caveo Internet BV
>
>
>> -----Oorspronkelijk bericht-----
>> Van: Jeroen Wunnink [mailto:jeroen at easyhosting.nl]
>> Verzonden: woensdag 15 oktober 2008 12:32
>> Aan: Bardo Cornelissen
>> CC: foundry-nsp at puck.nether.net
>> Onderwerp: Re: [f-nsp] Odd CPU spikes on Jetcore BI4K, IP level
>>
>> Hi Bardo,
>>
>> Except for the CAM usage, also look on how saturated your IP cache is.
>> 'sh ip cache' and look at the top at the total number of cache entries.
>>
>> The default max is 140000, if it exceeds this maximum it'll start
>> throwing packets over the CPU, if it's at or near it's max, increase it
>> with 'system-max ip cache 400000' (reload required)
>>
>>
>> Bardo Cornelissen wrote:
>>
>>> Hi folks,
>>>
>>> I'm facing an odd issue causing high CPU load on a Jetcore BI4K, with
>>>
>> for
>>
>>> what seems to be caused in a ip transit uplink.
>>>
>>> We are running BGP on multiple jetcore BigIrons. Now there are 3
>>>
>> border
>>
>>> routers which each have an ip transit uplink. After some time (a day
>>>
>> or
>>
>>> sometimes more like a week) we see a high CPU load and it doesn't
>>>
>> stop until
>>
>>> we shutdown the BGP session with one of these carriers. So far we
>>>
>> have seen
>>
>>> 2 crashes which may have been caused by this.
>>>
>>> So we moved that carrier to a different router (same model), replaced
>>>
>> the
>>
>>> used SFP, upgraded the software on the router but nothing solves the
>>> problem. Also our carrier relocated the port to a complete different
>>>
>> switch
>>
>>> and router, but still no luck. My carrier double checked everything
>>>
>> over and
>>
>>> over and with their reputation it's most unlikely they would be
>>>
>> failing
>>
>>> here.
>>>
>>> Now the load is on IP level when issuing a 'sh proc cpu' command and
>>>
>> NOT on
>>
>>> the BGP level.
>>>
>>> I hope someone somehow has ever seen a situation alike and found the
>>>
>> issue,
>>
>>> or maybe has some recommendations on how to debug this. Maybe someone
>>>
>> knows
>>
>>> any additional configuration that may help me out here.
>>>
>>> The concerned config is:
>>> !
>>> vlan 20 name TRANSIT-BGP by port
>>> untagged ethe 1/1
>>> router-interface ve 20
>>> !
>>> interface ethernet 1/1
>>> route-only
>>> no spanning-tree
>>> sflow forwarding
>>> !
>>> interface ve 20
>>> ip address ???.???.???.??? 255.255.255.252
>>> no ip redirect
>>> ip arp-age 120
>>> !
>>>
>>> Thanks in advance for any help and efforts!
>>>
>>> Kind regards,
>>>
>>>
>>> Bardo Cornelissen.
>>> Caveo Internet BV
>>>
>>>
>>> _______________________________________________
>>> foundry-nsp mailing list
>>> foundry-nsp at puck.nether.net
>>> http://puck.nether.net/mailman/listinfo/foundry-nsp
>>>
>>>
>> --
>>
>> Met vriendelijke groet,
>>
>> Jeroen Wunnink,
>> EasyHosting B.V. Systeembeheerder
>> systeembeheer at easyhosting.nl
>>
>> telefoon:+31 (035) 6285455 Postbus 48
>> fax: +31 (035) 6838242 3755 ZG Eemnes
>>
>> http://www.easyhosting.nl
>> http://www.easycolocate.nl
>>
>>
>>
>
>
>
>
--
Met vriendelijke groet,
Jeroen Wunnink,
EasyHosting B.V. Systeembeheerder
systeembeheer at easyhosting.nl
telefoon:+31 (035) 6285455 Postbus 48
fax: +31 (035) 6838242 3755 ZG Eemnes
http://www.easyhosting.nl
http://www.easycolocate.nl
More information about the foundry-nsp
mailing list