[cisco-bba] cpu load due to PPP IPCP (Add Route)

Tassos Chatzithomaoglou achatz at forthnet.gr
Wed Jun 22 12:46:46 EDT 2005


Hi Dennis,

Here is another strange thing i noticed today which i don't know if it has something to do with the 
IPCP * cpu load problem.

Check out the multiple AAA in the following va interface.

router#sh int vi704
Virtual-Access704 is up, line protocol is up
   Hardware is Virtual Access interface
   Interface is unnumbered. Using address of Loopback0 (194.219.252.144)
   MTU 1500 bytes, BW 622080 Kbit, DLY 100000 usec,
      reliability 255/255, txload 1/255, rxload 1/255
   Encapsulation PPP, LCP Open, multilink Closed
   Closed: IPCP
   PPPoVPDN vaccess, cloned from AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, 
AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, AAA, Virtual-Template1
   Vaccess status 0x44
   Protocol l2tp, tunnel id 807, session id 24697, loopback not set
   Keepalive set (20 sec)
   DTR is pulsed for 5 seconds on reset idle 00:02:42
   Last input 00:00:09, output never, output hang never
   Last clearing of "show interface" counters 02:10:17
   Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
   Queueing strategy: fifo
   Output queue: 0/40 (size/max)
   30 second input rate 0 bits/sec, 0 packets/sec
   30 second output rate 0 bits/sec, 0 packets/sec
      1187 packets input, 19310 bytes, 0 no buffer
      Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
      0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
      1189 packets output, 19333 bytes, 0 underruns
      0 output errors, 0 collisions, 0 interface resets
      0 output buffer failures, 0 output buffers swapped out
      0 carrier transitions


Although this particular user has got an ip address (shown with "sh ip int vi704"), he hasn't got 
his per-user acl through radius, because radius was returning a wrong attribute.

Jun 22 16:14:37: %AAA-3-PARSEERR: Error(2) parser is unable to parse permit ip x.x.40.24 0.0.7 any 
per-user command



Tassos Chatzithomaoglou wrote on 21/6/2005 7:32:

> This 7200 is using 12.3(12b)...is the correct ios version ;-)
> 
> Tassos Chatzithomaoglou wrote on 21/6/2005 7:29:
> 
>> Hi Dennis,
>>
>> That's exactly what i also said to my colleague who did "clear ppp 
>> queues".
>> But according to his sayings and to our graphs the cpu problem was 
>> resolved just afterwards.
>> I don't know if this was just a coincidence.
>>
>> Also after a while (or maybe at the same time but we hadn't noticed 
>> it) we had another problem.
>> The vpdn tunnel which was terminating to this 7200 couldn't accept any 
>> more L2TP sessions. It was like incoming ppp was blocked (continuesly 
>> "LCP: O CONFREQ" with no answer from the peer). We had to tear down 
>> the tunnel manually in order to make it work again.
>>
>> This 7200 is using 12.2(12b).
>>
>> Dennis Peng wrote on 21/6/2005 2:36:
>>
>>> What version was this? I'm not sure how this would happen. It's
>>> strange that "clear ppp queues" did anything since that should only
>>> clear the statistics (counters). If this occurs again, can you grab
>>> "debug ppp events"?
>>>
>>> Dennis
>>>
>>> Tassos Chatzithomaoglou [achatz at forthnet.gr] wrote:
>>>
>>>> Today we had the cpu of one of our 7200's reaching 100%. We found 
>>>> that this was due to the "IPCP Skipped - Add Route:" which was 
>>>> constantly increasing (we supply routes per user through radius). 
>>>> After we did a "clear ppp queue", it returned to normal.
>>>>
>>>> Any idea what might have happened?
>>>>
>>>> router#sh proc cpu sorted | e 0.00
>>>> CPU utilization for five seconds: 99%/70%; one minute: 99%; five 
>>>> minutes: 99%
>>>>  PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process
>>>>   77    14284408   3108213       4595 20.47% 21.03% 21.11%   0 PPP 
>>>> IPCP         <---
>>>>  163    84786456   2270413      37344  1.86%  0.79%  0.64%   0 
>>>> VTEMPLATE Backgr
>>>>  148    75342260   4779805      15762  1.19%  0.76%  0.76%   0 IGMP 
>>>> Input
>>>>  150    92048440  14330863       6423  1.12%  1.02%  1.00%   0 PIM 
>>>> Process
>>>>   31    42891736   4708264       9109  1.12%  0.42%  0.38%   0 
>>>> Per-Second Jobs
>>>>  156    32672884 184875306        176  1.04%  1.03%  1.04%   0 PPP 
>>>> Events
>>>>   79    84550988   7024517      12036  0.97%  0.69%  0.67%   0 CEF 
>>>> process
>>>>  129    48902960    889685      54966  0.59%  0.53%  0.53%   0 
>>>> Compute load avg
>>>>   50    56443568 247614090        227  0.29%  0.38%  0.37%   0 IP Input
>>>>    5    32496292   1692766      19197  0.22%  0.17%  0.22%   0 Check 
>>>> heaps
>>>>  167    18315212  18625936        983  0.22%  0.21%  0.24%   0 OSPF 
>>>> Router
>>>>   63     4082388   5244432        778  0.14%  0.12%  0.11%   0 IP 
>>>> Background
>>>>  160     1580920 997329231          1  0.14%  0.04%  0.01%   0 SAA 
>>>> Event Proces
>>>>   76     9643460   1590124       6064  0.07%  1.58%  1.62%   0 PPP 
>>>> IP Route
>>>>
>>>>
>>>> router#sh ppp queues
>>>>
>>>> 5 Event Queues
>>>>                  size   max      kicks     starts    false   
>>>> suspends ticks(ms)
>>>>  1 PPP Events       0     9      27514      27512        7          
>>>> 0        20
>>>>  5 PPP Bind         0     2        445        445        0          
>>>> 0        20
>>>>  3 PPP IPCP         0     0          0          0        0          
>>>> 0        20
>>>>  2 PPP IP Route     0     3        226        226        0          
>>>> 0       100
>>>>  4 PPP Hooks        0     1          1          1        0          
>>>> 0        20
>>>>
>>>>  30 Events
>>>>  #  Q Name                 Events   Queued  MaxQueued Suspends  
>>>> usec/evt max/evt
>>>>  1  1 Setup                     1        0        1        0       
>>>> 654       654
>>>>  2  1 Free PPP                661        0        3        0       
>>>> 111      3172
>>>>  3* 1 Timer                 28072 4294967290 4294967295        
>>>> 1      1047      9575
>>>>  4  1 Cstate                   18        0        3        0      
>>>> 1041      4249
>>>>  5  1 Restart CP                0        0        0        0         
>>>> 0         0
>>>>  6    UNREGISTERED
>>>>  7  1 Hard Disc               409        0        2        0       
>>>> 893      2466
>>>>  8  1 Soft Disc                77        0        2        2      
>>>> 3174      9366
>>>>  9  1 Packet                    2 4294967294 4294967295        
>>>> 0       380       475
>>>> 10  1 Auth Packet               0        0        0        0         
>>>> 0         0
>>>> 11* 3 IPCP Packet               0        0        0        0         
>>>> 0         0
>>>> 12* 2 Add Route                 0        0        0        0         
>>>> 0         0
>>>> 13* 2 Remove Route            247        0        3        0       
>>>> 293      1358
>>>> 14* 2 Remove Top                0        0        0        0         
>>>> 0         0
>>>> 15* 2 Remove Flag               0        0        0        0         
>>>> 0         0
>>>> 16* 2 Neg Addr Add              0        0        0        0         
>>>> 0         0
>>>> 17* 2 Neg Addr Remov            0        0        0        0         
>>>> 0         0
>>>> 18* 2 DHCP Opt Add              0        0        0        0         
>>>> 0         0
>>>> 19* 2 DHCP Opt Del              0        0        0        0         
>>>> 0         0
>>>> 20* 2 IPCP UP                   0        0        0        0         
>>>> 0         0
>>>> 21  1 Set LCP Open            448        0        2        0       
>>>> 497      1794
>>>> 22  1 Virtualize                0        0        0        0         
>>>> 0         0
>>>> 23  1 Redirect                  0        0        0        0         
>>>> 0         0
>>>> 24  1 Forwarded                 1        0        1        0       
>>>> 317       317
>>>> 25  4 Hook                      1        0        1        0        
>>>> 17        17
>>>> 26  1 AAA Response              3        0        2        0       
>>>> 994      1568
>>>> 27  1 Static Bind               1        0        1        0       
>>>> 379       379
>>>> 28  5 Dynamic Bind            448        0        2        0       
>>>> 585      1893
>>>> 29  1 Bound                     0        0        0        0         
>>>> 0         0
>>>> 30  1 Virtual Profil            0        0        0        0         
>>>> 0         0
>>>>
>>>> Pre-processed Requests  LCP:         0  IPCP:         0
>>>> PPP Request Failures: 0
>>>> PPP Hold Queue Drops: 14248
>>>> IPCP Skipped - Add Route:         545  AddrAdd:         0  
>>>> AddrRem:         0
>>>> PPP Handles:  alloc[5719489] free[5717015] outstanding[2474]
>>>>
>>>> _______________________________________________
>>>> cisco-bba mailing list
>>>> cisco-bba at puck.nether.net
>>>> https://puck.nether.net/mailman/listinfo/cisco-bba
>>>
>>>
>>>
>>>
>>
> 



More information about the cisco-bba mailing list