[j-nsp] High CPU Utilization

daniel daniel at claudius.demon.nl
Tue Mar 2 18:21:54 EST 2004


Milind,

On 13:20 Tue 02 Mar     , Milind Deshpande wrote:
> 
> 
> 
> 
> CPU Utilization on Router is showing very high, using "show task
> accounting" command, how to find which  process are causing high
> utilization.
> 
> 
> RouterA> show chassis sfm
>                      Temp  CPU Utilization (%)   Memory    Utilization (%)
> Slot State            (C)  Total  Interrupt      DRAM (MB) Heap     Buffer
>   0  Online            41     81          2        64        23         55
>   1  Online - Standby  45     75          0        64        22         47

this is the CPU on the SFMs. It's a little unusual to see the CPU on the standby
being this high. Is this continually the case ? For the active SFM, are you doing
any logging in your firewall filters (then syslog or then log, or possibly sampling ?)

If so, would be worth to reduce or possibly disable that. If neither of those are active
you'd probably want to open a case with TAC as they can guide you through enabling
profiling on the SFM to troubleshoot further. Just in case you're familiar with acccessing
the S-board; i would recommend against hacking around yourself as with profiling you
need to use some sensible settings as you'll otherwise do more harm than good (especially
when the CPU is already high..).

In regards to below info. This is all related to the CPU on the RE. Hence unrelated to
above output. Using task accounting as you did below is useful to troubleshoot high RPD
CPU. Pls make sure to disable the accounting again as you'll otherwise be waisting CPU
cycles. 


Hope this helps,
-Daniel

> 
> RouterA> show chassis routing-engine
> Routing Engine status:
>   Slot 0:
>     Current state                  Master
>     Election priority              Master
>     Temperature                 59 degrees C / 138 degrees F
>     DRAM                      2048 MB
>     Memory utilization          14 percent
>     CPU utilization:
>       User                      12 percent
>       Background                 0 percent
>       Kernel                    27 percent
>       Interrupt                 26 percent
>       Idle                      35 percent
>     Model                          RE-3.0
>     Serial ID                      P10865701798
>     Start time                     2003-08-28 00:26:23 IST
>     Uptime                         187 days, 12 hours, 43 minutes, 34
> seconds
>     Load averages:                 1 minute   5 minute  15 minute
>                                        0.73       0.92       0.96
> Routing Engine status:
>   Slot 1:
>     Current state                  Backup
>     Election priority              Backup
>     Temperature                 35 degrees C / 95 degrees F
>     DRAM                      2048 MB
>     Memory utilization          10 percent
>     CPU utilization:
>       User                       0 percent
>       Background                 0 percent
>       Kernel                     0 percent
>       Interrupt                  0 percent
>       Idle                     100 percent
>     Model                          RE-3.0
>     Serial ID                      P10865700570
>     Start time                     2003-08-28 08:03:04 IST
>     Uptime                         187 days, 4 hours, 46 minutes, 29
> seconds
> 
> RouterA>show task accounting
> Task accounting is enabled.
> 
> Task                       Started    User Time  System Time  Longest Run
> Scheduler                  3839795       37.160     1:21.856        0.000
> LMP Client                      40        0.000        0.000        0.000
> IF                              42        0.000        0.000        0.000
> INET6                           20        0.000        0.000        0.000
> INET                            20        0.000        0.000        0.000
> ISO                             20        0.000        0.000        0.000
> Memory                          27        0.000        0.001        0.000
> RPD Server.0.0.0.0+666          22        0.000        0.000        0.000
> RT                             578        0.024        0.035        0.002
> RouterDiscoveryServer           20        0.000        0.000        0.000
> ICMP                           102        0.001        0.002        0.000
> Router-Advertisement            40        0.000        0.000        0.000
> MLD                             40        0.000        0.000        0.000
> IGMP                           297        0.006        0.016        0.000
> PIM Recv                     20299        1.948        3.925        0.003
> OSPFv2 I/O./var/run/ppmd_     9886        0.108        0.215        0.000
> LDP Conn.97.251.242.24+11      541        0.009        0.019        0.000
> LDP Conn.97.251.242.23+11      545        0.008        0.020        0.000
> LDP Conn.97.251.242.15+38      495        0.007        0.019        0.000
> LDP Conn.97.251.242.13+29      495        0.009        0.017        0.000
> LDP Conn.97.251.242.17+26      495        0.008        0.018        0.000
> LDP Conn.97.251.242.16+24      495        0.007        0.018        0.000
> LDP Conn.97.251.242.14+44      495        0.008        0.017        0.000
> AA-WW                 4287        0.216        0.416        0.000
> PIM.BB                    3697        0.244        0.455        0.003
> PIM.master                    3309        0.256        0.296        0.002
> LDP Conn.97.251.242.12+40      495        0.007        0.018        0.000
> LDP Conn.97.251.242.18+28      495        0.008        0.017        0.000
> LDP Conn.97.251.242.11+28      495        0.009        0.020        0.000
> LDP Conn.97.251.224.72+64      495        0.009        0.021        0.000
> LDP Conn.97.251.224.70+64      495        0.012        0.023        0.000
> BGP RT Background              138        0.000        0.001        0.000
> BGP.0.0.0.0+179                 24        0.000        0.000        0.000
> LDP I/O.0.0.0.0+646           4492        0.112        0.246        0.000
> LDP                           6826        0.156        0.317        0.002
> AA-OSPF                   8550        0.135        0.257        0.003
> OSPF                         10403        0.180        0.324        0.007
> BGP_18101.97.251.224.14+1     6289        0.248        0.470        0.000
> BGP_18101.97.251.224.70+2     6823        0.271        0.509        0.001
> TED                             20        0.000        0.000        0.000
> MPLS                            20        0.000        0.000        0.000
> Resolve inet.3                   2        0.017        0.000        0.009
> KRT Resolve/IPv4                23        0.001        0.003        0.000
> KStat                      3586997     1:22.217     2:54.571        0.001
> KRT Request                     23        0.000        0.000        0.000
> KRT Ifstate                     12        0.000        0.000        0.000
> KRT                         159895        8.683       17.379        0.009
> Redirect                        29        0.000        0.000        0.000
> MGMT_Listen./var/run/rpd_       21        0.001        0.002        0.000
> SNMP Subagent./var/run/rp      261        0.012        0.025        0.002
> 
> 
> RouterA> show task accounting detail
> Task accounting is enabled.
> 
> Task                       Started    User Time  System Time  Longest Run
> Scheduler                  3623296       35.007     1:17.296        0.000
> LMP Client                      40        0.000        0.000        0.000
>                   Changes       40        0.000        0.000        0.000
> 
> IF                                    42        0.000        0.000
> 0.000
>                   Changes       42        0.000        0.000        0.000
> 
> INET6                             20        0.000        0.000        0.000
>                   Changes       20        0.000        0.000        0.000
> 
> INET                               20        0.000        0.000
> 0.000
>                   Changes       20        0.000        0.000        0.000
> 
> ISO                                 20        0.000        0.000
> 0.000
>                   Changes       20        0.000        0.000        0.000
> 
> Memory                          26        0.000        0.001        0.000
>                    Timers         26        0.000        0.001        0.000
> 
> RPD Server.0.0.0.0+666          22        0.000        0.000        0.000
>                   Changes       22        0.000        0.000        0.000
> 
> RT                                  522        0.022        0.032
> 0.002
>                   Changes         20        0.000        0.000        0.000
>                      Jobs           502        0.022        0.032
> 0.002
> 
> RouterDiscoveryServer           20        0.000        0.000        0.000
>                   Changes       20        0.000        0.000        0.000
> 
> ICMP                           102        0.001        0.002        0.000
>                       I/O       82        0.001        0.002        0.000
>                   Changes       20        0.000        0.000        0.000
> 
> Router-Advertisement            40        0.000        0.000        0.000
>                   Changes       40        0.000        0.000        0.000
> 
> MLD                             40        0.000        0.000        0.000
>                   Changes       40        0.000        0.000        0.000
> 
> IGMP                           279        0.006        0.014        0.000
>                       I/O      226        0.005        0.013        0.000
>                   Changes       40        0.000        0.000        0.000
>                    Timers       13        0.000        0.000        0.000
> 
> PIM Recv                     19220        1.839        3.719        0.003
>                       I/O    19220        1.839        3.719        0.003
> 
> OSPFv2 I/O./var/run/ppmd_     9294        0.102        0.203        0.000
>                       I/O     9254        0.101        0.203        0.000
>                   Changes       40        0.000        0.000        0.000
> 
> LDP Conn.97.251.242.24+11      512        0.008        0.018        0.000
>                       I/O      178        0.001        0.003        0.000
>                      Jobs      334        0.007        0.014        0.000
> 
> LDP Conn.97.251.242.23+11      516        0.008        0.018        0.000
>                       I/O      180        0.001        0.003        0.000
>                      Jobs      336        0.007        0.015        0.000
> 
> LDP Conn.97.251.242.15+38      468        0.007        0.018        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.006        0.014        0.000
> 
> LDP Conn.97.251.242.13+29      468        0.008        0.016        0.000
>                       I/O      156        0.001        0.002        0.000
>                      Jobs      312        0.006        0.013        0.000
> 
> LDP Conn.97.251.242.17+26      468        0.008        0.016        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.006        0.013        0.000
> 
> LDP Conn.97.251.242.16+24      468        0.007        0.017        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.006        0.014        0.000
> 
> LDP Conn.97.251.242.14+44      468        0.007        0.015        0.000
>                       I/O      156        0.001        0.002        0.000
>                      Jobs      312        0.006        0.013        0.000
> 
> PIM.AA                 4055        0.205        0.392        0.000
>                   Changes       42        0.000        0.000        0.000
>                      Jobs     2533        0.173        0.323        0.000
>                    Timers     1480        0.031        0.068        0.000
> 
> PIM.BB                    3489        0.224        0.427        0.003
>                   Changes       42        0.000        0.000        0.000
>                      Jobs      425        0.090        0.101        0.001
>                    Timers     3022        0.134        0.325        0.003
> 
> PIM.master                    3126        0.241        0.283        0.002
>                   Changes       42        0.000        0.000        0.000
>                      Jobs      358        0.189        0.179        0.002
>                    Timers     2726        0.051        0.103        0.000
> 
> LDP Conn.97.251.242.12+40      468        0.007        0.017        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.006        0.014        0.000
> 
> LDP Conn.97.251.242.18+28      468        0.007        0.016        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.006        0.013        0.000
> 
> LDP Conn.97.251.242.11+28      468        0.008        0.018        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.007        0.015        0.000
> 
> LDP Conn.97.251.224.72+64      468        0.008        0.020        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.007        0.016        0.000
> 
> LDP Conn.97.251.224.70+64      468        0.011        0.022        0.000
>                       I/O      156        0.001        0.003        0.000
>                      Jobs      312        0.010        0.018        0.000
> 
> BGP RT Background              130        0.000        0.001        0.000
>                    Timers      130        0.000        0.001        0.000
> 
> BGP.0.0.0.0+179                 24        0.000        0.000        0.000
>                   Changes       20        0.000        0.000        0.000
>                      Jobs        4        0.000        0.000        0.000
> 
> LDP I/O.0.0.0.0+646           4249        0.106        0.234        0.000
>                       I/O     4229        0.106        0.233        0.000
>                   Changes       20        0.000        0.000        0.000
> 
> LDP                           6448        0.148        0.299        0.002
>                      Jobs        2        0.000        0.000        0.000
>                    Timers     6446        0.147        0.299        0.002
> 
> AA-OSPF                   8080        0.126        0.237        0.003
>                      Jobs     2113        0.049        0.082        0.003
>                    Timers     5967        0.076        0.155        0.000
> 
> OSPF                          9760        0.172        0.307        0.007
>                      Jobs     2335        0.084        0.109        0.007
>                    Timers     7425        0.087        0.197        0.001
> 
> BGP_18101.97.251.224.14+1     6016        0.237        0.450        0.000
>                       I/O     2981        0.027        0.057        0.000
>                      Jobs     2981        0.208        0.389        0.000
>                    Timers       54        0.001        0.003        0.000
> 
> BGP_18101.97.251.224.70+2     6534        0.261        0.486        0.001
>                       I/O     3240        0.028        0.058        0.000
>                      Jobs     3240        0.232        0.426        0.001
>                    Timers       54        0.000        0.001        0.000
> 
> TED                             20        0.000        0.000        0.000
>                   Changes       20        0.000        0.000        0.000
> 
> MPLS                            20        0.000        0.000        0.000
>                   Changes       20        0.000        0.000        0.000
> 
> Resolve inet.3                   2        0.017        0.000        0.009
>                      Jobs        2        0.017        0.000        0.009
> 
> KRT Resolve/IPv4                23        0.001        0.003        0.000
>                       I/O       23        0.001        0.003        0.000
> 
> KStat                      3384571     1:17.467     2:44.885        0.001
>                       I/O  2807075       28.082     1:00.802        0.000
>                      Jobs   577496       49.384     1:44.083        0.001
> 
> KRT Request                     22        0.000        0.000        0.000
>                      Jobs       22        0.000        0.000        0.000
> 
> KRT Ifstate                     12        0.000        0.000        0.000
>                       I/O       12        0.000        0.000        0.000
> 
> KRT                         150828        8.181       16.411        0.009
>                   Changes       20        0.000        0.000        0.000
>                      Jobs   150782        8.181       16.411        0.009
>                    Timers       26        0.000        0.000        0.000
> 
> Redirect                        29        0.000        0.000        0.000
>                   Changes       20        0.000        0.000        0.000
>                    Timers        9        0.000        0.000        0.000
> 
> MGMT_Listen./var/run/rpd_       20        0.001        0.002        0.000
>                       I/O       20        0.001        0.002        0.000
> 
> SNMP Subagent./var/run/rp      261        0.012        0.025        0.002
>                       I/O      252        0.007        0.011        0.000
>                    Timers        9        0.004        0.013        0.002
> 
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> http://puck.nether.net/mailman/listinfo/juniper-nsp


More information about the juniper-nsp mailing list