[c-nsp] High CPU usage on 3640

Robert Johnson fasterfourier at gmail.com
Wed Jul 29 10:30:01 EDT 2009


Hello list,
I would appreciate any help with going through the following configuration
and making suggestions to reduce CPU usage on this router. The example
router is a 3640 with a single FE interface run to a 2924 switch. It is
loaded at peak times with less than 2000 PPS and 9 Mbps aggregate on the FE
interface. The bulk of the traffic is flowing between the f0/0.300 and
f0/0.302 interfaces. There is some ACL checking and QOS marking going on for
both of these interfaces in multiple directions. This is done to ensure
voice priority on wireless links that use 802.1p to form queues. All (for
the most part) of the CPU usage is due to interrupts.

Suggestions?

router>sho proc cpu hist

router   02:15:17 PM Wednesday Jul 29 2009 UTC


    5555555555544444444444444444444444443333344444333333333344
    2111114444466666666666666611111888882222200000222228888800
100
 90
 80
 70
 60
 50 **************************     *****
 40 ************************************     *****     *********
 30 ************************************************************
 20 ************************************************************
 10 ************************************************************
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5
               CPU% per second (last 60 seconds)

    5565666666777666566545466566666666678768655666567666545446
    1106144112101388799093804673397940104983291383955552869770
100
 90
 80                                     ** *        *
 70           *** ** **       *  ***   *##*#    *  ****
 60   ***#****###****#*    ******###****####** ****##**** *  * *
 50 **#*##############******##################**#########****#**
 40 ############################################################
 30 ############################################################
 20 ############################################################
 10 ############################################################
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5
               CPU% per minute (last 60 minutes)
              * = maximum CPU%   # = average CPU%

    86543223342226394789888887553333234323345777877776544 3234222124411223
    4660478341898942827940584834138343317626230724265716090656046791268613
100                    *
 90                *  **  **
 80 *              *  ********                 **  **
 70 **           * * *********               *********
 60 ***          * * *********               *********
 50 #**          * ***########**           ****###*##***     *
 40 ##**     *   * **#########**  *   *  * ***########***   **     **     *
 30 ###**************##########***** ********#########*** ****  * ***  * **
 20 ####**********##############***********############** *****************
 10 ####################################################*****#***####**#*#**
   0....5....1....1....2....2....3....3....4....4....5....5....6....6....7.
             0    5    0    5    0    5    0    5    0    5    0    5    0
                   CPU% per hour (last 72 hours)
                  * = maximum CPU%   # = average CPU%


Configuration:

version 12.4
service timestamps debug uptime
service timestamps log uptime
service password-encryption
!
hostname router
!
boot-start-marker
boot system flash c3640-jk9o3s-mz.124-3.bin
boot-end-marker
!
no logging console
enable secret 5 *****
!
no aaa new-model
!
resource policy
!
ip subnet-zero
no ip source-route
!
ip cef
!
class-map match-all assure
 match ip dscp af31
class-map match-all critical
 match ip dscp cs6
class-map match-all expedite
 match ip dscp ef
class-map match-any rtp
 match ip rtp 13456 13462
 match ip rtp 13556 13560
 match ip rtp 13656 13660
 match ip rtp 13756 13760
class-map match-all sip
 match protocol sip
class-map match-all voice
 match packet length min 1 max 200
 match class-map rtp
!
!
policy-map output-cos
 class expedite
  set cos 6
 class assure
  set cos 5
 class critical
  set cos 7
policy-map input-mark
 class sip
  set ip dscp af31
 class voice
  set dscp ef
!
buffers tune automatic
!
interface FastEthernet0/0
 description Trunk to cat2924
 no ip address
 full-duplex
!
interface FastEthernet0/0.5
 description Switch management segment
 encapsulation dot1Q 5
 ip address 10.1.5.254 255.255.255.0
 ip access-group mgmt-only in
 ip access-group mgmt-only out
 no snmp trap link-status
!
interface FastEthernet0/0.15
 description AP management segment
 encapsulation dot1Q 15
 ip address 10.1.15.254 255.255.255.0
 ip access-group mgmt-only in
 ip access-group mgmt-only out
 no snmp trap link-status
!
interface FastEthernet0/0.25
 description CTM management segment
 encapsulation dot1Q 25
 ip address 10.1.25.254 255.255.255.0
 ip access-group mgmt-only in
 ip access-group mgmt-only out
 no snmp trap link-status
!
interface FastEthernet0/0.35
 description UPS management segment
 encapsulation dot1Q 35
 ip address 10.1.35.254 255.255.255.0
 ip access-group mgmt-only in
 ip access-group mgmt-only out
 no snmp trap link-status
!
interface FastEthernet0/0.50
 description Management link to anotherrouter
 bandwidth 9850
 encapsulation dot1Q 50
 ip address 10.1.50.254 255.255.255.0
 ip access-group mgmt-only in
 ip access-group mgmt-only out
 ip ospf message-digest-key 1 md5 7 ****
 ip ospf hello-interval 1
 ip ospf dead-interval 5
 no snmp trap link-status
!
interface FastEthernet0/0.51
 description Management link to yetanotherrouter
 encapsulation dot1Q 51
 ip address 10.1.51.254 255.255.255.0
 ip access-group mgmt-only in
 ip access-group mgmt-only out
 ip ospf message-digest-key 1 md5 7 ****
 ip ospf hello-interval 1
 ip ospf dead-interval 5
 no snmp trap link-status
!
interface FastEthernet0/0.52
 description Management link to stillanotherrouter
 bandwidth 10610
 encapsulation dot1Q 52
 ip address 10.1.52.254 255.255.255.0
 ip access-group mgmt-only in
 ip access-group mgmt-only out
 no snmp trap link-status
!
interface FastEthernet0/0.300
 description Production traffic link to anotherrouter
 bandwidth 9850
 encapsulation dot1Q 300
 ip address xxx.xxx.xxx.xxx 255.255.255.252
 ip ospf message-digest-key 10 md5 7 ****
 ip ospf dead-interval minimal hello-multiplier 4
 no snmp trap link-status
 service-policy output output-cos
!
interface FastEthernet0/0.301
 description Production traffic link to yetanotherrouter
 encapsulation dot1Q 301
 ip address xxx.xxx.xxx.xxx 255.255.255.252
 ip ospf message-digest-key 10 md5 7 ****
 ip ospf dead-interval minimal hello-multiplier 4
 no snmp trap link-status
 service-policy output output-cos
!
interface FastEthernet0/0.302
 description Production traffic link to stillanotherrouter
 bandwidth 10610
 encapsulation dot1Q 302
 ip address xxx.xxx.xxx.xxx 255.255.255.252
 ip access-group internet-edge-ingress in
 no snmp trap link-status
 service-policy input input-mark
 service-policy output output-cos
!
interface FastEthernet0/0.500
 description Customer access subnet
 encapsulation dot1Q 500
 ip address xxx.xxx.xxx.xxx 255.255.255.240
 ip verify unicast reverse-path
 rate-limit input access-group 100 768000 10000 200000 conform-action
transmit exceed-action drop
 rate-limit output access-group 100 768000 40000000 80000000 conform-action
transmit exceed-action drop
 no snmp trap link-status
 service-policy output output-cos
!
router ospf 1000
 log-adjacency-changes
 area 0.0.0.0 authentication message-digest
 passive-interface default
 no passive-interface FastEthernet0/0.300
 no passive-interface FastEthernet0/0.301
 network xxx.xxx.xxx.xxx 0.0.0.63 area 0.0.0.0
 network xxx.xxx.xxx.xxx 0.0.0.63 area 0.0.0.0
 network xxx.xxx.xxx.xxx 0.0.0.63 area 0.0.0.0
 network xxx.xxx.xxx.xxx 0.0.0.63 area 0.0.0.0
 default-information originate metric-type 1
!
router ospf 100
 log-adjacency-changes
 area 10.0.0.0 authentication message-digest
 area 10.0.0.0 stub no-summary
 passive-interface default
 no passive-interface FastEthernet0/0.50
 no passive-interface FastEthernet0/0.51
 network 10.0.0.0 0.255.255.255 area 10.0.0.0
!
router bgp yyyy
 no synchronization
 bgp log-neighbor-changes
 network xxx.xxx.xxx.xxx mask 255.255.255.192
 network xxx.xxx.xxx.xxx mask 255.255.255.192
 network xxx.xxx.xxx.xxx mask 255.255.255.192
 network xxx.xxx.xxx.xxx mask 255.255.255.192
 aggregate-address xxx.xxx.xxx.xxx 255.255.255.192 as-set summary-only
 aggregate-address xxx.xxx.xxx.xxx 255.255.255.192 as-set summary-only
 aggregate-address xxx.xxx.xxx.xxx 255.255.255.192 as-set summary-only
 aggregate-address xxx.xxx.xxx.xxx 255.255.255.192 as-set summary-only
 redistribute ospf 1000
 neighbor xxx.xxx.xxx.xxx remote-as xxxx
 neighbor xxx.xxx.xxx.xxx route-map pri-map out
 neighbor xxx.xxx.xxx.xxx remote-as yyyy
 neighbor xxx.xxx.xxx.xxx next-hop-self
 no auto-summary
!
no ip http server
no ip http secure-server
ip classless
!
ip access-list standard mgmt-only
 permit 10.0.0.0 0.255.255.255
 permit 192.168.101.0 0.0.0.255
!
ip access-list extended internet-edge-ingress
 deny   ip 10.0.0.0 0.255.255.255 any
 deny   ip 172.16.0.0 0.15.255.255 any
 deny   ip 192.168.0.0 0.0.255.255 any
 deny   ip 127.0.0.0 0.0.255.255 any
 deny   ip 224.0.0.0 31.255.255.255 any
 deny   ip 169.254.0.0 0.0.255.255 any
 deny   ip xxx.xxx.xxx.xxx 0.0.0.63 any
 deny   ip xxx.xxx.xxx.xxx 0.0.0.63 any
 deny   ip xxx.xxx.xxx.xxx 0.0.0.63 any
 deny   ip xxx.xxx.xxx.xxx 0.0.0.63 any
 permit ip any any
logging facility local5
logging 10.3.40.105
access-list 1 permit xxx.xxx.xxx.xxx 0.0.0.63
access-list 1 permit xxx.xxx.xxx.xxx 0.0.0.63
access-list 2 permit xxx.xxx.xxx.xxx 0.0.0.63
access-list 2 permit xxx.xxx.xxx.xxx 0.0.0.63
access-list 100 permit ip host xxx.xxx.xxx.xxx any
access-list 100 permit ip any host xxx.xxx.xxx.xxx
snmp-server community 3640stats RO mgmt-only
!
route-map pri-map permit 10
 match ip address 1
!
route-map pri-map permit 20
 match ip address 2
!
control-plane
!
!
banner login  Property of xxxx. Unauthorized access attempts will be
prosecuted. 
!
line con 0
 password 7 ****
 login
line aux 0
 password 7 ****
 login
line vty 0 4
 access-class mgmt-only in
 password 7 ****
 login
!
ntp clock-period 17179619
ntp server 10.3.40.105
!
end


More information about the cisco-nsp mailing list