[c-nsp] High CPU = CEF Disabled

Thu Jan 13 04:21:48 EST 2005

Hi,

on my 
cisco RPM-PR (NPE400) processor with 491520K/32768K bytes of memory.
R7000 CPU at 350Mhz, Implementation 39, Rev 3.3, 256KB L2, 4096KB L3 Cache

I see:

CPU utilization for five seconds: 98%/53%; one minute: 97%; five minutes: 97%
 PID Runtime(ms)   Invoked      uSecs   5Sec   1Min   5Min TTY Process 
  50    15840456  97495649        162 31.00% 29.71% 28.42%   0 IP Input         
  94     4433972  82246793         53  5.48%  5.93%  5.90%   0 Tag Input        
  95     1027404  77573922         13  4.42%  4.98%  4.76%   0 Tag Input        
 140      184124    152332       1208  1.63%  0.41%  0.32%   0 RIP Router       
 101       52172     21724       2401  0.40%  0.09%  0.07%   0 PPPoA Manager    
 123       93684   1622773         57  0.40%  0.22%  0.22%   0 PPP Events       
 142       37576     39002        963  0.32%  0.08%  0.06%   0 RIP Send         
  39       36516     92163        396  0.24%  0.18%  0.18%   0 CEF process      
 141       59944    171202        350  0.08%  0.30%  0.26%   0 BGP Router       
  41        7544      9950        758  0.08%  0.04%  0.03%   0 Compute load avg 
 122        6168   1525483          4  0.08%  0.06%  0.03%   0 PPP manager      
  35       53792    582543         92  0.08%  0.07%  0.07%   0 ATM OAM Input    
  27        5032     49723        101  0.08%  0.15%  0.13%   0 Per-Second Jobs  
 146       57704     75894        760  0.08%  0.27%  0.20%   0 BGP I/O          
  14           0        16          0  0.00%  0.00%  0.00%   0 AAA high-capacit 

RPM#              sh proc cpu his

    9999999999999999999999999999999999999999999999999999999999
    8888888888888888888877777777777777777777888887777788888555
100 **********************************************************
 90 **********************************************************
 80 **********************************************************
 70 **********************************************************
 60 **********************************************************
 50 **********************************************************
 40 **********************************************************
 30 **********************************************************
 20 **********************************************************
 10 **********************************************************
   0....5....1....1....2....2....3....3....4....4....5....5....
             0    5    0    5    0    5    0    5    0    5    
               CPU% per second (last 60 seconds)

which causes my :

Jan 13 09:10:20 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.41 4/0 (hold
 time expired) 0 bytes
Jan 13 09:10:49 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.41 Up
Jan 13 09:11:06 UTC: %TAC+: no address for get_server
Jan 13 09:11:06 UTC: %TAC+: no address for get_server
Jan 13 09:11:06 UTC: %TAC+: no address for get_server
Jan 13 09:11:06 UTC: %TAC+: no address for get_server
Jan 13 09:11:06 UTC: %TAC+: no address for get_server
Jan 13 09:11:06 UTC: %TAC+: no address for get_server
Jan 13 09:11:15 UTC: %SYS-5-CONFIG_I: Configured from console by aossama on vty2
 (62.240.110.232)
Jan 13 09:11:16 UTC: %TAC+: no address for get_server
Jan 13 09:11:16 UTC: %TAC+: no address for get_server
Jan 13 09:11:16 UTC: %TAC+: no address for get_server
Jan 13 09:11:32 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.10 4/0 (hold
 time expired) 0 bytes
Jan 13 09:11:59 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.10 Up
Jan 13 09:12:13 UTC: %TAC+: no address for get_server
Jan 13 09:12:13 UTC: %TAC+: no address for get_server
Jan 13 09:12:16 UTC: %TAC+: no address for get_server
Jan 13 09:12:16 UTC: %TAC+: no address for get_server
Jan 13 09:12:18 UTC: %TAC+: no address for get_server
Jan 13 09:12:30 UTC: %SYS-5-CONFIG_I: Configured from console by aossama on vty2
 (62.240.110.232)
Jan 13 09:12:35 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.26 Down BGP Notificatio
n sent
Jan 13 09:12:35 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.26 4/0 (hold
 time expired) 0 bytes
Jan 13 09:12:55 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.40 Down BGP Notificatio
n sent
Jan 13 09:12:55 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.40 4/0 (hold
 time expired) 0 bytes
Jan 13 09:13:00 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.26 Up
Jan 13 09:13:08 UTC: %TAC+: no address for get_server
Jan 13 09:13:08 UTC: %TAC+: no address for get_server
Jan 13 09:13:08 UTC: %TAC+: no address for get_server
Jan 13 09:13:08 UTC: %TAC+: no address for get_server
Jan 13 09:13:08 UTC: %TAC+: no address for get_server
Jan 13 09:13:08 UTC: %TAC+: no address for get_server
Jan 13 09:13:16 UTC: %TAC+: no address for get_server
Jan 13 09:13:16 UTC: %TAC+: no address for get_server
Jan 13 09:13:16 UTC: %TAC+: no address for get_server
Jan 13 09:13:22 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.40 Up
Jan 13 09:13:50 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.41 Down BGP Notificatio
n sent
Jan 13 09:13:50 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.41 4/0 (hold
 time expired) 0 bytes
Jan 13 09:14:39 UTC: %RPM_VIRTUAL_PORT-3-VRTLINFO: Starting bulk updates ...
Jan 13 09:14:40 UTC: %RPM_VIRTUAL_PORT-3-VRTLINFO: ... Completed Bulk updates
Jan 13 09:14:51 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.41 Up
Jan 13 09:15:00 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.10 Down BGP Notificatio
n sent
Jan 13 09:15:00 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.10 4/0 (hold
 time expired) 0 bytes
Jan 13 09:15:06 UTC: %TAC+: no address for get_server
Jan 13 09:15:06 UTC: %TAC+: no address for get_server
Jan 13 09:15:06 UTC: %TAC+: no address for get_server
Jan 13 09:15:06 UTC: %TAC+: no address for get_server
Jan 13 09:15:06 UTC: %TAC+: no address for get_server
Jan 13 09:15:06 UTC: %TAC+: no address for get_server
Jan 13 09:15:13 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface Virtual-Acc
ess157, changed state to down
Jan 13 09:15:16 UTC: %TAC+: no address for get_server
Jan 13 09:15:16 UTC: %TAC+: no address for get_server
Jan 13 09:15:16 UTC: %TAC+: no address for get_server
Jan 13 09:15:23 UTC: %SYS-5-CONFIG_I: Configured from console by xxx on vty2
 (62.240.110.232)
Jan 13 09:15:33 UTC: %TAC+: no address for get_server
Jan 13 09:16:01 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.26 Down BGP Notificatio
n sent
Jan 13 09:16:01 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.26 4/0 (hold
 time expired) 0 bytes
Jan 13 09:16:23 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.40 Down BGP Notificatio
n sent
Jan 13 09:16:23 UTC: %BGP-3-NOTIFICATION: sent to neighbor 172.31.1.40 4/0 (hold
 time expired) 0 bytes
Jan 13 09:16:28 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.26 Up
Jan 13 09:16:52 UTC: %BGP-5-ADJCHANGE: neighbor 172.31.1.40 Up

and i can see that :

Switch1.1 is up, line protocol is up    < all traffic goes here
  Interface is unnumbered. Using address of Loopback0 (172.31.1.2)
  Broadcast address is 255.255.255.255
  MTU is 4470 bytes
  Helper address is not set
  Directed broadcast forwarding is disabled
  Multicast reserved groups joined: 224.0.0.5 224.0.0.6
  Outgoing access list is not set
  Inbound  access list is not set
  Proxy ARP is enabled
  Local Proxy ARP is disabled
  Security level is default
  Split horizon is disabled
  ICMP redirects are always sent
  ICMP unreachables are never sent
  ICMP mask replies are never sent
  IP fast switching is enabled
  IP fast switching on the same interface is enabled
  IP Flow switching is disabled
  IP CEF switching is disabled   <-- i guess thats it
  IP Feature Fast switching turbo vector
  IP Null turbo vector
  IP multicast fast switching is disabled

which is weird cos i see:

RPM1#sh run | in ip cef
ip cef

What could be breaing my CEF, so it punts everything to CPU ?