[c-nsp] OSPF and iBGP session drops between 3640s

[Steve Bertrand]

Robert Johnson wrote:
> Hello list,
> I have a small network with four 3640s. Each router has 128/32MB ram, and a
> single FE interface connected to a catalyst 2924. Two of the routers are
> running BGP, each with a session to a (single) other provider, and a session
> between themselves. These are not carrying full tables. All four routers are
> running OSPF between each other. The problem is that occasionally (from once
> a week to 3x/day) OSPF neighbor relationships will bounce due to hello
> timers expiring. Just recently the iBGP session between two of the routers
> also bounced.
> 
> There do not appear to be any layer 1 or 2 connectivity problems that would
> cause this behavior. However, CPU usage on the 3640s seems high- 30%
> sustained and up to 90% peak, with only 1-2k max PPS. Also, I'm seeing
> buffer misses and failures.
> 
> CEF is enabled. There are several relatively long access lists that are
> being processed, and the routers are doing QoS classifying and tagging at
> layers 2 and 3 for VoIP performance.
> 
> Without any major hardware changes, where do I begin here?

The first thing that I would do is remove all of the common ACL "deny"
statements, and route all of those blocks to the discard interface
instead. You could also request a peering session with Team Cymru, and
they will feed to you the invalid routes dynamically.

Then, perhaps a basic configuration to measure if there is
excessive/unnecessary traffic making it to the receive interface(s).

This is a very basic one that I generally manipulate. It will allow and
count all, except for dropping everything in -DENY. Basically, I use it
as a counter, and then tweak to shape and drop traffic as the router
gains operational experience.

I find these methods quite effective in preserving resources in
older/lower end routers.

class-map match-all COPP-NORMAL
 match access-group name COPP-NORMAL
class-map match-any COPP-DENY
 match access-group name COPP-DENY
class-map match-all COPP-ROUTING
 match access-group name COPP-ROUTING
class-map match-all COPP-REMAINING
 match access-group name COPP-CATCHALL
!
policy-map COPP
 class COPP-DENY
   police 8000 1500 1500 conform-action drop  exceed-action drop
 class COPP-ROUTING
   police 125000 1500 1500 conform-action transmit  exceed-action transmit
 class COPP-NORMAL
   police 15000 1500 1500 conform-action transmit  exceed-action transmit
 class COPP-CATCHALL
   police 8000 1500 1500 conform-action transmit  exceed-action transmit
 class class-default
   police 8000 1500 1500 conform-action transmit  exceed-action transmit
!
ip access-list extended COPP-DENY
 permit tcp any any fragments
 permit udp any any fragments
 permit icmp any any fragments
 permit ip any any fragments
ip access-list extended COPP-NORMAL
 permit icmp any any echo
 permit icmp any any echo-reply
 permit icmp any any ttl-exceeded
 permit icmp any any unreachable
 permit icmp any any port-unreachable
 permit icmp any any packet-too-big
 permit udp host x.x.x.x any eq snmp
 permit tcp x.x.x.x 0 0.0.7.255 any eq ssh
 permit tcp x.x.x.x 0 0.0.0.7 eq ssh any established
ip access-list extended COPP-CATCHALL
 permit ip any any
ip access-list extended COPP-ROUTING
 permit tcp any gt 1024 any eq bgp
 permit tcp any eq bgp any gt 1024 established
 permit ospf x.x.x.x 0.0.0.3 any precedence internet
 permit ospf any any precedence internet

control-plane
 service-policy input COPP

Hope this helps,

Steve