[c-nsp] Memory exhaustion causing BGP sessions to randomly fail

[Oliver Boehmer (oboehmer)]

Drew Weaver <> wrote on Friday, February 09, 2007 4:29 PM:

>     Is it plausable that memory exhaustion on a 12000 series router
> can cause bgp sessions to seemingly at random start opening/closing.
> 
> Let me give you an example.
> 
> Today we had an issue where one of our interfaces (a 100mbps
> interface) was getting DDOS attacked, the router was trying to push
> 300Mbps down a 100Mbps interface.
> 
> During this time, the bgp session with one of our upstream providers
> would:
[...]
> recieve approximately 30000 out of 206900 routes
> 
> then close again.
> 
> This happened repeatedly until the traffic on said fastethernet
> interface was mitigated.
> 
> I did not notice a log message stating why specifically the sessions
> were dying, nor did i see anything where it appeared to be out of RAM.
> 
> Has anyone experienced a similar issue on a 12k router?

I'm pretty sure you've suffered from buffer depletion on the inbound
OC12 linecard causing all sorts of issues, also affecting your BGP
session (which also runs over this connection).
[no, I did not see this in my crystal ball, this is based on the info
you've provided offline to me].

http://www.cisco.com/warp/public/63/ignored_nomem_18003.html#symptoms
describes this problem where your outbound Engine1 (with the 100Mbps
connection) exhausted all frfab buffers and signalled backpressure to
your ingress Engine2/OC12, which then also started buffering and
possibly exhausted its tofab buffers.

On the 12000, you *always* want to use QoS to control the queues on your
outbound interfaces, either by limiting the tx-queue or by using
random-detect.
You can also prevent backpressure to exhaust your ingress linecard's
tofab buffers by implementing tofab QoS ("rx-cos-slot ..") and use RED
there. See http://www.cisco.com/warp/public/63/mdrr_wred_overview.html &
http://www.cisco.com/warp/public/63/legacy_cos.html

	oli