[c-nsp] ASR1000 - Software Redundancy

[Antonio Soares]

Yes, lesson learned, no software redundancy at least with the RP1 which
memory maximum is 4GB which means 700MB usable...

In the meanwhile, I saw that it's possible to switch to the underlying OS
and we can do linux commands like top:

top - 03:50:16 up 12:22,  0 users,  load average: 0.21, 0.13, 0.09
Tasks: 136 total,   2 running, 134 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.0%us,  2.6%sy,  0.0%ni, 96.0%id,  0.0%wa,  0.3%hi,  0.0%si,
0.0%st
Mem:   3874968k total,  1707248k used,  2167720k free,   127152k buffers
Swap:        0k total,        0k used,        0k free,  1075788k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

25147 root      20   0 26784  14m  12m S  1.3  0.4   6:40.53 imand

23063 root      20   0 28008  10m 8136 S  1.0  0.3   4:51.46 cmand

25922 root      20   0 1916m 403m 142m R  0.7 10.7   9:42.53 linux_iosd-imag
(...)

We see lots of free memory so I suspect we can change the default values
that IOSd is able to allocate.


Regards,

Antonio Soares, CCIE #18473 (R&S/SP)
amsoares at netcabo.pt
http://www.ccie18473.net



-----Original Message-----
From: Mark Tinka [mailto:mtinka at globaltransit.net] 
Sent: quarta-feira, 1 de Fevereiro de 2012 02:04
To: cisco-nsp at puck.nether.net
Cc: Antonio Soares
Subject: Re: [c-nsp] ASR1000 - Software Redundancy

On Tuesday, January 31, 2012 11:38:53 PM Antonio Soares
wrote:

> The box has 4 GB of memory but the IOSd only allocates
> 1,7 GB. Is this dynamic ? How do we control this ?

We turned on software redundancy on our ASR1002's a couple of years back,
while they were running at least 3x full BGP feeds. This was still on IOS XE
2.6.

Over several weeks, the box run out of memory and crashed. 
We traced the issue back to the software redundancy + large memory
consumption due to BGP routing.

We disabled software redundancy and have never turned it on since. If we
want control plane redundancy, we buy the ASR1006, which is one of the
reasons we never buy the ASR1004. Only the ASR1002 (size, cost) and the
ASR1006 (redundant, high capacity).

If you have 4GB DRAM in the router, IOSd itself will take 2GB and the other
2GB will be used for internal purposes. If you have SSO turned on for
software redundancy, the 2GB that was allocated to IOSd will be halved
further to 1GB for the native IOSd, and another 1GB for the redundant IOSd. 
However, other internal processes would consume memory from the remaining
1GB of the native IOSd, leaving with you about 600MB - 700MB of free memory
on that partition.

Now throw a couple of full BGP feeds into the remaining odd 700MB, and you
quickly see what is wrong with this picture.

There is a caveat (unofficial) that Cisco do not recommend running software
redundancy if the router is running BGP. 
You won't find this recommendation online anywhere, as it was an internal
discussion within the ASR1000 BU. But AFAIK, internal notes have been made
available to account teams in case customers have questions about this.

Bottom line, don't enable software redundancy if you have BGP running.
Personally, I don't enable software redundancy, period. I bought a box with
a single control plane. If I want redundancy, I'll buy one with two control
planes.

The idea is novel, but it doesn't really work.

Hope this helps.

Cheers,

Mark.