[c-nsp] CEF fun in SXF

[Chris Griffin]

In our internal testing, we have noticed that Cisco CF cards are *much* 
slower than commodity CF cards you can get at most stores.  Sandisk for 
example.  This testing was done on 128MB cards, so it may not be valid 
for 512MB cards, but you might want to take a look at it.  In our 
testing booting from a Sandisk CF card was VERY close to booting from 
internal flash and MUCH faster (something like 3 or 4 times faster) than 
the Cisco card (that cost 10X as much).
--
Chris Griffin                           cgriffin at ufl.edu
Sr. Network Engineer - CCNP             Phone: (352) 392-2061
CNS - Network Services                  Fax:   (352) 392-9440
University of Florida                   Gainesville, FL 32611

Gert Doering wrote:
> Hi,
> 
> On Tue, Dec 13, 2005 at 12:44:32AM -0800, Roman Sokolov wrote:
> 
>>Monday, December 12, 2005, 11:58:51 PM, you wrote:
>>GD> Hmmm, I'm not sure that applies.  This seems to be just "special" stuff,
>>GD> not "generic unicast packets travelling through the box".
>>urpf (surprise! surprise!)
> 
> 
> Little uRPF on that box - it's a core router with only one customer-facing
> link.  So all packets not coming from that interface shouldn't be uRPF-
> limited, no?
> 
> 
>>GD> Well, the main problem was a quite unexpected memory leak in the BGP
>>GD> process.  The box has more than sufficient memory for the number of routes
>>GD> it's carrying (only 170k "Internet" routes, and some 200 VRF routes), 
>>GD> but as BGP was leaking like hell, we got caught by surprise.
>>3BXL? And you've lost whole gig??
> 
> 
> 3B, non-XL, but 512 Mb normally are more than enough - after reboot and
> BGP convergence, we have 140 Mb of free memory.  Over time, this went 
> down to about 25 Mb, which was low enough that we had scheduled a reboot
> "next maintenance window" - but the remaining memory was fragmented 
> badly enough so that it made CEF seriously unhappy.  Or so.
> 
> 
>>GD> Fast reboot isn't available on Sup720 - the box will always do a 
>>GD> very-slow-reboot...  especially with the huge SXE and SXF images :(
>>Some checks shows that difference in loading between -wan and -lan software
>>with almost clean config is just about 30 seconds. But sometimes some
>>unexpected delays could exist.
> 
> 
> I haven't taken the time recently, but just the fact that SP and RP
> boot the same image sequentially, and that the compact flash access is
> MUCH slower than internal flash (which is not big enough to hold SXE/SXF
> WAN images) really adds up...
> 
> 
>>GD> Sure, but that doesn't keep my from trying to improve things :)
>>Sometimes it's much better to leave it on it's own :)
> 
> 
> How much worse can it get...?  A now-current BGP implementation that's worse
> (stability-wise) than what was available 5 years ago?  A nice and shiny
> hardware router that has more bugs in its distributed FIB tables than 
> "ip route-cache distributed" had, 10 years ago?  A router that's shipped
> from the factory with insufficient internal flash to hold the image
> that was bought with it?
> 
> (Well, to be fair, overall the Sup720 is a nice device.  But some parts
> of Cisco quality control suck big time)
> 
> gert