[c-nsp] Sup32 and IOS 12.2

Sat Jul 8 14:22:48 EDT 2006

On Sat, Jul 08, 2006 at 12:28:01PM -0400, Robert Blayzor wrote:
> Saku Ytti wrote:
> > Since 2006-06-01 you can include bootflash upgrade (melody) to your order 
> > for 0 USD. Gives you cflash adapter + 512MB bootflash.
> 
> CF has never been an issue.  The Sup32 comes with a 256MB CF card
> on-board, and one external CF slot.  CF cards are cheap enough now days
> where this isn't even an issue.

One interesting point to add to the discussion is the speed at which you 
can read data off the flash. Obviously anyone who owns a 6500/7600 knows 
just how long they can spend twiddling their thumbs and cursing Cisco 
waiting for one of these things to boot. This has always been a major 
advantage of Foundry products, where even if you have a crash or unstable 
code the box can be back up at a significantly lower customer pain 
threshold (30 secs instead of 5+ minutes).

If you look at the boot sequence on a 6500/7600, much of the time is spent 
reading and decompressing the (rather large) image off flash in order to 
boot the sup, then an image is transfered to the MSFC so that it can boot, 
and then images are transfered to cards to they can boot. The process is 
slow and doesn't try to do much in parallel, but still a huge chunk of the 
time is spent reading those large images.

I haven't had the free time to do actual reboot time tests, but I think 
you can see a lot of the issue just by doing copy to null: tests. On a 
completely idle SRA lab box, you can see that images read off a standard 
SanDisk 512MB CF at almost exactly 1MB/s:

router#copy disk1:s72033-advipservicesk9_wan-mz.122-33.SRA.bin null:

86709252 bytes copied in 87.208 secs (994208 bytes/sec)

87 seconds is a fairly significant amount of time in the boot process, so 
I wanted to look and see if this could be improved upon. The time taken on 
SXF seems to be roughly the same, though the transfer rate obviously gets 
slower if the box is running BGP and doing real work in the background 
too:

router#copy disk0:s72033-advipservicesk9_wan-mz.122-18.SXF3.bin null:

76943908 bytes copied in 87.704 secs (877314 bytes/sec)

Coping an image from sup-bootflash: to null: seems to be almost exactly 
twice as fast, 2MB/s in my tests:

router#copy sup-bootflash:s72033-pk9sv-mz.122-18.SXD4.bin null:

46633792 bytes copied in 23.467 secs (1987177 bytes/sec)

Of course not all CF is created equal. The "standard grade" SanDisk CF is 
rated at a read speed of only 2MB/s, while many newer CF cards intended 
for digital camera use (at prices in the $50-100 range too) are spec'd at 
133x-140x and 20-21MB/s read speeds. Of course the host board has to be 
able to take advantage of that speed, and the CF 2.0 spec appears to top 
out at 16.6MB/s (IDE PIO mode 4 basically), but this is still a 
significant improvement over 1-2MB/s.

Unfortunately I haven't yet had the time to do any serious benchmarking or 
see if the SUP720 is able to take advantage of the faster CF. Of course 
these days 512MB is well below the optimium price curve for CF too, I 
stumbled across a 150x 8GG CF for $150 without even trying, but I haven't 
seen any tests as to whether a SUP720 can support those either. :P One 
preliminary report I've heard from one person who tried loading high speed 
camera flash into their SUP720 was that it still topped out at 1MB/s, but 
this hasn't been confirmed yet. If that was in fact the case, the increase 
in speed on the sup-bootflash would seem to be either because a) the 
internal flash is quicker to read from than the external CF, or b) the 
64mb flash is quicker to read from than the larger flashes in some way.

Maybe by posting this I can get a bunch of folks to bring their camera 
flash to the colo and run some tests, since I'm very curious to see what 
comes out of it. If faster CF really doesn't help things, doubling the 
speed of the flash and cutting at least 43 secs off the boot time on a 
current SRA image (with no signs that these things are going to start 
getting smaller any time soon :P) may be a good reason to upgrade the 
internal sup flash chip rather than use external CF.

-- 
Richard A Steenbergen <ras at e-gerbil.net>       http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)