[c-nsp] Acceptance Test Procedure for New Cisco Devices

[Luan Nguyen]

Going a bit further...how's about looking at those benchmarking RFCs
http://www.ietf.org/html.charters/bmwg-charter.html

In particular
http://www.ietf.org/rfc/rfc2544.txt
for the 1861 and
http://www.ietf.org/rfc/rfc3511.txt
for the ASA

Regards,

Luan Nguyen
Chesapeake NetCraftsmen, LLC.
[W] http://www.netcraftsmen.net
[M] luan at netcraftsmen.net
[Blog] http://cnc-networksecurity.blogspot.com/




-----Original Message-----
From: cisco-nsp-bounces at puck.nether.net
[mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of Phil Mayers
Sent: Tuesday, January 20, 2009 8:06 AM
To: Ziv Leyes
Cc: cisco-nsp at puck.nether.net
Subject: Re: [c-nsp] Acceptance Test Procedure for New Cisco Devices

Ziv Leyes wrote:
> Ok, let me be more specific When we buy devices for our own use, we
> just open it, plug it, and start using them, if there are any
> problems, we call the provider and they fix the problem (RMA or
> whatever) In this case, we're going to sell the equipment as a kind
> of turn-key project, and the customer asked us to provide them with
> "our" ATP, which we don't really use for ourselves, so I'd like to
> implement one sort of testing procedure from now on for this type of
> cases. We're going to attach this to a legal statement so we can't
> just type some BS there and that's it, we want to actually implement
> it, and if we write we do a,b,c,d then we'll going to do a,b,c,d
> procedure for real. I was thinking some of you guys may already use
> this kind of test routines and can help me creating one. I don't need
> some really serious stuff, I can imagine I'll check the delivery
> status of the package, open it, check all the contents that need to
> be there are there, to plug the device and see it works, perhaps load
> some configuration, plug the hardware that is planned to hold if any
> (HWICS and so), perform some soft and hard reboots, see the device
> responds, there are links on all interfaces, and pack it back exactly
> as it was. The problem is I don't know how exactly write it down on a
> kind of form that there's a checkbox for each test. Does anybody have
> some ready to go stuff?

Well, it's going to depend very much on the kind of equipment.

For example, a mandatory step when we get anything for our 6500s is a 
complete run passing all GOLD tests (including the disruptive tests). We 
maintain a spare chassis specifically for this.

I don't know if ASA5510 and 1861 have diagnostics, but I don't think so. 
In that case, you're probably going to want something like:

  * Build a standard config involving (at least) your ASA & 18xx router, 
which all or a large subset of the features are enabled

  * For each pair of devices you distribute, load the standard config on 
and run some test traffic

  * Leave it powered up for long enough to count as "burn in" i.e. 7 days?

So you'd write something like:

"""Party X will undertake to:

  * Unpack all equipment and check inventory
  * Check that equipment will power up
  * Load on a standard config, which tests:
    * OSPF routing
    * BGP routing
    * Packet forwarding
    * IPSec
    * Coffee making
  * Run test traffic for 48 hours, to ensure the devices compare to a 
known-good platform
  * Leave the config running for 7 days, to eliminate early-life failure

...before shipping to Customer Y"""
_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/