[c-nsp] Data Center switch replacement

[chris at lavin-llc.com]

>
> Now that's a good opportunity to clean up broken server configurations and
> connections.
>
> "If it's meant to be redundant, and it isn't, then it's not the network's
> fault if it breaks.  Go and fix it!"
>
> gert
> --

Thanks to everyone that responded. I appreciate learning how much several
of us have in common. I especially liked those that shared stories with me
about similar challenges of server and NIC settings for what should be a
redundant design with Primary/Secondary configurations.

I'll update my recommended options to include a third scenario.
1. Complete blackout to power down each switch and replace it with the new
one.
2. Eat the cabling/rack/etc. cost and stand up the new switches and
migrate the connections in one night (performing some due diligence ahead
of time) and hoping all servers are properly configured for a
Primary/Secondary network connection.
3. Eat the cabling/rack/etc. cost and stand up the new switches and
migrate slowly over a period of several maintenance windows while hoping
we don't have any more line card failures during the extended migration
period.

Much appreciated,
-chris