[c-nsp] 3560 questions

[Adrian Chadd]

On Thu, Apr 05, 2007, Alex Campbell wrote:
> 
> Hi all,
> 
> I'm looking at putting two 3560Gs in front of a couple of servers in a
> high-availability configuration, and hoping that someone who has tried a
> similar setup could provide some guidance.  A rough diagram of the
> approach I'm considering is here
> http://alexjcampbell.com/3560g_diagram.gif.

(Not that I'm one of those crazy types who has 3550's as transit routers
in a very controlled environment, oh -no-..)

> Basically each 3560 will take a default route from a different provider,
> and one of them will take ~2000 prefixes from a local IX.

First thing: set route protocol min-priority so it gets -some- CPU, look
into limiting the impact of things such as ARP storms and spanning tree
recalculations. The IX local to me had a rather nasty outage a couple months
ago because of a silly customer and some ARP storms; locked up the 3550's
rather tight.

> We'll run IBGP over a port-channel between the 3560s, although I'm not
> sure whether the port-channel connecting them should be configured as L2
> or L3 interfaces.  Should the IBGP session be between the SVI on each
> device, or between the IP on each L3 interface?

Ideally? Loopbacks? I run port channel between my two 3550's as I have
some VLANs I'd like shared between them. I then run iBGP on the SVIs on
a "router" VLAN.

> Inside the network, we'll have servers with two-port Intel server NICs,
> with each port on the NIC connecting to an access port on each 3560.  It
> seems the best (only?) way to uplink the servers to two different
> switches is to use Intel's "Switch Fault Tolerance" feature, which
> basically allows the two-port NIC to participate in STP, thus providing
> L2 failover.

Make sure you investigate setting port priorities and rootguard correctly
to make sure a server doesn't "magically" become preferred as the link
between two switches.. The defaults work fine until they don't.

> Both switches will have an SVI interface for the servers' VLAN, and
> we'll use HSRP on the SVIs to provide L3 failover for the servers.

Read the URL I Posted a few days ago from the CIsco site - unicast flooding
in campus networks - as you might see similar issues crop up.

> I guess my questions here are:
> 1) will this work and will it achieve full, automatic failover
> throughout the network?
> 2) is there an easier way of achieving this?
> 
> And one more, slightly unrelated question - 
> 3) if our transit pipes are say 50mbps but delivered over FastEthernet,
> how will the 3560s react if we start pushing 51mbps?

You'll send to your ISP; they'll say "hell no" and either shape or drop.
Its best to investigate egress shaping/policing options on the 3560
(3550 only polices, not so good for small pipesizes) so you don't congest
on sending. I'm pretty certain the 3560 has more options than the 3550
for egress QoS in the manner you want - read up on the relevant chapter
in the 3560's software configuration guide.

> Any guidance or insight would be most appreciated.

* watch your SDM prefer template and TCAM limits; don't overflow;
* give your IGP/EGP processes a minimum amount of time to run or things
  might get hairy;
* put aggressive limits on the amount of advertisements you'll accept
  from the IX or one spot of deaggregated announcements will ruin your day;
* be wary of unicast flooding
* watch stuff very closely!



Adrian, who knows its pretty evil to run 3550's as edge devices speaking
BGP but heck, they're cheap, and they "route" real damned fast too..