[c-nsp] question about service provider network design

[Nathan]

Hi,

I'm re-designing a service provider MPLS network, and I'd
appreciate some macro-level input.

I have two major sites connected by two gigabit WAN lines. I
have or will have about a dozen Cisco switches (3508, 2960,
3548, 3550...), half a dozen C7206s for customer termination,
four J4350s for eBGP, and miscellaneous junk, all routers
connected with OSPF and MP-BGP.

Currently I have no layer2 loops, so only "accidental" STP,
and no VTP. Basically the WANs each come in on one of two
switches, and all routers have one link to each of the two
switches, (with as needed further links to access/distribution
switches). Off the cuff only about half the WAN link outages
have been accompanied by L1 ethernet link loss.

I've done without STP and VTP because I'm not completely
comfortable with them, due to lack of experience and scary
stories about spanning tree loops and such (at least one of
the WAN links is over fiber so one-way communication is very
possible). All switches are L2 only even though I know that
some are capable of L3, I've never really understood or seen
documentation on how an L3 ospf-running switch actually works in
a production network.

To get cross-site L2 service I've envisaged running multiple
vlans over the WAN lines, or setting up VPLS (not on 7206) or
pseudo-wires (not multipoint), but I've never actually done
either. I know how to do the first, I'm not sure about the last
two.

As I move to Gbit bandwidths and multicast and want to reduce
failover times, use both WAN links, and provide cross-site L2
service for myself and for clients, and having been bitten by
NPE300s not forwarding at 100M line speed, I thought I'd replace
that with an L2 square running some sort of STP and VTP. I
documented myself on VTP, on MST, and then I found the "Cisco
Campus Network for High Availability Design Guide" which among
many other things says "avoid L2" and "avoid square", "avoid
STP", and "avoid VTP". The only square in there is that they're
very squarely recommending not to do what I was thinking to
do. I didn't find any service provider version BTW.

So what would be state of the art with room for expansion for my
quite limited network? I do not have four inter-site Gbit links
as recommended; the two I have are costly, and if I go adding
more directly to my routers I soon won't have enough interfaces
(a third site would be a projected expansion). Can L3 OSPF on
WAN-connected switches help me detect link loss instantly even
though the switches are connecting routers running MPLS and
BGP? If I connect the WAN links directly to routers, I'll have
four expensive routers mainly passing packets for the other
routers, that doesn't seem cost-effective. Am I missing or
misunderstanding some crucial documentation or insight?

Thanks for any comments,
-- 
Nathan