[c-nsp] Best Practices for connecting MPLS core to Internet?

[Mark Tinka]

On Monday, August 08, 2011 10:43:19 PM Ross Halliday wrote:

> Unfortunately I think I haven't communicated our scale
> very well. Most of the network is a single 6509 at a
> telephone central office with equipment hanging off of
> it. The primary goal is to carry voice traffic around.
> Our data subscribers are typically residential and small
> businesses; the average for "high end" is a /29 sent to
> their Fortigate or SonicWall.

So since you're looking at making new purchases, I'd 
consider removing the star topology you currently have your 
lone 6509 running, and spread that risk across at least one 
extra chassis - but make it core (P) and let customers and 
services terminate elsewhere.

Of course, this requires a re-engineering of your topology, 
which I believe is a natural evolution of your growth 
anyway, since you're buying new kit and re-thinking your 
concept.

> We're looking at a gear list like:
> 
> 1x 6509 for Internet connectivity

The 6500 is an old platform, and unless you're looking at 
going with the SUP2T, I'd certainly say it's overkill (and 
somewhat under-featured) for border routing.

I'd, rather, suggest going with 2x ASR1002's instead of 1x 
6509.

> 7x 6509 as MPLS P/PE wonder boxes

As P boxes, they should be okay even with SUP720-3B (if 
you're doing pure MPLS forwarding in the core).

As PE routers, depending on what you want to do and how 
kinky you expect to get, the SUP2T currently has the best 
features (on paper, anyway). Otherwise, I'd look at another 
platform such as the ASR1000, ASR9000 or Juniper MX.

> 1x 7204 VXR NPE-G2 in a colo somewhere as MPLS P/PE
> (P-router except for a single EoMPLS)

Get a 7206 chassis, the price difference with the smaller 
cousin is nothing to worry about, unless you have space 
constraints.

Still not sure what you want to use these for. Non-transport 
services, e.g., DNS, mail, e.t.c.?

> ?x assorted 7200s
> as PEs for T1s and PPPoE

Should be fine.

Think of route reflectors too. 7201's help a lot, are cheap, 
small and will crunch a full table faster than a SUP720.

> We have a very small amount of customers that would
> consider a demarcation point anything other than a jack
> on a wall. Most of our deployed CEs consists of 2900
> switches, with some 2620s where T1s appear.

Even your low customer numbers wouldn't necessarily prevent 
you from deploying a topology that would scale well :-).

> My reasoning behind this is for management access. I very
> much like the idea of keeping any management networks
> away from Internet networks on an edge device.
> Compounding this is the software train we're using
> (12.2(33)SXI) has pretty lame VRF support for management
> protocols, which rules out the reverse of having
> management in a VRF and Internet in the global.

Given the difference in system requirements, potential 
issues and overall operational concerns, the price you're 
paying to carry the Internet in a VRF just so you don't 
merge that with Management traffic is quite high, IMHO.

I'd say, perhaps, consider running an OoB network that 
connects to the "Management" ports on your devices, or 
better yet, since you're buying new kit, go for a platform 
that offers decent Management VRF support if you're unhappy 
with the 6500 in this regard.

I just think that to separate Management and Internet 
traffic, putting the full table in a VRF is a pretty steep 
sale.

For us, if we want to separate Management traffic from the 
Internet, we either build an OoB network into each 
"Management" port on the device, or use a terminal server 
and connect up the local "Console" ports.

We sleep easier at night knowing the Internet is running in 
the global table :-).

> I had thought about that - keeping the same ASN and using
> iBGP - but it seems to me that I'd need to not only set
> up the base VPNv4 stuff across the MPLS core but also
> additional peering from the Internet-connected system to
> the "internet" VRF on every single PE out there. It's
> definitely a possibility but seems pretty clunky to me.

Of course, if you're running the Internet in a VRF, this 
gets more complicated to implement, much less scale.

If you're running the Internet in the global table, just 
deploy a route reflector in the "middle" of the network, and 
have your border and edge routers peer with that.

As evil as MPLS is, one of the biggest advantages it has 
brought to the table is the ability to run a cheaper core 
that doesn't need to carry the full BGP table. Your 
SUP720-3B's should be right at home in the core, of an MPLS 
network that supports the DFZ.

> I'll have to look more into route reflectors and what
> they can do. Like I wrote before, we're learning quite a
> bit :)

So you definitely want to buy these, because full iBGP 
meshes across the network will get old quickly.

They're simple - rather than having all PE routers peering 
iBGP with each other, have them peer with at least 2x route 
reflectors, and only have 2x iBGP sessions on each PE router 
to the route reflectors, rather than as many as you have PE 
routers less one (the N-squared problem).

> On a larger network I'd definitely agree. However most of
> those 6509s I mentioned above are SUP720-3B non-XL so
> can't hold an Internet view anyway.

A large network may have nothing to do with the equipment in 
use. I know a very large network handling some 20Gbps of 
traffic using more than 300x 7206-VXR's as edge routers :-).

But like I said, since you have only 1x 6509 today, and 
you're looking to buy more, either get the SUP2T or go with 
a more decent platform that will get you what you need now, 
and let you scale later.

> At present we have
> only one customer that actually needs a full feed. Might
> grow that to two customers one of these days... ;)

Don't underestimate how quickly you can grow, for whatever 
reason. Better to put in scaling properties now instead of 
having to re-plan another migration in the future when you 
realize the customer base (or the network) has grown 50-
fold.

That said, for the full table eBGP sessions, I've seen 
networks deploy centralized routers that have a full table, 
and run eBGP Multi-Hop with customers who connect to edge 
routers that are less-than-capable BGP-wise.

Mark.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
URL: <https://puck.nether.net/pipermail/cisco-nsp/attachments/20110809/b8d78397/attachment-0001.pgp>