[j-nsp] Core network design for an ISP

[Saku Ytti]

On 25 March 2016 at 01:57, Matthew Crocker <matthew at corp.crocker.com> wrote:

Hey,

> I’m running MPLS now and have full tables in the default route instance.   Does it make more sense (i.e. more secure core) to run full tables in a separate virtual-router?  I’ve been doing this small ISP thing for 20+ years, Cisco before, Juniper now, I’ve always bashed my way through.

If you're gonna run L3 MPLS VPN's for what ever purpose, or might run
in future, I strongly recommend putting Internet in VRF. Global table
is annoying special case and doing route injection between global
table and vrf is huge PITA in JunOS. Having Internet in VRF completely
removes this problem.

> MPLS?  BGP? IS-IS? LDP? etc.

Today I think you need good reason and justification not to run MPLS
and default to running it. I would certainly run MPLS. As it is
greenfield I'd try to see if running segment routing is an option
instead of LDP or RSVP. If SR is not an option, decision between LDP
and RSVP would depend on if you need strategic or tactical traffic
engineering. If you have sufficient capacity to carry all traffic in
best path during normal situation and single failure definitely no
RSVP, if you do not have sufficient capacity to carry all traffic in
best path during normal situation then definitely RSVP. If you can run
all traffic in best path, but not during single failure then LDP/RSVP
might be debatable.
Even if you choose LDP, you probably want to enable RSVP on links
without configuring any LSPs just in case you can do ad-hoc tactical
TE for specific needs. Like maybe some PE<->PE pair requires two
non-fate-sharing paths. Or maybe your capacity planning cocked up and
you can't turn-up customer until some capacity delivery is done, you
might want to run this customer's traffic on offSPT while waiting for
capacity planning to catch up.
With SR you can cover both LDP use-cases and tactical RSVP use-cases
and not run any new protocol. Your core would run only one protocol,
IGP.

If you're going to have real core (i.e. devices which do not connect
customers) then core can be BGP free, as long as your edge will have
iBGP full-mesh or your route reflectors support ORR (optimal route
reflection).
I would start with RR iBGP day1, because RIB scale likely will hit you
before hardware FIB scale. I would work very hard to do off-path RR
with vMX or equivalent, but I would absolutely require ORR to be there
for this solution to be acceptable.

I'm great supporter of separating control-plane from services and
would run IPv6 in 6PE until one day fork lift network IPV6 only and
put IPv4 in 4PE. Only reason why I might not run 6PE is if I'd run SR.
Goal would be to keep signalling and state in network to minimum.
Software from all vendors is extremely bad, and the less codepaths you
need to explore, the better.


-- 
  ++ytti