[c-nsp] Design Recommendation for Dual WAN Connectivity with Multiple MPLS Service Providers

[Brian Knight]

On 2026-01-21 07:49, Muhammad Atif Jauhar via cisco-nsp wrote:
> Greetings,
> 
> We are planning to deploy two WAN links from different Service 
> Providers,
> both offering MPLS connectivity with BGP peering. As part of the design
> review, we would like to align on the best-practice approach to ensure 
> a
> stable, loop-free, and predictable routing architecture.
> Given the use of multiple MPLS providers with potentially different 
> latency
> characteristics, our key objectives are to avoid routing loops, prevent
> asymmetric traffic issues, and ensure controlled traffic engineering 
> and
> failover behavior.
> Any recommendations or considerations based on your experience,
> particularly with respect to MPLS interconnects and BGP traffic 
> engineering.

Sounds like you have an enterprise network connecting to NSPs. Your 
network is probably not an NSP network itself.

It's not strictly on topic for this mailing list, but I'll provide 
what's worked well for me.

Use a different RFC6996 private ASN for each site.

If you're doing a green-field deployment, use 10/8 RFC1918 space. Assign 
a /16 to each site, so your second octet becomes your site prefix, and 
your VLANs become your third octet.

Choose one NSP as primary, the other as backup. At each site, on your 
incoming route policy from the primary NSP, set all routes to high 
localpref, say, 1000. Set all routes from secondary provider to a lower 
localpref, say 900.

Use BFD with BGP for fastest failure detection. Ask your carriers to 
enable it on your BGP sessions if they support it. There are two 
settings with BFD, minimum interval and multiplier. Review those 
settings to see what fits your needs best. Detail here: 
https://www.cisco.com/c/en/us/td/docs/ios/12_0s/feature/guide/fs_bfd.html

Within each site, install two CE routers. Connect Router A to your 
primary NSP, router B to the secondary. Connect the two routers together 
back-to-back and configure iBGP across that link. Be sure to set 
next-hop-self on the neighbor.

Pick two hub sites; one a primary site, the other a secondary site. Set 
up the hub sites to announce all routes including those from the spoke 
sites. Have the spokes announce only locally originated routes.

Connect those routers to the LAN in each location. Think about how 
you'll integrate routing with your LAN gear.

* If you use an IGP internally at any site, avoid routing loops by 
taking care not to redistribute locally originated routes from BGP back 
into your IGP.

* If you use static routing, make sure a FHRP is set up on the CEs to 
provide failover.

Consider setting up VPN backup through GREoIPSec or VTI tunnels to your 
two hub sites. Run BGP across those tunnels, same BGP process as you use 
for connecting to your NSPs.

Use front-door VRF and ACLs to secure the Internet-facing interfaces on 
the routers.

For routes received via VPN to the primary hub, set those to even lower 
localpref, say 800.

For VPN to the secondary hub, set those to lowest localpref, say 700.

If you have any back-door links between sites, like Metro Ethernet, 
switched Ethernet, or optical waves, integrate those by using BGP as 
well. Announce locally originated routes with a higher local preference 
than the MPLS networks, say, 1100.

But be careful sharing WAN routes learned via MPLS across a backdoor 
link. Don't set those to a higher local preference than the MPLS network 
itself. Avoid if possible.

Also, don't forget about the key management aspects for routers: syslog, 
AAA / TACACS, availability and performance monitoring (SNMP), NTP, 
Netflow, and FTP repository for router code. Use a loopback IP on each 
router for your management traffic. Avoid SFTP or SCP to your code repo 
due to known performance issues with OpenSSH and file transfers.

Consider setting up out-of-band access. Best is to build a small 
parallel WAN/LAN over VPN with a firewall, LAN switch, and serial 
console server. Get a separate Internet connection for the firewall 
(business broadband works), and use it only for out-of-band access.

If you decide to configure QOS over the WAN, try to use standard packet 
markings like EF for voice media and AF31 for signaling. Check with your 
providers on what markings they support. The less re-marking of traffic 
you have to do, the better.

> Regards,
> 
> Atif.

Hope all this made sense. I'm happy to clarify anything.

This config served me well in my enterprise days and later on as an NSP 
integrator. 25 years of experience in those spaces.

Hope this helps,

-Brian