[c-nsp] BGP multihoming with redundancy config problem

[Pete Templin]

Ganbold wrote:

>> Step 2: Remove ALL bgp "network" statements on your border routers, 
>> and related Null0 routes.  Let your core announce your prefixes "up", 
>> so that if you lose a core-border link you don't end up in trouble.
> 
> I'm a kind of confused. Maybe because I'm newbie to bgp.
> Will bgp work without "network" statement? How will load sharing and 
> redundancy work in such case?
> Can you explain more about it to me?  Can you share to me some working 
> config if you have?

Let me flatten out your topology (because ASCII art doesn't always work 
well with line breaks):

ISP1---BDR1---CORE1---CORE2---BDR2---ISP2

BDR1 through BDR2 are all in your ASN.  Each of the BGP routers will 
share routes with their neighbors.  Since they'll share routes, only one 
router has to "create" the BGP route.  If it "creates" the route within 
BGP, the others will learn it.  You can announce the routes via the 
"network" command (if the route is in the RIB, make a BGP route and 
advertise it to neighbors), the "redistribute <protocol>" command (not 
recommended, but if the route is in the RIB via <protocol>, make a BGP 
route and advertise it to neighbors), and/or the "aggregate-address" 
command (if a match or longer prefix is in the RIB, announce the 
aggregate; I wouldn't recommend this either).  However, I would 
recommend announcing the route on both core routers, so that you can 
interconnect core1 to border2 and core2 to border1, etc., and tolerate a 
core router failure.

>> Step 3: Any chance you can NOT announce the /24s, but just the /19 
>> instead?  (Had to put a plug in for aggregation...)
> 
> 
> I can't do it right now. Some of those IPs are allocated for customers.

Here is where I get worried.  If the /24s are in the /19, the /19 will 
provide reachability.  You don't need the /24 announcements in BGP for 
these customers to be reachable.  You can help save excessive growth in 
everyone's routers by not announcing the /24s unless you have specific 
needs for traffic engineering, or a /24 customer is themselves multihoming.

>> Step 4: What link were you breaking, such that failover wasn't 
>> happening?  If it was a core-border link, see step 2.
> 
> 
> It is a core-border link.

OK, exactly as predicted.  Because the border router had a "network" 
statement, during the test your network looked like this:

ISP1---BDR1   CORE1---CORE2---BDR2---ISP2

But BDR1 was announcing your /19 to ISP1 still, even though it had no 
ability to deliver inbound packets to CORE1 and beyond.  If you remove 
the network statement(s) from the border routers, your network would 
have looked like this (slid to the left for simplicity):

CORE1---CORE2---BDR2---ISP2

Now, both core routers are sending packets towards border2, and ISP2 is 
the only ISP receiving packets destined for your prefix(es).  The 
flexible thing is you could (instead) conduct a failure on border2, 
thereby reducing your network to:

ISP1---BDR1---CORE1---CORE2

>> Step 5: Do you have alternate connectivity between the LEFT part of 
>> your network and the RIGHT part of your network?  I'm a little worried 
>> about your diagram at first.
> 
> 
> No, Core routers are connected directly with each other by gigabit 
> ethernet interface.

OK, you may want to reconsider this as well.  If the ONLY failure is the 
gigabit link between core routers, you will also have problems.  Your 
network will look like:

ISP1---BDR1---CORE1   CORE2---BDR2---ISP2

Devices connected to core1 will send all outbound traffic towards ISP1, 
but the responses could come in through ISP2 (BGP often leads to 
asymmetric routing) where it can't be delivered back to core1, and vice 
versa.  Fully meshing your border and core routers would be a good idea.

pt