[c-nsp] 2 locations, 2 ISPs, 2 ASAs, configuration confusion

[Phil Mayers]

Winders, Timothy A wrote:
>>>
>>>
>>>           Internet
>>>            /    \
>>>         ISP1    ISP2
>>>          |       |
>>>         ER1 --- ER2
>>>          |       |
>>>         ASA1    ASA2
>>>          |       |
>>>        CORE1 --- CORE2
>>>
>>
>> If you are running iBGP all the way to CORE1 & CORE2 then you will 
>> need a full mesh.  So you are missing iBGp from ER1 to CORE2, ER2 to 
>> CORE1, and CORE1 to CORE2.
>>
>> Alternatively if you use ER1/ER2 as your BGP edge then you could run 
>> OSPF Between the edge routers, and from the edge routers to the ASAs 
>> and inside the ASAs also. If ER1 / ER2 are injecting default routers 
>> into OSPF then I think that would do the trick.
> 
> Thanks, Peter.
> 
> A full BGP mesh would not work.  We need the routing protocol to know
> that CORE2 is 2 hops away from ER1 and vice-versa.  This is what will

You're misunderstanding how iBGP works. iBGP routers advertise their 
iBGP routes as going via their loopbacks, and you additionally run an 
IGP e.g. OSPF which gives you the costs to the loopbacks, which achieves 
what you're talking about.

The full mesh simply refers to a full mesh of routing sessions (which 
are not adjacencies in iBGP) which you would need unless you use a route 
reflector (barely worth it for this number of routers). I imagine iBGP 
was suggested out of a combination of "best practice" and "policy 
control" thinking. Whether it's a reasonable suggestion depends on your 
needs and the size of the routing table, plus whether you've got much 
expertise in running it. If you can, I'd consider using it.

We have a somewhat similar setup to this, thought CORE1 and CORE2 are on 
the same site, not geographically remote. It's currently using OSPF, 
planned to move to iBGP shortly. Some things you need to watch for:

The routing costs from CORE1-ASA1-ER1-ER2 and CORE1-CORE2-ASA2-ER2 need 
to be different. That probably means making the ER1-ER2 link have a cost 
of 10 and the CORE1-CORE2 link a cost of 11, and the ERn-COREn paths 
have a high cost (say, 50) to prevent traffic flowing anywhere odd, such 
as out the firewalls and back in.

If there are any downstream routers attached to both core1 and core2 you 
will similarly need to bias the metrics such that odd things don't happen.

Finally, if you do put a tunnel in via the internet, you probably want 
to rig it such that er1 only advertises core1 and er2/core2 - else the 
traffic might flow into er1, back out via the tunnel and into er2 and 
effectively halve your bandwidth.

Good luck.