[c-nsp] BGP and OSPF - redesign

[Jay Hennigan]

chris at lavin-llc.com wrote:
> Along the lines of the recent discussions about eBGP, iBGP and OSPF intertwined routing, I have a redesign to deal with. An enterprise solution that 
> currently runs eBGP, iBGP and OSPF with the iBGP and OSPF fully mixed. By that I mean there lacks a policy of seperating the two. Rather than having 
> OSPF carry only the required /32s for the purpose of building the full iBGP mesh, OSPF and BGP are contributing to the forwarding tables for all 
> traffic. This is causing some odd and unpredictable behavior for route announcements and path selection.
> 
> The problem I'm struggling with is how to transition the routes out of OSPF so that iBGP is used to carry the traffic, thus reducing OSPF based routes 
> to only be responsible for building the full iBGP mesh. Most of the appropriate goodies are in place, like locked in router-id's and no synch. But the 
> jenga-like configurations of redistribution and network statements make for a mind bending exercise for trying to migrate to the ISP Essentials formula.

Here's how we did it.

1. Originally we had infrastructure participating in OSPF, 
redistributing connected and static customer routes into OSPF.  BGP was 
primarily used externally.  iBGP was used only to interconnect border 
routers.  We found the OSPF tables getting bloated.  Reconvergence after 
a link flap was painful and rippled through routers that shouldn't have 
been affected.

2. At each site we brought all routers into iBGP.  Non-borders got a 
filter-list that included just local origin and downstream customer 
ASes.  This so as not to overwhelm small routers with full tables.  If 
you have several routers per site, it's more scalable with peer-groups 
and route reflectors.  All iBGP should be done to loopbacks, and the 
loopbacks should be routed throughout your AS via OSPF.  Configure 
next-hop-self and send-community.

We then carefully redistributed static and connected routes into iBGP 
with a route-map, thusly:

router bgp [nnnn]
  ....
  redistribute connected route-map cust-to-bgp
  redistribute static route-map cust-to-bgp
...
route-map cust-to-bgp permit 10
  match ip address prefix-list local-nets
  set origin igp
  set community no-export
...
ip prefix-list local-nets description Customer allocations
ip prefix-list local-nets seq 10 permit <customer subnet>/nn le 32

At this point all routers should have your customer networks in their 
BGP tables.  As iBGP has an AD of 200 and OSPF is 110, the routes to the 
customer networks will still show up as OSPF external in the IP routing 
tables.

3. Verify that the links interconnecting the routers and the loopbacks 
show as OSPF routes (not OSPF E1 or E2).  Verify, one router at a time, 
that customer routes redistributed into OSPF are in the BGP tables of 
other routers in your AS pointing to the loopback.  Verify that you 
aren't spewing all of these small subnets to your eBGP neighbors. 
(That's what the no-export and send-community are for.)

4. "No out" the redistribute statements for connected and static in your 
router OSPF, one router at a time.  You can set up a continuous ping to 
a customer target on a different router and you probably won't even lose 
a packet if you've checked everything first and your CPU is below 90%.

5.  Verify that your customer routes are now shown in the routing table 
as BGP.

6.  Verify that your OSPF routes are now lean and mean, with just 
infrastructure links and loopbacks.


--
Jay Hennigan - CCIE #7880 - Network Engineering - jay at impulse.net
Impulse Internet Service  -  http://www.impulse.net/
Your local telephone and internet company - 805 884-6323 - WB6RDV