[c-nsp] BGP and HSRP

[Lamar Owen]

On Wednesday 09 May 2007, myNET NOC - Bernd Ueberbacher wrote:
> Hi everyone!
>
> I'm reading this list for a couple of months now and tonight I got my
> first question :-)
[snip details]

It is a really good list, isn't it?  I certainly have found it to be.  In any 
case, maybe this will help you think it through:

Ok, you have two upstreams, and three routers.  Let's call the first 
upstream's router 'U1', the second upstream's router 'U2', and the internal 
third router 'I3'.

Now, U1 will need to BGP peer with its upstream router.  U2 will need to BGP 
peer with its upstream router.  U1 and U2 need an iBGP neighbor relationship 
between them. (meaning you need an AS number; you can probably get your 
upstreams to filter a private ASN for you if you don't have your own ASN).

I3 would ideally run an interior gateway routing protocol to get to U1 and U2 
(and the rest of your network) rather than HSRP, which is designed to provide 
failover for workstations that only have a default route (well, any device 
with only a default route).

BGP itself will provide all the automatic failover from your upstream routers 
back to U1 and U2; you neither need nor really want HSRP on the upstream side 
of things.  And given that the upstreams are not on the same subnet, HSRP 
won't even work (HSRP won't work on a /30 anyway, as there aren't enough IP 
addresses: you need an absolute minimum of 3 usable addresses for the gateway 
side of HSRP, not counting the stations/routers with their default gateway 
pointing to the HSRP virtual IP, and your /30's have only two usable 
addresses; a /29 is the smallest subnet on which HSRP will work).

Now, if you REALLY want HSRP on the LAN side, it will work, but you then don't 
run iBGP on that side; I3 would have a simple default route to the HSRP 
virtual address, and U1 and U2 would have LAN interfaces on the same subnet 
as I3's interface.

I'm doing something similar to this here with a pair of 7401's at the provider 
end of an OC3, using a Catalyst 5505 as a 'port expander' for the 7401's, and 
talking through what I'm doing might help you see how to use HSRP and BGP 
appropriately in your instance.  The 7401's and the 5505 are at the 
co-location and upstream PoP facility; the OC3 is a non-Internet WAN link 
from the co-lo/PoP to my site, and the Internet connection is over Fast 
Ethernet.

The OC3 is configured with APS redundancy; each 7401 has a PA-POS-OC3-SMI in 
it, but only one is active at any given time, with the other as a hot standby 
(the APS terminology is 'working' and 'protect' with only one of them 
being 'active' at a time).  

Each 7401 has two GigE interfaces, one of which is set up as an 802.1Q trunk 
to the 5505 (the second port on each 7401 is being connected to another 
Catalyst for layer 2 redundancy, but that's not finished yet).  On the GigE 
trunk, I have a VLAN for the internet connections going to a port on the 5505 
that connect to my upstream's 7609 (yes, I'm upstreaming with a /29 and two 
BGP sessions over it; a second /29 is going to be implemented for a second 
upstream a little later); I also have a VLAN for the co-lo servers connected 
to the 5505. There are other VLAN's configured, but they aren't important for 
this discussion.

The Internet VLAN subinterfaces on the 7401's run BGP to my upstream (in this 
case, redundancy to the single upstream due to SONET APS).  The two 7401's 
have an iBGP connection between them, and I'm not redistributing the BGP 
routing into the OSPF.  The VLAN for the co-lo servers runs HSRP tied to the 
OC3 interface status, so that an APS 'working-protect' transition event also 
switches the HSRP active.  I'm running OSPF between the two 7401's and the 
routers on the local side of the APS protected OC3, and failover is pretty 
quick.  (Oh, and NAT is in play here, too, with Stateful NAT failover and 
HSRP NAT groups, but that muddies the waters).  Oh, and in case anyone is 
curious, the two POS interfaces are configured with the same IP address for 
least confusion in the routing.

But the HSRP on the co-lo side from the servers works well, and the BGP 
routing out works well too, but they solve different problems.  Dual BGP 
sessions to upstreams don't need HSRP, and it would be more trouble than it's 
really worth to try to get working.

I hope this helps you think through the problem you're really trying to solve 
here, which, unless I misunderstand, is getting failover between your two 
upstreams.  There are several Ciscopress as well as other publisher's books 
that address this topic; also, you might want to read the Cisco whitepaper 
that talks about enterprise multihoming with NAT, as it gives a good diagram 
of part of what I've implemented here (although I'm not doing the type of NAT 
they describe).

If you'd like pointers to some good books, let me know and I'll reply offlist.
-- 
Lamar Owen
Chief Information Officer
Pisgah Astronomical Research Institute
1 PARI Drive
Rosman, NC  28772
(828)862-5554
www.pari.edu