[c-nsp] 2 locations, 2 ISPs, 2 ASAs, configuration confusion

Fri Jan 19 09:27:13 EST 2007

Timothy,

We are doing what you are trying to do, with the exception that we are not 
allowing asymmetric routing through the (in this case) PIXes.

-    we run eBGP over both connections to the Internet (to diverse 
providers)
-    we run iBGP between the gateway routers facing the Internet
-    we run OSPF on the internal network

In our network configuration:
-    customers in city A transmit / receive Internet data via provider A
-    customers in city B transmit / receive Internet data via provider B
-    if the link to provider A goes down, city A customers are routed to 
provider B
-    if the link to provider B goes down, city B customers are routed to 
provider A

In basic terms:
-    we receive a default route from both our providers
-    we inject the default routes into OSPF
-    the gateway routers send default routes to each other via iBGP tagged 
with a community string
-    devices in city A prefer the default route to provider A
-    devices in city B prefer the default route to provider B
-    during normal operations, we advertise only the netblocks in city A to 
provider A
-    during normal operations, we advertise only the netblocks in city B to 
provider B
-    utilizing conditional BGP functionality, the gateway router in city A 
only advertises the city B netblocks to provider A if it can still see those 
netblocks via OSPF, and if it is not receiving the default route from its 
iBGP peer
-    utilizing conditional BGP functionality, the gateway router in city B 
only advertises the city A netblocks to provider B if it can still see those 
netblocks via OSPF, and if it is not receiving the default route from its 
iBGP peer

In this way, we in effect form a redundant ring with the Internet. If there 
is a breakage anywhere on the ring, the devices on either side of the 
breakage transmit / receive Internet data via the provider they can still 
"see".

Not sure if this helps -- I think the main concept here which makes it work 
is being willing to forego the asymmetric routing requirement.

Thanks,
Adam

---
Adam Greene
Webjogger Internet Services
http://www.webjogger.net
(845) 757-4000 x134

----- Original Message ----- 
From: "Winders, Timothy A" <twinders at southplainscollege.edu>
To: <cisco-nsp at puck.nether.net>
Sent: Thursday, January 18, 2007 2:25 AM
Subject: [c-nsp] 2 locations, 2 ISPs, 2 ASAs, configuration confusion

> Hello -
>
> I have a multi-city network with two exit points to the internet.  For
> simplicity, this is what the physical network looks like:
>
> ISP1 --- RTR1 --- ASA1 --- CORE1 --- CORE2 --- ASA2 --- RTR2 --- ISP2
>
> CORE1 and CORE2 are in different cities.  They are connected via GigE
> with full VLAN trunking.  I've been chasing my tail for months trying to
> get a stable configuration working.  Everytime I think I have something
> working, it breaks in some other way.
>
> Here are the goals:
>
> 2 entry and exit points to the network
> Full firewall at both entry points to the network
> dynamic routing internal
> bgp to ISPs
> if the connection between CORE1 and CORE2 goes down, each site maintains
> internet connectivity independantly and announces routes to it's bgp
> peer for only the networks which remain local to it.
> optionally, if CORE1 and CORE2 lose their connection, a secure tunnel
> connection will be established over the internet to connect the sites
> again, transparent to users.
> All routers (edge and core) are SUP720 based.
> Firewalls are ASA5520s with ASA-SSM-20 IPS module
> asymetric routing must be taken into consideration
>
>
> After many failed attempts and several different cases with Cisco TAC, I
> am seeking guidance from the list.  TAC tells me this can't be done, but
> I don't believe it.  Many organizations have global networks with
> multiple ISP connections which must be secured and also maintain
> internal network links.  So, my rather simple network should be doable.
>
> My next configuration attempt will be as follows.  Please let me know
> what problems you see or suggestions/changes you have.
>
> RTR1 and ISP1 run eBGP
> RTR2 and ISP2 run eBGP
> RTR1 and RTR2 run iBGP
> internal network routing is OSPF.  Inject OSPF into BGP and announce to
> BGP peers.
> ASAs run in ACTIVE/ACTIVE multiple context failover.  (Must be
> ACTIVE/ACTIVE because each firewall must pass traffic simultaneously,
> right?)  ACTIVE/ACTIVE only supports multiple contexts.
> Firewall transparent mode.  (In routed mode, multiple contexts don't
> support dynamic routing protocols, but must be transparent so RTRx and
> COREx can speak OSPF.)
> Single admin context with inside/outside interface configurations
> asr-group on the outside interface
> failover link configured on 3rd ASA interface on dedicated VLAN for
> failover and stateful failover
>
> I think this will work.  But, there are some things still bothering me.
>
> In ACTIVE/ACTIVE mode, are all contexts passing traffic, or does it work
> like ACTIVE/STANDBY where one context is active on one ASA and in
> standby on the other?
>
> In failover configuration, is the complete configuration of the context
> shared between the two ASAs?  What about the system configuration?
>
> In Transparent mode, there is a note in the configuration guide which
> says "The transparent firewall requires a managment IP address... ...The
> management IP address must be on the same subnet as the connected
> network."  OK, that's fine, except, in multiple context mode, the IP
> address for management goes in the context configuration.  If the
> context configuration is shared, how can I have an IP address in the
> same subnet as the connected network when the two connected networks
> between ASA1 and ASA2 will be different?  One possibility...  if RTR1 is
> directly connected to ASA1 which is directly connected to CORE1 (and
> same for side 2), then, put the inside RTRx and ouside COREx interfaces
> in a single VLAN.  In that case, the logical network might look like the
> asr-group example in the configuration guide:
> http://www.cisco.com/en/US/products/ps6120/products_configuration_guide_
> chapter09186a008045247e.html#wp1102712
>
> The problem here is this example has context A and B and I can't tell if
> it's routed or transparent.  This is also where I get my confusion about
> contexts being active and standby in an active/active configuration (my
> first question above).  From that section of the guide: "The traffic is
> forwarded though the outside interface of context A on the unit where
> context A is in the standby state and returns through the outside
> interface of context A on the unit where context A is in the active
> state."  So, you see, this talks about an ACTIVE/ACTIVE configuration
> where one context is standby.  Hmmmm.
>
> Finally, in an active/active transparent configuration, should the ASA
> have DIRECT connections to the routers on it's inside and outside
> interfaces?  Or, could the router and asa connect to a switch in the
> middle and the packets just "know where to go"?  I'm thinking here...
> the ASAx outside interface and RTRx inside interface could be on the
> same non-routed VLAN.  Then, the ASAx inside interface and COREx outside
> interface could be on a different non-routed VLAN.  I'm not sure if this
> would work.  Then, if I did have the configuration where all that was on
> a single IP subnet, it's possible packets could end up coming in ISP1 to
> RTR1 to ASA2 back to CORE1 destined for servers hanging off of CORE1.
> If the routing were setup right, this *SHOULDN'T* happen, but, it could
> easily be the case if I weren't careful.
>
> So, you see how my head goes spinning round-and-round.  After working on
> this for months, I'm ready to get it working right and move on.
>
> Thanks for your help.
>
>
>
> Tim Winders | Associate Dean of Information Technology | South Plains
> College
>
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
>
>
>
> 