[c-nsp] 2 locations, 2 ISPs, 2 ASAs, configuration confusion

[Winders, Timothy A]

Hello -
 
I have a multi-city network with two exit points to the internet.  For
simplicity, this is what the physical network looks like:
 
ISP1 --- RTR1 --- ASA1 --- CORE1 --- CORE2 --- ASA2 --- RTR2 --- ISP2
 
CORE1 and CORE2 are in different cities.  They are connected via GigE
with full VLAN trunking.  I've been chasing my tail for months trying to
get a stable configuration working.  Everytime I think I have something
working, it breaks in some other way.
 
Here are the goals:
 
2 entry and exit points to the network
Full firewall at both entry points to the network
dynamic routing internal
bgp to ISPs
if the connection between CORE1 and CORE2 goes down, each site maintains
internet connectivity independantly and announces routes to it's bgp
peer for only the networks which remain local to it.
optionally, if CORE1 and CORE2 lose their connection, a secure tunnel
connection will be established over the internet to connect the sites
again, transparent to users.
All routers (edge and core) are SUP720 based.
Firewalls are ASA5520s with ASA-SSM-20 IPS module
asymetric routing must be taken into consideration
 
 
After many failed attempts and several different cases with Cisco TAC, I
am seeking guidance from the list.  TAC tells me this can't be done, but
I don't believe it.  Many organizations have global networks with
multiple ISP connections which must be secured and also maintain
internal network links.  So, my rather simple network should be doable.
 
My next configuration attempt will be as follows.  Please let me know
what problems you see or suggestions/changes you have.
 
RTR1 and ISP1 run eBGP
RTR2 and ISP2 run eBGP
RTR1 and RTR2 run iBGP
internal network routing is OSPF.  Inject OSPF into BGP and announce to
BGP peers.
ASAs run in ACTIVE/ACTIVE multiple context failover.  (Must be
ACTIVE/ACTIVE because each firewall must pass traffic simultaneously,
right?)  ACTIVE/ACTIVE only supports multiple contexts.
Firewall transparent mode.  (In routed mode, multiple contexts don't
support dynamic routing protocols, but must be transparent so RTRx and
COREx can speak OSPF.)
Single admin context with inside/outside interface configurations
asr-group on the outside interface
failover link configured on 3rd ASA interface on dedicated VLAN for
failover and stateful failover
 
I think this will work.  But, there are some things still bothering me.
 
In ACTIVE/ACTIVE mode, are all contexts passing traffic, or does it work
like ACTIVE/STANDBY where one context is active on one ASA and in
standby on the other?
 
In failover configuration, is the complete configuration of the context
shared between the two ASAs?  What about the system configuration?
 
In Transparent mode, there is a note in the configuration guide which
says "The transparent firewall requires a managment IP address... ...The
management IP address must be on the same subnet as the connected
network."  OK, that's fine, except, in multiple context mode, the IP
address for management goes in the context configuration.  If the
context configuration is shared, how can I have an IP address in the
same subnet as the connected network when the two connected networks
between ASA1 and ASA2 will be different?  One possibility...  if RTR1 is
directly connected to ASA1 which is directly connected to CORE1 (and
same for side 2), then, put the inside RTRx and ouside COREx interfaces
in a single VLAN.  In that case, the logical network might look like the
asr-group example in the configuration guide:
http://www.cisco.com/en/US/products/ps6120/products_configuration_guide_
chapter09186a008045247e.html#wp1102712
 
The problem here is this example has context A and B and I can't tell if
it's routed or transparent.  This is also where I get my confusion about
contexts being active and standby in an active/active configuration (my
first question above).  From that section of the guide: "The traffic is
forwarded though the outside interface of context A on the unit where
context A is in the standby state and returns through the outside
interface of context A on the unit where context A is in the active
state."  So, you see, this talks about an ACTIVE/ACTIVE configuration
where one context is standby.  Hmmmm.
 
Finally, in an active/active transparent configuration, should the ASA
have DIRECT connections to the routers on it's inside and outside
interfaces?  Or, could the router and asa connect to a switch in the
middle and the packets just "know where to go"?  I'm thinking here...
the ASAx outside interface and RTRx inside interface could be on the
same non-routed VLAN.  Then, the ASAx inside interface and COREx outside
interface could be on a different non-routed VLAN.  I'm not sure if this
would work.  Then, if I did have the configuration where all that was on
a single IP subnet, it's possible packets could end up coming in ISP1 to
RTR1 to ASA2 back to CORE1 destined for servers hanging off of CORE1.
If the routing were setup right, this *SHOULDN'T* happen, but, it could
easily be the case if I weren't careful.
 
So, you see how my head goes spinning round-and-round.  After working on
this for months, I'm ready to get it working right and move on.
 
Thanks for your help.
 
 

Tim Winders | Associate Dean of Information Technology | South Plains
College