[c-nsp] 2 locations, 2 ISPs, 2 ASAs, configuration confusion
Winders, Timothy A
twinders at southplainscollege.edu
Mon Jan 22 16:58:44 EST 2007
I met with my Cisco SE today. This is the logical configuration we have
come up with. I'm waiting on one more hardware piece before I can test
it out. Anyone have any feedback here. From below, here is the summary
of goals:
* 2 entry and exit points to the network
* Full firewall at both entry points to the network
* dynamic routing internal
* bgp to ISPs
* if the connection between CORE1 and CORE2 goes down, each site
maintains internet connectivity independantly and announces routes to
it's bgp peer for only the networks which remain local to it.
* optionally, if CORE1 and CORE2 lose their connection, a secure tunnel
connection will be established over the internet to connect the sites
again, transparent to users.
* All routers (edge and core) are SUP720 based.
* Firewalls are ASA5520s with ASA-SSM-20 IPS module
* asymetric routing must be taken into consideration
Internet
/ \
ISP1 ISP2
| |
ER1 --- ER2
| |
ASA1 ASA2
| |
CORE1 --- CORE2
1 and 2 are two different sites with a GigE connection between them.
The connections between CORE1/CORE2 and ER1/ER2 are across the same
physical link, but are on different VLANs. We have full dot1Q trunking
between sites.
ER1 runs eBGP to ISP1
ER2 runs eBGP to ISP2
ER1 and ER2 run iBGP
ER1 runs iBGP with CORE1
ER2 runs iBGP with CORE2
CORE1 and CORE2 run OSPF
The two ASA run independently in routed firewall mode. We take full
routes plus default from each ISP1 and ISP2. We inject OSPF into iBGP.
We create a high admin distance between CORE1 and CORE2.
With this configuration, the default route for each COREx router will be
it's local edge router. The Edge routers will decide which ISP to send
the traffic OUT. Incoming traffic can be asymetric from the ISPs, that
is traffic going OUT ER1 can come back in ER2. However, through iBGP
routing, the inflow of traffic should always come back in through the
correct ASA and we shouldn't have to worry about stateful inspection
dropped packets.
After this is working, I'll build a tunnel between CORE1 and CORE2
across the internet, so if the link between COREs goes down, the two
sites will have connectivity over the Internet.
I think this will all work. Comments?
Tim Winders | Associate Dean of Information Technology | South Plains
College
> -----Original Message-----
> From: cisco-nsp-bounces at puck.nether.net
> [mailto:cisco-nsp-bounces at puck.nether.net] On Behalf Of
> Winders, Timothy A
> Sent: Friday, January 19, 2007 10:04 AM
> To: Adam Greene; cisco-nsp at puck.nether.net
> Subject: Re: [c-nsp] 2 locations, 2 ISPs, 2 ASAs,
> configuration confusion
>
> Yes! Thank you, Adam. This is exactly the information I need.
>
> Some more questions...
>
> Are your PIX's running in failover mode with each other?
> Do you run routed or transparent mode?
>
> What is the "conditional BGP functionality"? Can you send me the
> configuration settings for that?
>
> I *can* forego the ASR requirement, if that's what it will take. As
> long as we can have the redundant connections and as long as
> we can set
> it up so that if the internet network breaks the two sites can connect
> to each other through the Internet, then this will fullfill
> the rest of
> my requirements.
>
> Do you have physical and logical diagrams of these network connections
> available?
>
> Tim Winders | Associate Dean of Information Technology | South Plains
> College
>
>
>
> > -----Original Message-----
> > From: Adam Greene [mailto:maillist at webjogger.net]
> > Sent: Friday, January 19, 2007 8:27 AM
> > To: Winders, Timothy A; cisco-nsp at puck.nether.net
> > Subject: Re: [c-nsp] 2 locations, 2 ISPs, 2 ASAs,
> > configuration confusion
> >
> > Timothy,
> >
> > We are doing what you are trying to do, with the exception
> > that we are not
> > allowing asymmetric routing through the (in this case) PIXes.
> >
> > - we run eBGP over both connections to the Internet (to diverse
> > providers)
> > - we run iBGP between the gateway routers facing the Internet
> > - we run OSPF on the internal network
> >
> > In our network configuration:
> > - customers in city A transmit / receive Internet data via
> > provider A
> > - customers in city B transmit / receive Internet data via
> > provider B
> > - if the link to provider A goes down, city A customers
> > are routed to
> > provider B
> > - if the link to provider B goes down, city B customers
> > are routed to
> > provider A
> >
> > In basic terms:
> > - we receive a default route from both our providers
> > - we inject the default routes into OSPF
> > - the gateway routers send default routes to each other
> > via iBGP tagged
> > with a community string
> > - devices in city A prefer the default route to provider A
> > - devices in city B prefer the default route to provider B
> > - during normal operations, we advertise only the
> > netblocks in city A to
> > provider A
> > - during normal operations, we advertise only the
> > netblocks in city B to
> > provider B
> > - utilizing conditional BGP functionality, the gateway
> > router in city A
> > only advertises the city B netblocks to provider A if it can
> > still see those
> > netblocks via OSPF, and if it is not receiving the default
> > route from its
> > iBGP peer
> > - utilizing conditional BGP functionality, the gateway
> > router in city B
> > only advertises the city A netblocks to provider B if it can
> > still see those
> > netblocks via OSPF, and if it is not receiving the default
> > route from its
> > iBGP peer
> >
> > In this way, we in effect form a redundant ring with the
> > Internet. If there
> > is a breakage anywhere on the ring, the devices on either
> side of the
> > breakage transmit / receive Internet data via the provider
> > they can still
> > "see".
> >
> > Not sure if this helps -- I think the main concept here which
> > makes it work
> > is being willing to forego the asymmetric routing requirement.
> >
> > Thanks,
> > Adam
> >
> > ---
> > Adam Greene
> > Webjogger Internet Services
> > http://www.webjogger.net
> > (845) 757-4000 x134
> >
> >
> >
> > ----- Original Message -----
> > From: "Winders, Timothy A" <twinders at southplainscollege.edu>
> > To: <cisco-nsp at puck.nether.net>
> > Sent: Thursday, January 18, 2007 2:25 AM
> > Subject: [c-nsp] 2 locations, 2 ISPs, 2 ASAs, configuration
> confusion
> >
> >
> > > Hello -
> > >
> > > I have a multi-city network with two exit points to the
> > internet. For
> > > simplicity, this is what the physical network looks like:
> > >
> > > ISP1 --- RTR1 --- ASA1 --- CORE1 --- CORE2 --- ASA2 ---
> > RTR2 --- ISP2
> > >
> > > CORE1 and CORE2 are in different cities. They are
> > connected via GigE
> > > with full VLAN trunking. I've been chasing my tail for
> > months trying to
> > > get a stable configuration working. Everytime I think I
> > have something
> > > working, it breaks in some other way.
> > >
> > > Here are the goals:
> > >
> > > 2 entry and exit points to the network
> > > Full firewall at both entry points to the network
> > > dynamic routing internal
> > > bgp to ISPs
> > > if the connection between CORE1 and CORE2 goes down, each
> > site maintains
> > > internet connectivity independantly and announces routes
> to it's bgp
> > > peer for only the networks which remain local to it.
> > > optionally, if CORE1 and CORE2 lose their connection, a
> > secure tunnel
> > > connection will be established over the internet to connect
> > the sites
> > > again, transparent to users.
> > > All routers (edge and core) are SUP720 based.
> > > Firewalls are ASA5520s with ASA-SSM-20 IPS module
> > > asymetric routing must be taken into consideration
> > >
> > >
> > > After many failed attempts and several different cases with
> > Cisco TAC, I
> > > am seeking guidance from the list. TAC tells me this can't
> > be done, but
> > > I don't believe it. Many organizations have global networks with
> > > multiple ISP connections which must be secured and also maintain
> > > internal network links. So, my rather simple network
> > should be doable.
> > >
> > > My next configuration attempt will be as follows. Please
> > let me know
> > > what problems you see or suggestions/changes you have.
> > >
> > > RTR1 and ISP1 run eBGP
> > > RTR2 and ISP2 run eBGP
> > > RTR1 and RTR2 run iBGP
> > > internal network routing is OSPF. Inject OSPF into BGP and
> > announce to
> > > BGP peers.
> > > ASAs run in ACTIVE/ACTIVE multiple context failover. (Must be
> > > ACTIVE/ACTIVE because each firewall must pass traffic
> > simultaneously,
> > > right?) ACTIVE/ACTIVE only supports multiple contexts.
> > > Firewall transparent mode. (In routed mode, multiple
> contexts don't
> > > support dynamic routing protocols, but must be transparent
> > so RTRx and
> > > COREx can speak OSPF.)
> > > Single admin context with inside/outside interface configurations
> > > asr-group on the outside interface
> > > failover link configured on 3rd ASA interface on
> dedicated VLAN for
> > > failover and stateful failover
> > >
> > > I think this will work. But, there are some things still
> > bothering me.
> > >
> > > In ACTIVE/ACTIVE mode, are all contexts passing traffic, or
> > does it work
> > > like ACTIVE/STANDBY where one context is active on one ASA and in
> > > standby on the other?
> > >
> > > In failover configuration, is the complete configuration of
> > the context
> > > shared between the two ASAs? What about the system configuration?
> > >
> > > In Transparent mode, there is a note in the configuration
> > guide which
> > > says "The transparent firewall requires a managment IP
> > address... ...The
> > > management IP address must be on the same subnet as the connected
> > > network." OK, that's fine, except, in multiple context
> mode, the IP
> > > address for management goes in the context configuration. If the
> > > context configuration is shared, how can I have an IP
> address in the
> > > same subnet as the connected network when the two
> connected networks
> > > between ASA1 and ASA2 will be different? One
> > possibility... if RTR1 is
> > > directly connected to ASA1 which is directly connected to
> CORE1 (and
> > > same for side 2), then, put the inside RTRx and ouside
> > COREx interfaces
> > > in a single VLAN. In that case, the logical network might
> > look like the
> > > asr-group example in the configuration guide:
> > >
> > http://www.cisco.com/en/US/products/ps6120/products_configurat
> > ion_guide_
> > > chapter09186a008045247e.html#wp1102712
> > >
> > > The problem here is this example has context A and B and I
> > can't tell if
> > > it's routed or transparent. This is also where I get my
> > confusion about
> > > contexts being active and standby in an active/active
> > configuration (my
> > > first question above). From that section of the guide:
> > "The traffic is
> > > forwarded though the outside interface of context A on the
> > unit where
> > > context A is in the standby state and returns through the outside
> > > interface of context A on the unit where context A is in
> the active
> > > state." So, you see, this talks about an ACTIVE/ACTIVE
> > configuration
> > > where one context is standby. Hmmmm.
> > >
> > > Finally, in an active/active transparent configuration,
> > should the ASA
> > > have DIRECT connections to the routers on it's inside and outside
> > > interfaces? Or, could the router and asa connect to a
> switch in the
> > > middle and the packets just "know where to go"? I'm
> > thinking here...
> > > the ASAx outside interface and RTRx inside interface
> could be on the
> > > same non-routed VLAN. Then, the ASAx inside interface and
> > COREx outside
> > > interface could be on a different non-routed VLAN. I'm not
> > sure if this
> > > would work. Then, if I did have the configuration where
> > all that was on
> > > a single IP subnet, it's possible packets could end up
> > coming in ISP1 to
> > > RTR1 to ASA2 back to CORE1 destined for servers hanging off
> > of CORE1.
> > > If the routing were setup right, this *SHOULDN'T* happen,
> > but, it could
> > > easily be the case if I weren't careful.
> > >
> > > So, you see how my head goes spinning round-and-round.
> > After working on
> > > this for months, I'm ready to get it working right and move on.
> > >
> > > Thanks for your help.
> > >
> > >
> > >
> > > Tim Winders | Associate Dean of Information Technology |
> > South Plains
> > > College
> > >
> > >
> > > _______________________________________________
> > > cisco-nsp mailing list cisco-nsp at puck.nether.net
> > > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > > archive at http://puck.nether.net/pipermail/cisco-nsp/
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
> >
> >
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>
More information about the cisco-nsp
mailing list