[f-nsp] Multiple VIPs with Different Subnets

[David Paul Zimmerman]

For what it's worth, I'm serving multiple subnets too on my SI GTs,  
currently with 6 server subnets and 1 management subnet (which wins  
the default route).  My requirements are somewhat simpler, though,  
than the cases below.  I treat the SI as a multihomed host, so with  
the router code running and "route-only" configured per interface (or  
globally, but a bug with that is fixed only very recently) each  
physical interface is simply an L3 port on a different subnet.  I  
require DSR for all servers using this SI, so all of the symmetry from  
a subnet firewall/router perspective is maintained.

	dp

On Dec 10, 2009, at 1:19 PM, Jack Stewart wrote:

>
> Hi Lazuardi,
>
> I'm running the routing code. To the best of my knowledge, route- 
> maps apply only to the routing code.
>
> route-maps allow you to set the gateway by source address,  
> destination address, or port #. Static routes allow you to set  
> gateway by destination. The VIP is the source address of the  
> outgoing traffic. So in the example below, gateway for a VIP is  
> based on its address (and not the destination client). This was a  
> really hard concept for me to wrap my head around.
>
> It isn't clear to me that your case is the same. My setup is very  
> atypical. I had a lot of trouble debugging it (traffic would leave  
> the client but never come back to it).  You might be able to get at  
> it by looking at the interface traffic of the gateways.
>
> Let me know the solution you come up with, I'm curious.
>
> ---Jack
>
> Lazuardi Nasution wrote:
>> Hi Jack,
>> I think this solution is for Switch Code since with Router Code I can
>> have many Management IP even with different subnets. The default
>> gateway can be specified statically on the routing table or by using
>> routing protocol from the routers.
>> Best regards,
>> On Fri, Dec 11, 2009 at 2:25 AM, Jack Stewart  
>> <jstewart at caltech.edu> wrote:
>>> Hi Lazuardi,
>>>
>>> I ran into similar issues - this is what ultimately work in my  
>>> environment.
>>> It may not be the same but hopefully there are some takeaways.  
>>> Please let me
>>> know how much of this makes sense - that feedback will be helpful  
>>> with my
>>> documentation.
>>>
>>> First, DNS is special and the following is generic.
>>>
>>> You can only have one management IP and one default route. The  
>>> management IP
>>> should live on the same subnet that has the default route. The  
>>> first public
>>> subnet with the management IP & default route (Pub_Subnet_1) was  
>>> not a
>>> problem.
>>>
>>> All of my real servers are on a different private subnet that the  
>>> two public
>>> subnets and they all have the Load Balancer defined as their default
>>> gateway.
>>>
>>> To get subnet 2 (pub_subnet_2), I needed to define a router  
>>> interface for
>>> that subnetwork (ve2) and policy routing/route-maps. The route- 
>>> maps are for
>>> making sure that the return traffic goes out via the same gateway  
>>> that it
>>> came in for non directly attached subnets. The way the mapping  
>>> works for me
>>> in the configuration is:
>>>
>>> !
>>> ip access-list extended match_pub_subnet_2
>>> permit ip match_pubsub2/24 any
>>> !
>>> route-map more_default_routes permit 10
>>> match ip address match_subnet_2
>>> set ip next-hop subnet_2_gateway
>>> !
>>> ip policy prefer-direct-route
>>> ip policy route-map more_default_routes
>>>
>>> In route-maps, the 'permit #' is just the precedence order. You  
>>> can add
>>> additional entries to a route-map. Route-maps are processed before  
>>> static
>>> routes.
>>>
>>> Lastly, I defined an outside NAT policy on Public_Subnet_1 for  
>>> traffic
>>> originating private subnet traffic (i.e. directly attached  
>>> servers). I'm not
>>> 100% sure this is a requirement but it helps with traceroute, etc.
>>>
>>> In my case it was necessary to add VRRP but that is because I've  
>>> more than
>>> one box and it isn't clear you need that.
>>>
>>> Once this was done, everything worked nicely from outside to inside.
>>>
>>> This is a global static approach. Most people seem to route-maps  
>>> to filter
>>> routing protocols, but I'm not allowed to exchange LB routing  
>>> protocols with
>>> our routers by policy.
>>>
>>> For VIPs and real servers on the same private subnet, I found that  
>>> either
>>> DSR or source-nat with ACL's works well. If you are using DSR with  
>>> Linux (it
>>> seems to apply to other 2.6 kernels), you'll probably want to look  
>>> at the
>>> brocade wiki).
>>>
>>> With DNS, source-nat with ACL's is probably the simplest and  
>>> easier way to
>>> go.
>>>
>>> ---Jack
>>>
>>> Lazuardi Nasution wrote:
>>>> Hi Jack,
>>>>
>>>> Yes, there is different router per subnet and I have done the  
>>>> static
>>>> routing for that. VIP1 is in the same subnet with Management IP and
>>>> the Router1 is connected to eth1, so I just simply put Management  
>>>> IP
>>>> on eth1. Since Router2 is connected to eth2, should I do  
>>>> something on
>>>> eth2, ex. put another management IP on the eth2 which is in the  
>>>> same
>>>> subnet with VIP2 ? The other ethernet ports are for Real Server  
>>>> so I
>>>> have give ve1 for those ports.
>>>>
>>>> There is another weird problem. I have made DNS binding from VIP1  
>>>> and
>>>> RE1 and I have put ve1 IP in the same subnet with RE1. RE1 default
>>>> gateway is ve1 IP. I can query the DNS through VIP1 but RE1  
>>>> cannot do
>>>> traceroute to the Internet, stuck on the ServerIron. What's happen
>>>> here ?
>>>>
>>>> Best regards,
>>>>
>>>> On Thu, Dec 10, 2009 at 3:41 AM, Jack Stewart  
>>>> <jstewart at caltech.edu>
>>>> wrote:
>>>>> Hi Lazuardi,
>>>>>
>>>>> Yeah! A question that might be up my alley. I've done this  
>>>>> however I need
>>>>> some more details.
>>>>>
>>>>> Do these VIPs need different "static" default gateways on a per  
>>>>> subnet
>>>>> basis? It's possible with the routing code and I can send out  
>>>>> the details
>>>>> if
>>>>> you are interested.
>>>>>
>>>>> Otherwise the main trick with subnet A to subnet B traffic is to  
>>>>> make
>>>>> sure
>>>>> that the return traffic goes though the load balancer. The  
>>>>> client &
>>>>> server
>>>>> need to see the Load Balancer as the gateway between subnet A &  
>>>>> subnet B.
>>>>> DSR and source NAT are also options.
>>>>>
>>>>> So more details, please. Depending on what you need to do it  
>>>>> might help
>>>>> knock out some of my documentation.
>>>>>
>>>>> ---Jack
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Lazuardi Nasution wrote:
>>>>>> Hi,
>>>>>>
>>>>>> Is it possible to have multiple VIPs with different Subnets on
>>>>>> ServerIron 4G or ServerIron ADX1000 ? How can I do that ? I'm  
>>>>>> using
>>>>>> router code of firmware.
>>>>>>
>>>>>> Best regards,
>>>>>> _______________________________________________
>>>>>> foundry-nsp mailing list
>>>>>> foundry-nsp at puck.nether.net
>>>>>> http://puck.nether.net/mailman/listinfo/foundry-nsp