<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 12 (filtered medium)"><!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Raja,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Thanks for the quick answer… I do have some insights here with a new question:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>First of all, was your NLB cluster setup as multicast, IGMP multicast, or unicast? The reason I ask is that we are using several NLB clusters setup for UNICAST (as we have dedicated NIC’s for NLB in each server, plus one for normal LAN activities), and it’s working just fine. No need for any static ARP or MAC in this situation from what we’ve seen. Either that, or I’m not seeing something here that I need to look into? We are NOT using the switch for routing any NLB traffic, that is handled solely by our firewalls. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>We run easily into the 30/70K packets per second on the clusters, and they all seem to be getting their fair share of load.. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>That being said, I can confirm that STATIC ARP on the SX’s have to be associated to a single port (you can’t enter it unless you specify one and only one port on the switch). Which is kind of dumb actually. ‘arp {static entry #} {IP address} {MAC address} eth {PORT/SLOT}’ on an SX-800 or Super-X. You can NOT get away from NOT SPECIFYING the ‘eth SLOT/PORT’ parameter.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>On a Cisco – you simply do an ‘arp 192.168.0.1 xxxx.xxxx.xxxx arpa’ to set the static arp .. then add your static mac to the port(s) you need.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Same goes for a Extreme Summit - ‘configured iparp add 192.168.0.1 xx:xx:xx:xx:xx:xx’ to set the static arp… then add the static MAC to each port.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I can’t understand why the Foundry would force the ARP to bind to a PORT on a specific MAC.. On a specific VLAN I could understand (broadcast domain makes sense). <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Inside of a vlan, I can set a static arp inspection ‘arp 192.168.0.1 xxxx.xxxx.xxxx inspect’ but that doesn’t really set a static arp entry.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Over-simplifying it a bit: IP address goes to ARP lookup which returns a MAC address. MAC lookup returns which port(s) it’s been seen on / been bound to. Send out the packet for that IP to the port(s) you found. Foundry’s method circumvents the MAC lookup by forcing the ARP lookup to do BOTH (saving a MAC lookup). But then that prevents the box from playing nice with NLB or other redundant / load-sharing devices that it’s plugged into. That’s a big functional hole in my view.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Anyways, if anyone from Foundry/Brocade looks at this post, and can shed some light as to why this is done instead of how some of the other big boys do it, I’d be really interested in hearing the answer..<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Raja, thanks for the heads-up on the 07.2.02d release… I’ll be upgrading soon – I tend to follow the “If it ain’t broke, don’t fix it” routine.. But I gather from your comment that something is INDEED BROKE.. so it needs fixing… Correct?<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I’d seriously like to see what IronWare 07.3.xx will bring – and if the static ARP functionality will now play nice with others and let our redundant / load-sharing solutions just work the way they should…<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Best regards,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Robert<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Raja Subramanian [mailto:rajasuperman@gmail.com] <br><b>Sent:</b> Wednesday, October 19, 2011 6:56 AM<br><b>To:</b> Robert Toth<br><b>Cc:</b> foundry-nsp@puck.nether.net<br><b>Subject:</b> Re: [f-nsp] Setup a WatchGuard Active-Active firecluster on FastIron SX-800 and/or Super-X<o:p></o:p></span></p></div><p class=MsoNormal><o:p> </o:p></p><p>Hi,<o:p></o:p></p><p>We had exactly the same problem with NLB on SX and FCX. All traffic would go to a single server, and the setup was never stable enough for production.<o:p></o:p></p><p>After a Brocade TAC case which went on for nearly 12 months, we concluded that multi port static ARP is not supported on the FastIron platform. Only multi port static MAC is supported.<o:p></o:p></p><p>You need to run L2 code on FastIron to get NLB, etc to work correctly.<o:p></o:p></p><p>We are hanging dual FCXs stacked and running L2 code. These are connected redundantly to dual SX800 L3 cores. Servers, firewalls, etc are connected to the FCX stack.<o:p></o:p></p><p>Suggest you run 7202d on your SX boxes. Don't run any other 7.x code release at this time.<o:p></o:p></p><p>- Raja<o:p></o:p></p><div><p class=MsoNormal>On Oct 19, 2011 2:14 PM, "Robert Toth" <<a href="mailto:rtoth@iperceptions.com">rtoth@iperceptions.com</a>> wrote:<o:p></o:p></p><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Hi everyone,<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Was just referred to this list, hopefully somebody can help me with a very stubborn problem here..<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>I have been fighting for a few months now to get my WatchGuard Firecluster's up and running with stability on both a Super-X and SX-800. Here's a quick overview:<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Datacenter: SX-800 with V07.2.00 Full Layer 3 software, redundant mgmt & fabric, and eight 424C modules.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>This is our core switch that handles all network connectivity to our datacenter servers.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>1 Layer-3 segment (VLAN-9) that is primary gateway for INTERNAL remote network routing across a 2GB dedicated fiber trunk to head office.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Multiple VLAN's used to partition both external and internal network as Layer-2 segments, with firewalls being used to manage and provide controls between them as needed (FastIron does NOT participate in Internal/External access).<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Fireboxes are the Peak 5500e series, with Fireware XTM Pro V11.3.4 - setup for Firecluster H/A support for load-sharing and redundancy. These ports on firebox are given MULTICAST MAC addresses, with UNICAST IP's.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Firewall ports are ALL plugged into various Layer-2 segments.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>On the primary trusted network, as well as every other Layer-2 segment (VLAN) the Firebox is the default gateway - however, there is a Layer-3 router on the SX-800 for that one segment that provides an alternate lower-cost route to distant internal networks via RIP so that internal traffic is sent directly across the 2GB trunk line instead of being sent through the firewall at slower speed (and higher load on the firewalls themselves handling inter-office traffic via VPN's). ACL's were in place to limit and control who had access across that link, but those were removed to faciliate debugging for the moment.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Head office: Similar to the above, but using a Super-X instead of the SX-800, and a pair of Firebox Core 1250e's - same release of software on both sets of devices.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>The instructions from Watchguard are clear: We need to setup Static ARP and MAC addresses as needed to correctly direct traffic to/from the switches and fireboxes. The samples given are for a Cisco 3750 and an Extreme Summit 15040 (ref: <a href="http://www.watchguard.com/help/docs/wsm/11/en-us/content/en-us/ha/cluster_example_cisco_wsm.html" target="_blank">http://www.watchguard.com/help/docs/wsm/11/en-us/content/en-us/ha/cluster_example_cisco_wsm.html</a>) . This is what I have tried to replicate to the best I can but have had no luck with setting the static arp address on the switch outside of the single layer-3 segment we have, and to set it on more than one port. Setting the multicast mac is fine for both ports needed to support the pair of firewalls by interface )(ex: static-mac xxx.xxxx.xxxx eth 1/yy eth 2/yy - works fine on every Vlan I need it on).<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Digging further into Google (which ended up pointing me at this list), I found this article about "hairpinning" to correctly support a specific setup of Microsoft NLB clusters (which we also use extensively at the datacenter) here: <a href="https://puck.nether.net/pipermail/foundry-nsp/2010-June/002498.html" target="_blank">https://puck.nether.net/pipermail/foundry-nsp/2010-June/002498.html</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>While I haven't found need to do this for the NLB clusters themselves as they appear to be working perfectly (up to 5 physical servers per cluster running IIS7 & Win2K8), it appeared to be the best solution to adapt for the Firecluster problem... And getting a static arp assigned to multiple ports.<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Anyways, long story short - it's not working well. It seems that the traffic is being forced from port to port by only having the ARP/MAC entry recognized on a single port at a time (and being kicked around constantly). Apparently, if this was a Cisco - this would work brilliantly.. But because we can't map static ARP's as we should, the switch is constantly moving things around instead of sending it out to both ports at the same time for each interface. <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Has ANYONE seen or tried this kind of setup using similar hardware ? If so, did you get it working correctly? Am I on the right track with the hairpinning solution?<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Is Brocade making any changes to Ironware to provide similar functionality so that I can get this working properly any time soon, or will I have to resort to a couple of dumb Layer-2 switches that apparently won't need any configuration and will just work, and cobble the whole thing together into a massive mess to get it all tied together? Are there any issues with WatchGuard Firecluster that I need to know about that make it work differently than documented, and prevent the FastIron's from being able to cope or perform as expected ( I realize that last question isn't really a Foundry question... My apologies, but I hope that someone else with Fireboxes has seen this problem on Foundry or perhaps found another solution using other hardware if the FastIron's or other Foundry products just couldn't do what was needed. This assumes that FireCluster will work given the right infrastructure setup - maybe THAT assumption is wrong? If so, I'd gladly listen to a solution that DOES work?)<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Best regards,<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b>Robert Toth</b><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:10.0pt'>Director of Information Technology Services</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b>iPerceptions Inc</b><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Tel: <a href="tel:514.488.3600" target="_blank">514.488.3600</a> ext. [284] | <a href="tel:877.796.3600" target="_blank">877.796.3600</a> ext. [284]<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Fax: <a href="tel:514.484.2600" target="_blank">514.484.2600</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>London - Montreal - New York - Toronto<o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>[<a href="mailto:rtoth@iperceptions.com" target="_blank">rtoth@iperceptions.com</a>] | <a href="http://www.iperceptions.com" target="_blank">www.iperceptions.com</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Subscribe to our blog: <a href="http://blog.iperceptions.com/" target="_blank">blog.iperceptions.com</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Follow us on Twitter: <a href="http://www.twitter.com/iperceptions" target="_blank">www.twitter.com/iperceptions</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>Register for our upcoming webinar “Is the check engine light ‘on’ for your analytics” at <a href="https://www1.gotomeeting.com/register/501539032" target="_blank">https://www1.gotomeeting.com/register/501539032</a><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:9.0pt'>The contents of this email and any attachments are confidential. They are intended for the named recipient(s) only and are not to be communicated to anyone else without permission. If you have received this email in error please notify the sender immediately and do not make copies or disclose the contents to anyone.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><img border=0 width=148 height=46 id="_x0000_i1025" src="cid:image001.png@01CC8ED1.DE95CC90" alt="cid:image001.png@01CB8183.17346640"><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div></div><p class=MsoNormal><br>_______________________________________________<br>foundry-nsp mailing list<br><a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br><a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank">http://puck.nether.net/mailman/listinfo/foundry-nsp</a><o:p></o:p></p></div></div></body></html>