[f-nsp] Quick question.

[Drew Weaver]

Sorry for the length of this message, I edited it several times to try and brief it up a bit.

We have a ServerIron XL which is acting as a L2 switch and a load balancer for a group of servers which is connected to a Firewall.

Here is a slight diagram.

Users -> Rest of Network/gateway -> ServerIron XL -> Firewall -> Servers - > Application

Everything works, but I am noticing a few oddities that I am trying to work out.

The clients connect to the VIP, the data transfers everything is happy and that all works, then it looks like for some reason try to send an RST to the real server.

2008-12-31 15:16:15 Deny x 10.1.0.134 http/tcp 2859 80 0-External unknown TCP RST packet without an associated connection, firewall drop 40 238 (internal policy)  tcpinfo="offset 5 R 2648464113 win 0" rc="104" 	Traffic
2008-12-31 15:16:15 Deny x 10.1.0.133 http/tcp 49863 80 0-External unknown TCP RST packet without an associated connection, firewall drop 40 241 (internal policy)  tcpinfo="offset 5 R 2680363108 win 0" rc="104" 	Traffic
2008-12-31 15:16:19 Deny x 10.1.0.134 http/tcp 2866 80 0-External unknown TCP RST packet without an associated connection, firewall drop 40 248 (internal policy)  tcpinfo="offset 5 AR 2025327831 win 0" rc="104" 	Traffic
2008-12-31 15:16:19 Deny x 10.1.0.134 http/tcp 2860 80 0-External unknown TCP RST packet without an associated connection, firewall drop 40 248 (internal policy)  tcpinfo="offset 5 AR 3249673470 win 0" rc="104" 	Traffic
2008-12-31 15:16:30 Deny x 10.1.0.133 http/tcp 65374 80 0-External unknown TCP RST packet without an associated connection, firewall drop 40 248 (internal policy)  tcpinfo="offset 5 AR 4265145048 win 0" rc="104" 	Traffic
2008-12-31 15:16:38 Deny x 10.1.0.133 http/tcp 1915 80 0-External unknown TCP RST packet without an associated connection, firewall drop 40 238 (internal policy)  tcpinfo="offset 5 AR 689393954 win 0" rc="104" 	Traffic
2008-12-31 15:16:48 Deny x 10.1.0.134 http/tcp 50167 80 0-External unknown TCP RST packet without an associated connection, firewall drop 40 117 (internal policy)  tcpinfo="offset 5 AR 3901071967 win 0" rc="104" 	Traffic

The firewall obviously blocks these packets.

The other odd thing I've been noticing is that the connection count between 'something' and 'something else' continues to increase over time.

That sounds odd, let me give you an example of what I mean.

The web servers themselves show 300 connections each, the load balancer shows a steady 300 connections, then that count begins creeping up, and then eventually it gets up to the 4000 connections per server range.

At this point the web servers still all show they are at 300 connections each with hardly any load.

Also, I notice that when the connection numbers in the ServerIron begin spiking that the 'arbitrary' connection limit that watchguard places on their hardware ends up getting reached which means that somewhere connections are being created and not being destroyed between the firewall and the ServerIron.

I also noticed that the number of sessions is ridiculously high for the number of connections:

sz5           6        382   8594      
sz3           6         95    2009       
sz4           6        400   9251     

It seems like somewhere connections and sessions just aren't getting shut down between the ServerIron and the Firewall. I've contacted support but that has been somewhat slow going. 

Does anyone have any suggestions?

Thanks,
-Drew