[c-nsp] 6509-IOS Firewalls causing RST storm that max CPU

[Grant Moerschel]

Hello all,
I have an environment with two 6509 switches linked together via a trunk 
passing one vlan.  Both chassis run 12.2.18sxd3 IOS firewall code.  We 
use ip inspect inbound on both sides of the vlan providing the link.  We 
are seeing strange behavior (often with ftp) that when the client ends a 
connection correctly using fins (fin, then fin, then ack, then ack), the 
server will shortly thereafter send a rst to the client which will kick 
off a rst storm ping pong match back and forth between the client and 
server to the point that it makes one sup2 go to 99% cpu and the sup720 
on the other side go to 80%.

Here's the strange thing.  The hosts are not doing it but the firewall 
function is. This was proven with sniffer traces showing zero traffic 
actually hitting the vlans where the hosts were during a storm. We think 
that the firewall thinks the host is under attack and is resetting the 
connection on the hosts behalf.  Bear in mind that this ios rev does 
_not_ support IP audit which is part of the cbac functionality. IP audit 
will allow you, on smaller platforms, to send rst packets in response to 
attacks. So this 6509 shouldn't even be doing this unless I am 
mistaken??  What am I missing?

I bumped the fin wait time to 15 seconds. Fin wait keeps the session in 
the firewall for x seconds after a session is torn down so that any 
other packets coming in won't ge bounced. Used with NFS I think which is 
not what we are doing but it was worth a try.

Here's a snippet from a flood I sniffed - starting at the end of a 
normal tcp session. Notice where the rst start going back and forth:

---
16:57:32.246355 IP 10.237.102.53.50270 > 10.232.188.200.80: S 
4081749102:4081749102(0) win 5840 <mss 1460,sackOK,timestamp 105526533 
0,nop,wscale 0>
16:57:32.286018 IP 10.232.188.200.80 > 10.237.102.53.50270: S 
482684002:482684002(0) ack 4081749103 win 16464 <mss 1372,nop,wscale 
0,nop,nop,timestamp 0 0,nop,nop,sackOK>
16:57:32.286185 IP 10.237.102.53.50270 > 10.232.188.200.80: . ack 1 win 
5840 <nop,nop,timestamp 105526537 0>
16:57:32.288077 IP 10.237.102.53.50270 > 10.232.188.200.80: P 1:329(328) 
ack 1 win 5840 <nop,nop,timestamp 105526537 0>
16:57:32.288157 IP 10.237.102.53.50270 > 10.232.188.200.80: P 
329:635(306) ack 1 win 5840 <nop,nop,timestamp 105526537 0>
16:57:32.327566 IP 10.232.188.200.80 > 10.237.102.53.50270: . ack 635 
win 15830 <nop,nop,timestamp 12111000 105526537>
16:57:32.476047 IP 10.232.188.200.80 > 10.237.102.53.50270: P 1:172(171) 
ack 635 win 15830 <nop,nop,timestamp 12111002 105526537>
16:57:32.476249 IP 10.237.102.53.50270 > 10.232.188.200.80: . ack 172 
win 6432 <nop,nop,timestamp 105526556 12111002>
16:57:32.478250 IP 10.232.188.200.80 > 10.237.102.53.50270: . 
172:1532(1360) ack 635 win 15830 <nop,nop,timestamp 12111002 105526537>
16:57:32.478316 IP 10.237.102.53.50270 > 10.232.188.200.80: . ack 1532 
win 9520 <nop,nop,timestamp 105526556 12111002>
16:57:32.518355 IP 10.232.188.200.80 > 10.237.102.53.50270: P 
2892:3203(311) ack 635 win 15830 <nop,nop,timestamp 12111002 105526556>
16:57:32.518503 IP 10.237.102.53.50270 > 10.232.188.200.80: . ack 1532 
win 9520 <nop,nop,timestamp 105526560 12111002,nop,nop,sack sack 1 
{2892:3203} >
16:57:32.518889 IP 10.232.188.200.80 > 10.237.102.53.50270: . 
1532:2892(1360) ack 635 win 15830 <nop,nop,timestamp 12111002 105526556>
16:57:32.518939 IP 10.237.102.53.50270 > 10.232.188.200.80: . ack 3203 
win 12240 <nop,nop,timestamp 105526560 12111002>
16:57:32.519095 IP 10.237.102.53.50270 > 10.232.188.200.80: F 635:635(0) 
ack 3203 win 12240 <nop,nop,timestamp 105526560 12111002>
16:57:32.560448 IP 10.232.188.200.80 > 10.237.102.53.50270: F 
3203:3203(0) ack 635 win 15830 <nop,nop,timestamp 12111003 105526560>
16:57:32.560464 IP 10.232.188.200.80 > 10.237.102.53.50270: . ack 636 
win 15830 <nop,nop,timestamp 12111003 105526560>
16:57:32.560629 IP 10.237.102.53.50270 > 10.232.188.200.80: . ack 3204 
win 12240 <nop,nop,timestamp 105526564 12111003>
16:57:32.600087 IP 10.232.188.200.80 > 10.237.102.53.50270: R 
482687205:482687205(0) win 0
16:57:32.600394 IP 10.237.102.53.50270 > 10.232.188.200.80: R 
482687205:482687205(0) win 15830
16:57:32.600581 IP 10.232.188.200.80 > 10.237.102.53.50270: R 0:0(0) win 
12240
16:57:32.600774 IP 10.237.102.53.50270 > 10.232.188.200.80: R 0:0(0) win 
15830
16:57:32.600953 IP 10.232.188.200.80 > 10.237.102.53.50270: R 0:0(0) win 
12240
16:57:32.601126 IP 10.237.102.53.50270 > 10.232.188.200.80: R 0:0(0) win 
15830
16:57:32.601298 IP 10.232.188.200.80 > 10.237.102.53.50270: R 0:0(0) win 
12240
16:57:32.601476 IP 10.237.102.53.50270 > 10.232.188.200.80: R 0:0(0) win 
15830
16:57:32.601649 IP 10.232.188.200.80 > 10.237.102.53.50270: R 0:0(0) win 
12240
16:57:32.601831 IP 10.237.102.53.50270 > 10.232.188.200.80: R 0:0(0) win 
15830
16:57:32.602002 IP 10.232.188.200.80 > 10.237.102.53.50270: R 0:0(0) win 
12240
16:57:32.602184 IP 10.237.102.53.50270 > 10.232.188.200.80: R 0:0(0) win 
15830
... and it keeps going ...

Grant Moerschel