[c-nsp] ASA 5520 icmp error inspection not functioning after upgrade

[Vinny_Abello at Dell.com]

Hi ASA firewall gurus,

I recently upgraded a pair of ASA 5520's from 8.2(5)48 up to 9.1(5). I followed the outlined upgrade path. I've got a DMZ with public IP's and no NAT involved on one interface. Here, everything works as expected. The is another inside interface which has dynamic NAT setup to an external IP address. After the upgrade I found that through the NAT I wasn't able to ping or traceroute from the inside network as I had previously been able to do. I've always allowed echo-reply in the outside interface as well as ttl-exceeded in the access-list applied to it. I eventually found that the global_policy policy map which references the class inspection_default didn't have icmp inspection or icmp error inspection enabled. I enabled both and pings started working. The odd thing is that even with icmp error inspection enabled, traceroutes still do not work through NAT like they did in 8.2. It's behaving exactly like icmp error inspection isn't enabled, but it clearly is and I've removed and added it back multiple times. The icmp inspection does affect pings from the inside network if I remove and add it back, but icmp error inspection never permits traceroute to work through the NAT regardless of adding and removing it. Again, traceroute works fine from the DMZ with no NAT and traceroute also works fine via IPv6 from both the DMZ and inside networks. It's only a problem with NAT.

I've been going through this configuration trying to find anything that could be causing this but I'm coming up short. No packet captures or debugs are showing anything useful either other than confirming the TTL exceeded messages are hitting the outside interface from each hop. It seems the ASA is just not translating it and passing it through to the originator running the traceroute. I've got other firewalls that are also running a 9.1(x) version that have no problem including 9.1(5). I'm beginning to think this is some sort of artifact of having upgrading the Active/Passive pair up through 9.1(5) and never having fully taken down the set at the same time as I was trying to maintain uptime and doing so was supported by Cisco as long as I follow the proper upgrade path.

Has anyone seen something like this before? I'm thinking I might just schedule a time to reboot both simultaneously at this point. If that fails, I might need to go to Cisco TAC, but I thought I'd ask around first.

Thanks!

-Vinny