[c-nsp] ASA 5585-X SSP-10 multi-context failover not stateful with IPv4

[Vinny_Abello at Dell.com]

Hi all,

I have a bizarre situation that isn't making sense to me.

I have two ASA 5585-X firewalls with SSP-10. They are in an Active/Standby configuration and running in multi-context mode. I have replication of state information between them working just fine. We're running both IPv4 and IPv6 and have the latest 9.1(2) code loaded.

The problem is if I force a failover from the system context, any open connections over IPv4 coming in the outside interface of a context via a NAT translation seems to get lost during the failover. I'm not positive if it's the state table or the NAT table that is having an issue or if they are one in the same on the ASA. The interesting part is my IPv6 connectivity persists without any problems during the failover. I can be transferring a file via FTP or stay connected via RDP to the machines behind the firewall (Windows servers) over IPv6 and everything is seamless as it should be. If I am connected via RDP over IPv4, the connection hangs, eventually resets and reconnects.

Nothing looks out of the ordinary as far as I can tell. This is the first environment I've worked on with the ASA's in multi-context mode.

>From the system context, this is the failover configuration:

failover
failover lan unit primary
failover lan interface failover GigabitEthernet0/7
failover replication http
failover mac address GigabitEthernet0/0 acf2.c5f2.d301 acf2.c5f2.d302
failover mac address GigabitEthernet0/1 acf2.c5f2.d311 acf2.c5f2.d312
failover mac address GigabitEthernet0/2 acf2.c5f2.d321 acf2.c5f2.d322
failover mac address GigabitEthernet0/3 acf2.c5f2.d331 acf2.c5f2.d332
failover mac address GigabitEthernet0/4 acf2.c5f2.d341 acf2.c5f2.d342
failover mac address GigabitEthernet0/5 acf2.c5f2.d351 acf2.c5f2.d352
failover mac address GigabitEthernet0/6 acf2.c5f2.d361 acf2.c5f2.d362
failover mac address TenGigabitEthernet0/8 acf2.c5f2.d393 acf2.c5f2.d394
failover link failover GigabitEthernet0/7
failover interface ip failover 172.16.255.1 255.255.255.0 standby 172.16.255.2

At first I thought it was some type of ARP issue which is why I have configured the mac addresses for primary and secondary units. I read the following in the Active/Standby guide:

If you do not configure virtual MAC addresses, you might need to clear the ARP tables on connected routers to restore traffic flow. The ASA does not send gratuitous ARPs for static NAT addresses when the MAC address changes, so connected routers do not learn of the MAC address change for these addresses.

That is the reason for the MAC address configuration above but it didn't seem to help.

All interfaces show Normal (Monitored) both in the system context and in the context in question. Stateful update statistics show the following in the system context:

Stateful Failover Logical Update Statistics
        Link : failover GigabitEthernet0/7 (up)
        Stateful Obj    xmit       xerr       rcv        rerr
        General         45334      0          1417726    20242
        sys cmd         31679      0          31678      0
        up time         0          0          0          0
        RPC services    0          0          0          0
        TCP conn        9634       0          1012694    327
        UDP conn        2055       0          196454     0
        ARP tbl         833        0          87033      0
        Xlate_Timeout   0          0          0          0
        IPv6 ND tbl     916        0          89861      280
        VPN IKEv1 SA    0          0          0          0
        VPN IKEv1 P2    0          0          0          0
       VPN IKEv2 SA    0          0          0          0
        VPN IKEv2 P2    0          0          0          0
        VPN CTCP upd    0          0          0          0
        VPN SDI upd     0          0          0          0
        VPN DHCP upd    0          0          0          0
        SIP Session     0          0          0          0
        Route Session   214        0          0          19635
        User-Identity   3          0          6          0
        CTS SGTNAME     0          0          0          0
        CTS PAC         0          0          0          0
        TrustSec-SXP    0          0          0          0
        IPv6 Route      0          0          0          0

        Logical Update Queue Information
                        Cur     Max     Total
        Recv Q:         0       39      1888781
        Xmit Q:         0       3       49646

In my context where the servers are sitting it shows the following:

Stateful Failover Logical Update Statistics
        Status: Configured.
        Stateful Obj    xmit       xerr       rcv        rerr
        RPC services    0          0          0          0
        TCP conn        793        0          1186       8
        UDP conn        735        0          420        0
        ARP tbl         151        0          206        0
        Xlate_Timeout   0          0          0          0
        IPv6 ND tbl     159        0          196        4
        VPN IKEv1 SA    0          0          0          0
        VPN IKEv1 P2    0          0          0          0
        VPN IKEv2 SA    0          0          0          0
        VPN IKEv2 P2    0          0          0          0
        VPN CTCP upd    0          0          0          0
        VPN SDI upd     0          0          0          0
        VPN DHCP upd    0          0          0          0
        SIP Session     0          0          0          0
        Route Session   14         0          0          12
        User-Identity   0          0          1          0
        CTS SGTNAME     0          0          0          0
        CTS PAC         0          0          0          0
        TrustSec-SXP    0          0          0          0
        IPv6 Route      0          0          0          0


Any suggestions or am I overlooking something obvious? I was going to open a TAC case, but figured I'd ask here first in case this looks familiar to anyone.

Thanks!

-Vinny