[c-nsp] "Strange" Cisco ASA5520 errors - Connection limit exceeded

[Peter Adkins]

Hi all,

The scenario is that we have two 5520s for this environment configured for
fail-over, these devices currently terminate a whopping 2x L2L IPSec VPNs
and a handful of SSL VPN sessions.

This morning we encountered a strange issue which was originally believed to
be due to ACLs not permitting traffic; effectively, if I were to log in to
one of the configured SSL VPNs I was unable to connect to any services
configured to be permitted through the VPN filter.  As a last ditch effort
to work out what was wrong I permitted ANY IP traffic through to the
required network, however, this still didn't fix the issue.

As an example of what we were seeing, when attempts to telnet into TCP port
1433 were failing, the following was found in the logs:

    ...
    %ASA-3-201011: Connection limit exceeded -35/5000 for input packet from
X.X.X.X/65374 to Y.Y.Y.Y/1433 on interface outside
    %ASA-3-201011: Connection limit exceeded -35/5000 for input packet from
X.X.X.X/65374 to Y.Y.Y.Y/1433 on interface outside
    %ASA-3-201011: Connection limit exceeded -35/5000 for input packet from
X.X.X.X/65374 to Y.Y.Y.Y/1433 on interface outside
    %ASA-3-201011: Connection limit exceeded -35/5000 for input packet from
X.X.X.X/65375 to Y.Y.Y.Y/1433 on interface outside
    %ASA-3-201011: Connection limit exceeded -35/5000 for input packet from
X.X.X.X/65375 to Y.Y.Y.Y/1433 on interface outside
    %ASA-3-201011: Connection limit exceeded -35/5000 for input packet from
X.X.X.X/65375 to Y.Y.Y.Y/1433 on interface outside
    ...

The Cisco website indicates that these sorts of messages would be presented
if the configured connection limits were, well, exceeded. However, I am
slightly perplexed as to the current count staying at -35 for all reported
messages -- as there was a large number of them.

    ...
    Interface outside:
      Service-policy: CONNS
        Class-map: CONNS
          Set connection policy: conn-max 5000 embryonic-conn-max 30
            current embryonic conns 0, current conns -35, drop 5622
          Set connection timeout policy:
            embryonic 0:40:00 half-closed 0:20:00 idle 2:00:00
            DCD: enabled, retry-interval 0:00:15, max-retries 5
            DCD: client-probe 530, server-probe 0, conn-expiration 106
    ...

I could understand if we were reaching a session limit, however, with only
two clients connected and a max of 5000 I don't believe this to be the case.
Also, as mentioned, the current session index being 'stuck' at -35 concerns
me slightly.

In the end, we had failed over to the redundant node which did not exhibit
this issue. However, as soon as we failed back the problem came straight
back. The only way to resolve the issue was a reload.

I'm trying to work out whether anyone has encountered this issue before on
an ASA55x0 running 8.2(4). Mainly to determine whether this was something
strange, or me just being daft. As much as I'd like to log a TAC case for
this one, this particular device does not have a valid support contract.
However, for my sanity I'd like to establish whether this is / was a
potential code issue, or a problem with the device itself.

Regards,
Peter Adkins