[c-nsp] Operational impact of switching from ingress to egress replication mode

[John Neiberger]

I see the "mls rate-limit multicast ipv4 connected" command, for
directly connected sources, but is there a command that would apply to
traffic going through box?

We have very little unicast traffic but a whole bunch of multicast
traffic, some of which is directly connected but much of it is simply
passing through.

Thanks again for all your help, I appreciate your time.

On Tue, Sep 21, 2010 at 3:12 PM, Phil Mayers <p.mayers at imperial.ac.uk> wrote:
> On 09/21/2010 09:58 PM, John Neiberger wrote:
>>
>> Sorry.  I also meant to say it's a Sup 720-3BXL. Based on what I can
>> see on CCO, that thing can forward 400 Mpps of ipv4 traffic. Does that
>> mean that I can set a rate limit of, say, 300 Mpps and somewhat guard
>> the CPU from meltdown for a few moments?
>
> I wish! 300Mpps hitting the CPU of a sup720 will kill it stone dead. It has
> (two) 600MHz CPUs, and they will not survive such a load.
>
> There's lots of info in this in the archives, but in brief: the sup720/pfc3
> architecture forwards most packets in hardware. Some packets are however
> punted to CPU; these include:
>
>  1. Packets which need ARP resolution ("glean")
>
>  2. Multicast packets, which are trickled to the CPU so the CPU can see them
> and build (and refresh) hardware forwarding state
>
>  3. ACL and uRPF denies, which are trickled to the CPU so it can maintain
> counters
>
>  4. Various other traffic like TTL failures, needing ICMP
>
>  5. Obviously, packets address to the CPU (routing traffic, layer 2 PDUs)
>
> Because the CPUs on these boxes are very, very puny, you want to limit what
> hits the CPU. There are two methods available:
>
>  1. The "mls rate-limit" commands; these will place a simple numeric rate
> cap on certain types of traffic, and is done in hardware. There's no
> prioritisation, but for certain types of traffic (e.g. TTL failures) you can
> and should IMHO set low-ish limits. You SHOULD NOT use the "general" CEF
> limiter; because you should use...
>
>  2. CoPP - basically QoS on traffic punted to CPU. This is superior because
> you can write ACLs defining what is most important to you, with very
> granular control over policy. It suffers a couple of problems -
> broadcast/multicast and non-IP traffic are done in software, and it can't
> distinguish between glean and receive traffic, making a default-deny policy
> tricky.
>
>
> In short, common advice seems to be:
>
>  1. Set low limits on the mls limiters for TTL & MTU failure, and optionally
> ACL-drop ICMP unreach:
>
> mls rate-limit unicast ip icmp unreachable acl-drop 0
> mls rate-limit all ttl-failure 100 10
> mls rate-limit all mtu-failure 100 10
>
>  2. Use CoPP for everything else; DO NOT use the glean or cef receive
> limiter
>
> Search the archives and the Cisco docs for more info; it's not something I
> can summarise in 5 minutes I'm afraid.
>
> In your case, if you are going to perform a task which will potentially punt
> a lot of multicast traffic to the CPU, I was suggesting that there are MLS
> limiters which will reduce this; see my earlier email; though we run with
> the defaults, which are (quite) high PPS values!
>
> sh mls rate-limit  | inc MC
>         MCAST NON RPF   Off                  -       -     -
>        MCAST DFLT ADJ   On              100000     100  Not sharing
>      MCAST DIRECT CON   Off                  -       -     -
>      MCAST PARTIAL SC   On              100000     100  Not sharing
>