[Outages-discussion] ICMP Filtering/Prioritization [was Re: [outages] Level3 Chicago]

[Jeremy Chadwick]

On Thu, Aug 20, 2009 at 12:03:57AM -0400, William R. Lorenz wrote:
> Hi Craig & Devon,
> 
> [Discussion thread moved to outages-discussion list.]
> 
> I have a few related questions about the ICMP prioritization you mention.
> 
> On Wed, 19 Aug 2009, Craig Pierantozzi wrote:
> 
> >Yes, and when pure prioritization is not available, ISPs will use
> >rate limiting too which is one of the ways Level 3 restricts ICMP
> >to the cpu on the core devices.
> 
> Is it fair to assume any such rate limiting is strictly inbound to a
> local router?  For example, if there was a network path similar to
> this example:
> 
>   point_a <-> level3_router_one <-> level3_router_two <-> point_b
> 
> Would it be fair to assume that ICMP packets would NOT be rate
> limited across the Level3 network from point_a to point_b?  Those
> IP/ICMP packets would be treated as transit traffic and not limited
> in any way, correct? Working within the Level3 example, those
> transit-bound IP packets should be treated the same by
> CAR/BAR/EBR/HSA routers, etc.  Is that accurate?

As I understand it, that is correct.  Routing of ICMP packets through a
router (e.g. the dst IP in the IP/ICMP packet is to an address which the
router is not responsible for) do not result in the packets getting
de-prioritised.

The packets I was referring to which get de-prioritised are 1) ICMP
packets directed to level3_router_*, and 2) ICMP responses from
level3_router_* sent back to the src (e.g. ICMP time exceeded).

> I haven't looked, but maybe there's a NANOG thread RE ICMP
> prioritization. Perhaps there's also an engineer from Level3's IP
> group that could chime in with additional details. :-)  Thanks, in
> advance, for your insights.

I'm not really sure backbone providers are willing to disclose this
stuff.  Engineers escalated to via NOCs will definitely use the "what
you're seeing is de-prioritisation of ICMP on our routers" excuse for
outages (in some cases even when you include packet captures indicating
TCP packets to a non-router destination are getting delayed or dropped).

In this day and age, I'm really not sure why ICMP is "throttled" (I'm
using the term loosely) in this way.  As I understand it, the original
concern which brought it forth was packet kids ping -f'ing routers.
Packet kids don't bother with that any more -- now it's TCP SYN, UDP
floods, or (the most popular) use of DDoS networks to overwhelm the
entire pipe with all sorts of garbage.

I'd be interested in knowing exactly what ICMP types and codes are
prioritised last or considered "best-effort", but providers would likely
argue that disclosing that information could result in them being open
to attacks.  Round and round we go...

-- 
| Jeremy Chadwick                                   jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.              PGP: 4BD6C0CB |
> 
> 
> --
> William R. Lorenz
> 
> 
> 
> On Wed, 19 Aug 2009, Craig Pierantozzi wrote:
> 
> >Also, the car device is an edge router so there could be
> >congestion on a customer port too when higher response times are
> >seen on the other side of a hop. Response times that settle could
> >be the control plane/data plane issue or could be once traffic
> >gets to a far end there's an asymetric path that goes a different
> >return path rather than back across the link seen on the forward
> >traceroute. All these are why simple pings and traceroutes don't
> >always tell the story.
> 
> >* Devon True was thought to have said:
> >
> >>'Jeremy Chadwick' wrote:
> >>
> >>>That's an excellent question -- and one I've always wondered myself.
> >>>
> >>>This is purely speculative, but I believe outbound ICMP (e.g.
> >>>sent from the router to whatever src solicited it) is what's
> >>>de-prioritised.
> >>>
> >>>Someone more familiar with Cisco and Juniper equipment might
> >>>know for certain.
> >>
> >>Usually packets destined to the control-plane of the system are
> >>prioritized based on criteria. It is better to let routing
> >>control protocols (e.g. ospf, bgp, isis) through first than
> >>someone pinging or running a traceroute. Packets *through* the
> >>router take the normal forwarding path and are not affected by
> >>this system.
> >>
> >>There may be system defaults based on hardware/software, but
> >>Cisco has CoPP and I believe Juniper uses a firewall on the lo0
> >>interface (been a while since I touched one) for user-defined
> >>rules.
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion at outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion