[nsp] ICMP: time exceeded (reassembly)

[Victor Sudakov]

Dmitry Volkov wrote:
> Usually router doesn't do fragmentation of TCP because vast majority of any
> type of hosts send packets with DF bit =1
> and it's end-hosts duties to do frag/defrag
> What traffic did You send through tunnel ? ICMP, UDP ? 

Does it make any difference? As already stated, the tunnel carrier
packets are being fragmented, not the tunnel payload.

> In this case router
> will do frag/defrag and as Tim said there is high probability that fragments
> were stopped in between on the way from sender to receiver.
> I don't understand how changing of encapsulation from GRE to IPIP would stop
> frag/defrag.

Neither do I. 

> If changing encaps stops generating ICMP 11 1 - I would assume that
> reassembly happens successfully - and probably software bug on router ?

Perhaps in the case of IPIP there is no fragmentation and therefore no
reassembly at all. This may be a bug in the FreeBSD 5.x gre
implementation, but this should be further investigated.

> However IP MTU for IPIP is 1480 and for GRE 1476
> 
> Dmitry
> 
> 
> > -----Original Message-----
> > From: cisco-nsp-bounces at puck.nether.net
> > [mailto:cisco-nsp-bounces at puck.nether.net]On Behalf Of Victor Sudakov
> > Sent: Tuesday, February 03, 2004 1:29 AM
> > To: Bulger, Tim
> > Cc: cisco-nsp at puck.nether.net
> > Subject: Re: [nsp] ICMP: time exceeded (reassembly)
> >
> >
> > Bulger, Tim wrote:
> > > The router sends ICMP messages because for the tunnel
> > packets, it acts
> > > as a host.
> >
> > I see. This means that the outer tunnel packets are being fragmented,
> > not the payload packets. Thanks for the hint.
> >
> > > It only acts as a router for the tunnel payload.  If the MTU
> > > of your tunnel interfaces is 1476, there should be no
> > fragmentation of
> >
> > Tunnel5 is up, line protocol is up
> >   Hardware is Tunnel
> >   Description: test
> >   Internet address is 212.73.125.5/30
> >   MTU 1514 bytes, BW 9 Kbit, DLY 500000 usec,
> >      reliability 255/255, txload 1/255, rxload 1/255
> >   Encapsulation TUNNEL, loopback not set
> >
> > The default seems to be MTU 1514 bytes, while "ip mtu" is certainly
> > 1476 (also default).
> >
> > > the tunnel packets themselves unless an interface on the
> > path between
> > > endpoints has a lower MTU than your endpoints.  It is quite possible
> > > that a firewall or ACL (more likely) is blocking those fragments,
> > > resulting in reassembly timeouts.
> >
> > It is important to understand why those packets get fragmented at all.
> > When we changed tunnel mode from GRE to IPIP, the tunnel started
> > working fine and there was no fragmentation.
> >
> > >
> > > Good luck,
> > > Tim
> > >
> > > -----Original Message-----
> > > From: Victor Sudakov [mailto:sudakov at sibptus.tomsk.ru]
> > > Sent: Monday, February 02, 2004 8:50 PM
> > > To: cisco-nsp at puck.nether.net
> > > Subject: [nsp] ICMP: time exceeded (reassembly)
> > >
> > > Colleagues,
> > >
> > > A GRE tunnel is configured between a Cisco router and a
> > FreeBSD host.
> > > The config on the router is:
> > >
> > > !
> > > interface Tunnel5
> > >  description test
> > >  ip address 212.73.125.5 255.255.255.252
> > >  ip verify unicast reverse-path
> > >  tunnel source 212.73.125.217
> > >  tunnel destination 212.192.122.147
> > > !
> > >
> > > The problem is that large datagrams cannot pass through the tunnel
> > > and the router sends the following ICMP messages to the other tunnel
> > > endpoint:
> > >
> > > 24782: ICMP: time exceeded (reassembly) sent to
> > 212.192.122.147 (dest
> > > was 212.73.125.217)
> > > 24783: ICMP: time exceeded (reassembly) sent to
> > 212.192.122.147 (dest
> > > was 212.73.125.217)
> > > 24826: ICMP: time exceeded (reassembly) sent to
> > 212.192.122.147 (dest
> > > was 212.73.125.217)
> > >
> > > I suppose the datagrams get fragmented because packets are
> > larger than
> > > the tunnel MTU which is the default 1476 on both sides. My
> > question is
> > > why is the router unable to reassemble the fragments?
> > >
> > > RFC792 reads:
> > >
> > > =========================
> > >
> > >    ICMP Fields:
> > >
> > >    Type
> > >
> > >       11
> > >
> > >    Code
> > >
> > >       0 = time to live exceeded in transit;
> > >
> > >       1 = fragment reassembly time exceeded.
> > >
> > > [dd]
> > >
> > >    Description
> > >
> > >       If the gateway processing a datagram finds the time
> > to live field
> > >       is zero it must discard the datagram.  The gateway
> > may also notify
> > >       the source host via the time exceeded message.
> > >
> > >       If a host reassembling a fragmented datagram cannot
> > complete the
> > >       reassembly due to missing fragments within its time limit it
> > >       discards the datagram, and it may send a time
> > exceeded message.
> > >
> > >       If fragment zero is not available then no time
> > exceeded need be
> > >       sent at all.
> > >
> > >       Code 0 may be received from a gateway.  Code 1 may be received
> > >       from a host.
> > > =========================
> > >
> > > Looks like a Cisco router is not supposed to send Code 1 messages at
> > > all, because it is a router and not a host.
> > >
> > > Any help is appreciated.
> > >
> > > --
> > > Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
> > > _______________________________________________
> > > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > > archive at http://puck.nether.net/pipermail/cisco-nsp/
> > >
> >
> > --
> > Victor Sudakov,  VAS4-RIPE, VAS47-RIPN
> > _______________________________________________
> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> > archive at http://puck.nether.net/pipermail/cisco-nsp/

-- 
Victor Sudakov,  VAS4-RIPE, VAS47-RIPN