[c-nsp] IP packet out of order.

[Andre Beck]

Hi,

On Fri, Jul 23, 2004 at 07:13:20PM +0200, Olivier Gosselet wrote:
> I have the following setup :
> 
> Router A - Ethernet LAN - Bridge Eth/ATM - ATM Network - Router B - Ethernet
> LAN

I assume the Bridge is a DSL CPE.
 
> IP packet 1 - 1400 Bytes
> IP packet 2 - 38 bytes

Any chance that these packets have differing TOS/DSCP? Not that they would
influence a bridge, though...
 
> Packets are sent out by router A to router B in the following order : 1 - 2
> Packets are sent out by router B onto the LAN in the following order : 2 - 1

So there are two locations where reordering might happen:

a) The bridge which takes Ethernet frames and serializes them into AAL5
   SNAP cell sequences
b) The router which reassembles these sequences into SNAP frames and
   then terminates them to l3 in some kind of hidden BVI (known as
   route-bridged encapsulation)

I'm getting some feeling that what happens, happens during reassembly.

> According to my understanding of "IP over ATM" IP Packet are chopped into
> ATM cells with IP Packet 1 contained into several ATM cells while IP Packet
> 2 fits into 1 cell. 

In your setup, it's not actually IP, it's Ethernet and SNAP. While you
could fit that 38byte packet into a single AAL5 cell using "aal5mux ip"
easily, I actually doubt you can fit the SNAP variant of that. But I'm
not having the data handy to verify, and actually it makes no difference
whether it goes in one cell or two.

> I know there is only one physical path in the ATM network from the bridge to
> the router. I have no control on the ATM Bridge and/or Network but I would
> assume they send and transport the resulting cells on a FIFO basis meaning
> all the cell of packet 1 followed by the cell of packet 2. 

That is true for the path unless it runs into policing and drops cells,
but then you wouldn't see both packets coming out reassembled in the end.
But there is still a chance that the bridge (the DSL CPE) intersplices
cells from packet 2 into the cell stream generated from packet 1 that
is still going out. Notice that such CPE is usually a bandwidth bottle-
neck, especially in the upstream direction of ADSL. As it is usually
also doing policing, it will send out the cell stream slow enough for
a second frame beeing able to enter the bridge while the first is still
beeing sent out in a cell stream. What happens all depends on how the
box is built, but honestly, I assume the second packet is queued up and
not sliced into cells until the first cell stream is out. But you cannot
be sure of that. With some parallelity in a hardware implementation of
serialization, it can indeed splice the cell(s) of packet 2 into the
still not completed ones of packet 1.

> Would there be any chances that the Router B dequeues the second packet
> first while the ATM cells were all received in the correct order.

I think this can happen. The reason for this beeing that there is not just
a dequeueing of a packet, but a reassembly of cells into a packet. If
this is done in parallel with hardware support, it could well happen that
packet 2 is reassembled earlier or at the same time that packet 1 is,
allowing it to come out earlier.

Do you actually have problems with out of order packets? I was under the
impression that todays implementations of IP (during fragment reassembly)
and TCP (segment reassembly) deal quite well with out of order packets,
it's just some firewalls that choke on it...

Andre.
-- 
                  The _S_anta _C_laus _O_peration
  or "how to turn a complete illusion into a neverending money source"

-> Andre Beck    +++ ABP-RIPE +++    IBH Prof. Dr. Horn GmbH, Dresden <-