[j-nsp] network engineering

Fri Feb 27 18:21:26 EST 2009

Hi!

Sorry for bringing this up again, but something bothers me.

On several targets the traceroute or mtr is not going through clean,  
whereas on my home dsl line it is. I thought about, that every target  
where we have asymmetric routing is behaving like this, but if you  
say, asymmetric routing is something completely normal, than the  
reason, why the mtr is not going through clean, has to be something  
different?

For example a trace to www.stern.de (german newspaper) from our  
datacenter

   1. 91.190.227.17                 0.0%     5    0.3   0.6   0.2    
1.9   0.7
   2. ge-00.cr1.ems.dlrz.net        0.0%     5   19.2  23.3  19.2   
27.7   3.9
   3. ge-00-508.tc2.ams.dlrz.net    0.0%     5    4.6   4.8   4.5    
5.5   0.4
   4. dus002isp005.versatel.de      0.0%     5    6.5   6.5   6.2    
7.1   0.4
   5. 10g-9-3.dor002isp006.versate  0.0%     5    7.0   6.8   6.6    
7.0   0.1
   6. 10ge-9-4.dor002isp005.versat  0.0%     5    7.8  10.7   6.4   
26.0   8.6
   7. 10g-8-4.hhb002isp005.versate  0.0%     5   16.4  13.3  12.3   
16.4   1.7
   8. hhb2guj.versatel.de           0.0%     5   12.3  12.7  12.3   
13.0   0.3
   9. ???                          100.0     5    0.0   0.0   0.0    
0.0   0.0
  10. lb-guj-www-04-6.net-hh.guj.d  0.0%     5   12.2  13.0  12.2   
13.9   0.6
  11. www.stern.de                  0.0%     5   12.6  13.0  12.6   
13.5   0.4

and from my home dsl line

   1. port-87-193-164-97.static.qs  0.0%     5    0.3   0.3   0.3    
0.4   0.1
   2. bsn2.mun.qsc.de               0.0%     5   24.5  25.1  24.5   
26.6   0.8
   3. core1.mun.qsc.de              0.0%     5   25.4  26.0  25.4   
26.9   0.6
   4. core4.dus.qsc.de              0.0%     5   29.6  29.5  28.6   
30.3   0.6
   5. core1.dus.qsc.de              0.0%     5   38.5  35.8  28.8   
53.6  10.8
   6. core2.dus.qsc.de              0.0%     5   31.0  29.2  28.1   
31.0   1.2
   7. peergw2-hansenet.dus.qsc.de   0.0%     5   29.0  29.1  28.6   
30.4   0.7
   8. ae0-101.cr02.dus.de.hansenet  0.0%     5   28.1  33.7  28.1   
52.4  10.5
   9. so-1-0-0-0.cr02.weham.de.han  0.0%     5   35.3  35.3  34.5   
36.4   0.8
  10. gi0-0-0.er03.asham.de.hansen  0.0%     5   34.6  34.8  33.6   
37.7   1.7
  11. 62.109.127.129                0.0%     5   33.9  35.7  33.9   
37.9   1.7
  12. 10.200.100.18                 0.0%     5   34.5  35.3  34.5   
36.6   0.8
  13. lb-guj-www-04-4.net-hh.guj.d  0.0%     5   41.5  36.8  35.2   
41.5   2.7
  14. www.stern.de                  0.0%     5   35.4  35.0  34.5   
35.4   0.4

Out of the datacenter, there is a hole at position 9. And that is  
something, we habe rather often:

To amazon.de from datacenter:

   1. 91.190.227.17                 0.0%     5    0.3   0.7   0.3    
1.9   0.7
   2. ge-00.cr1.ems.dlrz.net        0.0%     5   26.9  24.7  21.7   
28.8   3.1
   3. gi9-47.228.ccr01.dus01.atlas  0.0%     5    3.9   3.8   3.6    
4.0   0.2
   4. te7-2.mpd02.fra03.atlas.coge  0.0%     5    7.5   8.4   7.3   
11.9   2.0
   5. vl3491.mpd01.fra03.atlas.cog  0.0%     5   18.9   9.5   6.9   
18.9   5.2
   6. ???                          100.0     5    0.0   0.0   0.0    
0.0   0.0

from dsl:

   1. port-87-193-164-97.static.qs  0.0%     5    0.2   0.4   0.2    
0.7   0.2
   2. bsn2.mun.qsc.de               0.0%     5   25.6  31.9  24.9   
55.0  13.0
   3. core1.mun.qsc.de              0.0%     5   25.2  24.8  24.3   
25.4   0.5
   4. core4.dus.qsc.de              0.0%     5   29.7  31.9  29.5   
41.0   5.1
   5. core1.dus.qsc.de              0.0%     5   28.4  35.1  28.1   
61.5  14.8
   6. 212.162.17.169                0.0%     5   30.3  28.9  27.9   
30.3   0.9
   7. ae-32-56.ebr2.Dusseldorf1.Le  0.0%     5   43.9  38.0  33.1   
43.9   4.4
   8. ae-42.ebr1.Amsterdam1.Level3  0.0%     5   44.8  39.4  32.7   
44.8   5.1
   9. ae-45-105.ebr2.Amsterdam1.Le  0.0%     5   36.7  36.4  32.2   
45.4   5.4
  10. ae-2.ebr2.London1.Level3.net  0.0%     5   40.2  43.7  39.7   
50.0   4.9
  11. ae-1-100.ebr1.London1.Level3  0.0%     5   51.6  49.6  43.1   
58.2   6.1
  12. ae-5-5.car1.Dublin1.Level3.n  0.0%     5   50.9  53.0  50.8   
61.3   4.6
  13. ???                          100.0     5    0.0   0.0   0.0    
0.0   0.0

What is happening here? (The fact, that amazon is blocking packets is  
not the issue. It is, that I am lacking a network, that the packets  
are going through).

At the moment we have 4 transits and a router facing the AMS-IX. At  
the moment, we have configured our routers rather open. So packets may  
received on one transit and are send out on another.

Regards,

Matthias

Am 06.02.2009 um 11:29 schrieb Mark Tinka:

> On Friday 06 February 2009 05:09:30 pm Matthias Gelbhardt
> wrote:
>
>> We have asymmetric routing in several cases. I would like
>> to know, how you would deal against that?
>
> The moment you're multi-homed to the Internet, asymmetric
> routing is a fact of life; and it's not really a bad thing.
>
> How traffic leaves or enters a network is a function of best
> path dynamics vs. cost and bandwidth available. And this
> goes for both your network and the others on the Internet
> (including your upstreams/peers).
>
> Maximizing capacity and cost efficiency is an on-going
> exercise for any network engineer running a BGP network. The
> individual dynamics usually vary from network to network.
>
> Monitoring flows is a good way to find out how best to
> engineer your traffic.
>
> Cheers,
>
> Mark.