[outages] NY/MA area Level3 (and Comcast?) issues 1-1:15AM EDT

[Darrell Fixler]

Mixed results reaching destinations - some dying in Level3, some getting through. After about 15 min, all looked normal, but reviewing my router in Boston, MA area showed our Level3 BGP session had also bounced ~12 min prior... Couldn't hit www.google.com or microcenter.com from within Comcast, neither of which are via Level3 from Comcast Boston, so something larger was amiss.

(traces from within Comcast to Level3 destination)

tracert 199.233.98.16

Tracing route to 199.233.98.16 over a maximum of 30 hops

  1     3 ms     4 ms    14 ms  192.168.55.1
  2    16 ms    10 ms    11 ms  96.128.96.1
  3    11 ms    14 ms    14 ms  68.85.185.37
  4    14 ms    14 ms    14 ms  69.139.221.1 
  5     *        *        *     Request timed out.
  6     *        *        *     Request timed out.
  7     *        *        *     Request timed out.
  8     *        *        *     Request timed out.
  9     *        *        *     Request timed out.
 10     *        *        *     Request timed out.
 11     *        *        *     Request timed out.
 12     *        *        *     Request timed out.

but:

ping 4.2.2.2

Pinging 4.2.2.2 with 32 bytes of data:
Reply from 4.2.2.2: bytes=32 time=18ms TTL=55
Reply from 4.2.2.2: bytes=32 time=24ms TTL=55


Then better, but still not quite there:

C:\WINDOWS\system32>tracert -d 199.233.98.240

Tracing route to 199.233.98.240 over a maximum of 30 hops

  1     1 ms    <1 ms     1 ms  192.168.55.1
  2    13 ms    10 ms    10 ms  96.128.96.1
  3    13 ms    13 ms    26 ms  68.85.185.33
  4    15 ms    18 ms    11 ms  69.139.221.1
  5    21 ms    20 ms    18 ms  68.86.95.29
  6    40 ms    37 ms    38 ms  4.68.127.1
  7    39 ms    38 ms    38 ms  4.69.138.254
  8    37 ms    37 ms    38 ms  4.69.132.65
  9    39 ms    52 ms    38 ms  4.69.140.93
 10    39 ms    40 ms    38 ms  4.69.132.241
 11     *        *        *     Request timed out.
 12     *        *        *     Request timed out.
 13     *        *     ^C

15 min after I noticed, everything jelled:

tracert 199.233.98.16

Tracing route to 199.233.98.16 over a maximum of 30 hops

  1     1 ms     1 ms     1 ms  192.168.55.1
  2    10 ms    10 ms    11 ms  96.128.96.1
  3    16 ms    13 ms    17 ms  te-0-6-0-5-sur01.needham.ma.boston.comcast.net [
68.85.185.37]
  4    14 ms    12 ms    17 ms  be-39-ar01.needham.ma.boston.comcast.net [69.139
.221.1]
  5    17 ms    20 ms    21 ms  he-2-6-0-0-cr01.newyork.ny.ibone.comcast.net [68
.86.93.33]
  6    46 ms    39 ms    37 ms  ae12.edge1.NewYork2.level3.net [4.68.127.1]
  7    37 ms    39 ms    42 ms  vlan52.ebr2.NewYork2.Level3.net [4.69.138.254]
  8    39 ms    37 ms    40 ms  ae-2-2.ebr1.Chicago1.Level3.net [4.69.132.65]
  9    38 ms    40 ms    38 ms  ae-1-5.bar1.Boston1.Level3.net [4.69.140.93]
 10    41 ms    39 ms    50 ms  ae-7-7.car1.Boston1.Level3.net [4.69.132.241]
 11    44 ms    38 ms    38 ms  MULTIPLIXIT.car1.Boston1.Level3.net [4.30.121.10
2]
 12    41 ms    43 ms    39 ms  ns1.vinny.org [199.233.98.16]

No additional BGP flaps, all looks normal 1:15AM EDT <shrug>.

-- 

Darrell