[outages] Internap/XO/Savvis problems?

Wed Jul 2 22:15:29 EDT 2008

We received the following notice regarding scheduled maintenance from XO.
It is apparently due to some hardware upgrades.  While this notice only
refers to AZ, they may have scheduled similar outages in other areas today
as well.

Dear Customer,

This email notice is to inform you of scheduled maintenance occurring in the
Phoenix, AZ area affecting your XO Data/Access service.

On 7/2/2008, between the hours of 11:00 PM MST to 6:00 AM MST, you will
experience a 120 minute interruption of service. XO Communications Customer
Care and Network Teams will be monitoring this activity to ensure necessary
downtime is kept to a minimum while the maintenance is completed.

At your convenience, feel free to reply to this message or call us at the
toll free number listed on your XO bill, as our Customer Care teams will be
available to handle any questions, comments, or concerns.

Please reference Maintenance Request 53135 when contacting Customer Care.

Thank You,
XO Communications

On 7/2/08 5:56 PM, "Matt Palmer" <mpalmer at hezmatt.org> wrote:

> Greetings Outageers,
> 
> One of our customers is seeing some problems between themselves and our DC,
> and both-way traceroutes are showing problems just corewards of our
> upstream, Internap:
> 
> Customer to us:
> 
> traceroute to 74.201.255.113 (74.201.255.113), 64 hops max, 40 byte packets
>  1  192.168.1.1 (192.168.1.1)  3.188 ms  7.275 ms  5.663 ms
>  2  ATM1-0-60.win17.Melbourne.telstra.net (165.228.176.49)  12.096 ms  9.410
> ms  13.803 ms
>  3  TenGigE0-8-0-5.win-core1.Melbourne.telstra.net (203.50.80.129)  8.397 ms
> 8.633 ms  11.843 ms
>  4  Bundle-Pos1.ken-core4.Sydney.telstra.net (203.50.6.21)  22.303 ms  26.382
> ms  29.742 ms
>  5  Port-Channel1.pad-gw2.Sydney.telstra.net (203.50.6.29)  21.379 ms  26.954
> ms  31.464 ms
>  6  10GigabitEthernet1-0.sydp-core02.Sydney.reach.com (203.50.13.46)  115.516
> ms  24.861 ms  204.107 ms
>  7  i-0-0.wil-core02.net.reach.com (202.84.144.101)  167.032 ms  170.378 ms
> 175.605 ms
>  8  p9_0-ashc1.net.reach.com (202.84.140.90)  182.026 ms  184.921 ms  181.750
> ms
>  9  g4_14-pax06.net.reach.com (202.84.251.106)  181.076 ms  183.144 ms
> 181.113 ms
> 10  206.111.12.173.ptr.us.xo.net (206.111.12.173)  192.088 ms  324.819 ms
> 194.243 ms
> 11  * * te-11-0-0.rar3.sanjose-ca.us.xo.net (207.88.12.69)  196.075 ms
> 12  207.88.14.133.ptr.us.xo.net (207.88.14.133)  261.502 ms  259.664 ms
> 266.501 ms
> 13  207.88.14.134.ptr.us.xo.net (207.88.14.134)  264.905 ms  257.116 ms
> 258.146 ms
> (oh so many stars)
> 
> Us back to the customer:
> 
> traceroute to 165.228.176.49 (165.228.176.49), 30 hops max, 40 byte packets
>  1  33.254.201.74.static.ey03.engineyard.com (74.201.254.33)  0.907 ms  0.908
> ms  0.928 ms
>  2  76.254.201.74.static.ey03.engineyard.com (74.201.254.76)  0.210 ms  0.241
> ms  0.273 ms
>  3  core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24)  1.559 ms
> core1.te6-2-bbnet2.nym007.pnap.net (216.52.95.88)  1.338 ms
> core1.te6-1-bbnet1.nym007.pnap.net (216.52.95.24)  1.369 ms
>  4  208.173.129.61 (208.173.129.61)  1.441 ms  1.525 ms  1.658 ms
>  5  * * *
>  6  cr2-loopback.sfo.savvis.net (206.24.210.71)  98.457 ms  94.617 ms  101.634
> ms
>  7  pr1-so-0-0-0.PaloAltoPaix.savvis.net (204.70.200.193)  89.389 ms  89.397
> ms  89.478 ms
> (constellations from here on)
> 
> I'm fine from my connection on the same side of the pond, since it looks
> like my ISP has a direct peering with Internap:
> 
> traceroute to 74.201.255.113 (74.201.255.113), 30 hops max, 40 byte packets
>  1  172.14.17.1 (172.14.17.1)  34.982 ms  35.987 ms  48.326 ms
>  2  lns10.syd6.internode.on.net (150.101.197.88)  50.219 ms  51.812 ms  53.293
> ms
>  3  vl14.cor2.syd6.internode.on.net (150.101.197.83)  91.320 ms  92.807 ms
> 94.274 ms
>  4  po3.bdr1.syd6.internode.on.net (150.101.199.232)  226.619 ms  231.256 ms
> 234.897 ms
>  5  pos2-1.bdr1.lax1.internode.on.net (203.16.213.158)  228.912 ms  233.132 ms
> 236.411 ms
>  6  cr1-eqix-peer.lax009.internap.net (206.223.123.29)  245.254 ms  208.194 ms
> *
>  7  cr1.wdc005.inappnet-65.cr1.lax009.internap.net (66.79.146.141)  279.374 ms
> 283.716 ms  289.151 ms
>  8  cr1.nym009.inappnet-84.cr1.wdc005.internap.net (66.79.146.209)  309.854 ms
> 316.382 ms  318.303 ms
>  9  core3.nym.inappnet-82.cr1.nym007.internap.net (66.79.152.130)  298.775 ms
> 300.648 ms  307.357 ms
> 10  border2.pc2-bbnet2.nyj001.pnap.net (216.52.95.74)  323.780 ms  325.643 ms
> 331.222 ms
> 11  74.201.254.73 (74.201.254.73)  348.343 ms  350.727 ms *
> 12  113.255.201.74.static.ey03.engineyard.com (74.201.255.113)  305.041 ms
> 303.767 ms  305.289 ms
> 
> Wondering if anyone else is seeing similar problems, or if anyone knows of
> any widespread problems with Internap/XO/Savvis that I can pass on to my
> customer to let them know what's going on.
> 
> Thanks,
> - Matt
> _______________________________________________
> outages mailing list
> outages at isotf.org
> https://puck.nether.net/mailman/listinfo/outages