[Outages-discussion] [outages] Level 3 core router in Chicago down?

[Jeremy Chadwick]

Additionally, if this was scheduled maintenance on L3's part, why didn't
they adjust/decrease BGP route preference, say, 30 minutes in advance of
their work (to give announcements a chance to propagate), so that
packets through routes which had alternate paths available would
hopefully not take that L3 path any longer? 

We do this all the time at my workplace (JunOS: policy-statement that
contains an adjusted (lower) local-preference than usual, then use set
import/set export for that policy on the appropriate BGP neighbour).
Really not rocket science here.  It's not perfect, but it's effective
about 90% of the time.

It sure looks like someone just did their work without a care in the
world for the packets going through it.  L3 isn't the only offender when
it comes to this mentality; I'm not singling them out.  :-)

-- 
| Jeremy Chadwick                                   jdc at parodius.com |
| Parodius Networking                       http://www.parodius.com/ |
| UNIX Systems Administrator                  Mountain View, CA, USA |
| Making life hard for others since 1977.               PGP 4BD6C0CB |

On Thu, May 26, 2011 at 12:14:47AM -0500, Frank Bulk wrote:
> Seems a little early in the evening for maintenance.
> 
> Frank
> 
> -----Original Message-----
> From: Gaurav Taparia [mailto:gaurav.taparia at gmail.com] 
> Sent: Thursday, May 26, 2011 12:05 AM
> To: NANOG
> Subject: Re: [outages] Level 3 core router in Chicago down?
> 
> We saw it too. The paths are back up but the latency is higher suggesting
> that it is on a re-routed path. Their helpdesk is saying it is a planned
> maintenance.
> 
>  - Gaurav
> 
> 
> 
> On Thu, May 26, 2011 at 10:28 AM, Jay Ashworth <jra at baylink.com> wrote:
> 
> > ----- Original Message -----
> > > On 5/25/2011 11:25 PM, Brooks Bridges wrote:
> > > > We just lost a bunch of routes out of Chicago over Level 3.
> > > >
> > > > Seems to be dying at edge routers right before their core.
> > > >
> > > >   1. XXXXXXXXXXXXXXX 0.0% 10 0.5 0.4
> > > > 0.3 0.6 0.1
> >
> > > Of course, as soon as I hit send, it comes back...
> >
> > Well, thanks for fixin' that, Brooks.  Night, now.
> >
> > Cheers,
> > -- jr 'causation?  what's that?' a
> > --
> > Jay R. Ashworth                  Baylink
> > jra at baylink.com
> > Designer                     The Things I Think                       RFC
> > 2100
> > Ashworth & Associates     http://baylink.pitas.com         2000 Land Rover
> > DII
> > St Petersburg FL USA      http://photo.imageinc.us             +1 727 647
> > 1274
> >
> >
> 
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion at outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion