[c-nsp] Temporarily disable all forwarding on ASR9K

[Mattias Gyllenvarg]

This whole protecting/verifying redundant customer functionality is a
difficult problem.

I have found it difficult to implement mechanisms that detect all possible
faults on the main connection without making things very complicated.
Mostly for remote areas where there is only one local POP that in itself is
not redundant.

The worst gotcha I have found is that you test the redundancy (real event
or simulated) and you get management and smnp etc etc. But the customer is
still down because of some error in forwarding. Like default pointing
toward primary link that is broken behind next-hop.

So, I just want to say that you should verify traffic patterns on the
customer port(s) when testing this.

Regrading disabling forwarding, why not shutdown links from the adjacent
devices. Recovery time would be linkstate+igp+bgp sync. Not too slow on
ASR9k right?

//Mattias Gyllenvarg




On Thu, Aug 15, 2013 at 4:55 AM, John Neiberger <jneiberger at gmail.com>wrote:

> In my case, the best result would be to simulate taking the entire device
> offline, but I don't know if that's possible to reverse quickly. Taking the
> interfaces down would be great. Really, the simplest thing to do is just
> spend a minute to create a SMOP that shuts down the interfaces and be done
> with it. I'm just curious if there are other interesting ways to accomplish
> something similar. I wonder if there are some processes that can be killed
> to take the router offline and then restarted to bring it back online. Even
> if that were possible, I'm sure it would be a fairly bad idea. lol
>
> Another thought was to shutdown the linecards but leave them powered up. I
> haven't tried it but I wondered if maybe bringing them back up from a
> shutdown state would be faster than doing it from a fully powered down
> state. If so, that might be another option. Not even close to as fast as
> just shutting them down and rolling back, if necessary. That's going to be
> tough to beat.
>
>
> On Wed, Aug 14, 2013 at 7:43 PM, Pete Lumbis <alumbis at gmail.com> wrote:
>
> > This raises a good point.
> >
> > Is the goal to simulate a black-hole that could be seen with an incorrect
> > adjacency, where control plane is healthy but data plane is broken, or is
> > the goal to simulate taking this device offline?
> >
> > Do we care about carrier on the interfaces?
> >
> >
> > On Wed, Aug 14, 2013 at 6:19 PM, arulgobinath emmanuel <
> arulgobi at gmail.com
> > > wrote:
> >
> >> null0 doesn't cause the NHRP to trigger IMHO  this will be a disaster  .
> >> shut / no shut is the easiest but it doesn't simulate the whole part.
> >> real test comes when the modules crash when reloading specially after
> >> couple of years... :)
> >> what if we copy a empty config ??? and rollback the config ? i didn't
> >> test this anyway .
> >>
> >>
> >> On Wed, Aug 14, 2013 at 10:13 PM, Pete Lumbis <alumbis at gmail.com>
> wrote:
> >>
> >>> Copy/paste a bunch of null0 routes?
> >>>
> >>> deny any acls on interfaces?
> >>>
> >>>
> >>> On Wed, Aug 14, 2013 at 10:54 AM, John Neiberger <jneiberger at gmail.com
> >>> >wrote:
> >>>
> >>> > We need to upgrade some ASR9Ks that have a lot of connected devices
> >>> with
> >>> > complex interrelationships and we have to do a lot of work to make
> >>> sure all
> >>> > the correct redundancy is in place prior to the upgrade. Since the
> >>> router
> >>> > takes so long to reload, I'd like to find a way to essentially
> >>> simulate the
> >>> > loss of forwarding for a minute or so to verify that our redundancy
> >>> > preparations were thorough, but I need to be able to back out of it
> >>> > quickly. I thought about shutting down the linecards but that's
> still a
> >>> > fairly long restart. I'm hoping to find some method much faster than
> >>> that.
> >>> >
> >>> > The simplest and most straightforward way is to shut down all the
> >>> > interfaces manually and then rollback if necessary. We can take it
> out
> >>> of
> >>> > routing by setting the overload bit in ISIS, but that still leaves
> >>> routing
> >>> > and forwarding in place for locally connected interfaces, which is
> >>> what we
> >>> > want to stop. We were tossing around some ideas and wondered,
> probably
> >>> just
> >>> > academically, if there were a way to completely stop forwarding
> >>> > temporarily.
> >>> >
> >>> > Is there a way to disable forwarding through an ASR9K that is easily
> >>> and
> >>> > quickly reversible? We'll probably do the interface shutdown method
> >>> since
> >>> > it's so simple, but now I'm curious what other options might be
> >>> available.
> >>> > _______________________________________________
> >>> > cisco-nsp mailing list  cisco-nsp at puck.nether.net
> >>> > https://puck.nether.net/mailman/listinfo/cisco-nsp
> >>> > archive at http://puck.nether.net/pipermail/cisco-nsp/
> >>> >
> >>> _______________________________________________
> >>> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> >>> https://puck.nether.net/mailman/listinfo/cisco-nsp
> >>> archive at http://puck.nether.net/pipermail/cisco-nsp/
> >>>
> >>
> >>
> >
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>



-- 
*Med Vänliga Hälsningar*
*Mattias Gyllenvarg*