[c-nsp] IOS(-XE) 'configure replace' robustness

[Saku Ytti]

Hey James,

Not what I wanted to hear :(.

> I am working on this right now, I opened a TAC case yesterday and
> still warming up the TAC engine at present. Also just a note before,
> this is on CSR1000v for me not physical boxes if that makes a
> difference.

I did my testing on CSR1000v as well, mad props to kll and vrnetlab
for making it easy peasy.

> soon. Ideally I want the configuration to be locked when doing a full
> replace so that no-one else makes a configuration change at the same
> time AND automatic rollback MUST also be active so that we have
> guaranteed configuration state on the device (either the merge or
> replace operation completed 100% without issue or we are 100% as
> before the operation started). Otherwise automation is dead at the
> door step.

Call me naive, but I assume that when it boots and sees
startup-config, it runs the startup-config through some parser
function, which then outputs the actual config it'll load. If so, then
they really should run the B/candidate config through same parser
function, then try to do the replace.
But I have no real information how Cisco does any of this.

> I've been testing with a CSR1000v 3.16.4 image and it seems fucked.
> The main issue is that it seems to be applying the config NOT in a
> linear order. To clarify;

Can you try to recreate on everest or denali easily? I just want to
know if my lab was fluke that it worked, or is it robust in newer
images. As your experience sounds awfully like mine when the feature
first came.

> If I use "configure replace flash:candidate_config.txt" the operation
> fails. If the configuration archive feature is not enabled, the
> operation fails and the rollback fails so the devices is left in a
> mangled state of useless-ness. Sometimes running the replace operation
> a second time will get the device fully configured.

And you can repro it reliably? Can you compress the failure into
simplest possible test case you share with A and B config, so I can
try to recreate it?

> The replace option is failing with the following error:
>
> "Interface GigabitEthernet3's vrf  does not match with cfg vrf mgmt"
>
> Which comes from this line in config:
>
> "logging source-interface GigabitEthernet3 vrf mgmt"

Have you tried with 'configure replace source list' to observe what it
tries to do?

> candidate_config.txt is ordered similar to the output of “show run” so
> I was hoping it would either read the entire file and then apply the
> contents in the required or (VRFs first for example) or just apply the
> config lines in the order they are listed in candidate_config.txt from
> top to bottom, both of these scenarios should work. It seems like XE
> is applying the lines at random and it’s failing so TAC case it is for
> me.

Many thanks for your input James, much appreciate it, even though it's
not what I wanted to hear. I hope someone else has better news.

-- 
  ++ytti