[j-nsp] Automation - The Skinny (Was: Re: ACX5448 & ACX710)

[Mark Tinka]

On 24/Jan/20 12:10, Saku Ytti wrote:

> In my opinion we do roughly the same thing, the same way in networks,
> with the same protocols since my start of career in 90s, very little
> has changed and you could drop competent neteng from 90s to today and
> they'd be immediately productive. Compare this to what has happened to
> compute the difference is striking.

Agreed - but is it really enough to the extent that the common buzz
sentence nowadays is "Network engineers are dead, they'll all be
replaced by software [developers]"?

I mean, I'd wager that more than half of the problems you find with
automation and tooling development is a total lack of protocol between
software developers and network engineers; in the same company. While
there has been plenty of success with a software developer reading a
networking-related RFC and writing code for that without needing to
understand, really, how IP/MPLS networks work, it's a whole other issue
trying to teach a network engineer how to write code, or a software
developer what IS-IS actually does.

I can't remember if I gave this example here before, but I know of a
network operator in Vienna who had to scramble and get their engineer
trained on CLI when they'd been setting up peering sessions fine for 3
years via a GUI, and when the GUI and automation front-end all went to
hell, that network engineer didn't know how to fall back to simple CLI
to setup even simpler BGP sessions for peering, by hand.

While clicking on GUI's is great, I don't have confidence that a network
of any decent scale can be ran, today, without some form of CLI
jockeying. And on the back of that, do we want to kill off the basics of
a network engineer in favour of Day 1 university graduates eager to
click a GUI button when provisioning your backbone, and they don't
actually understand what the "Wide Metric" checkbox actually means?



> People who think that netconf and yang are solving big problems and
> are key to solve automation probably haven't done much automation.

Totally agreed. But to also be fair, NETCONF/YANG are normally being
touted by vendors (much like Segment Routing, 5G and SD-WAN, but I
digress). I've not really found actual operators with anything
meaningful and useful to say about NETCONF/YANG.

Raise your hands if I'm talking nonesense.

For us, we find this whole NETCONF/YANG thing to be too heavy for simple
instructions you need to send to devices, not to mention the fact that
support within and between vendors is questionable (FlowSpec, anyone?).

I mean, that's why Ansible was so pleasing to our fingertips - all you
need is SSH and a large-enough, repetitive problem you want to go away
quickly.


> Roughly netconf is new snmp and yang is new mib, what ever they enable
> could have been enabled by existing protocols decades ago, the
> advantages are modest and will remain so.

Completely agreed!


>  The key enabler for
> automation is device accepting arbitrary new B config when it is
> running arbitrary new A config and transition there hitlessly.
> Generating full new config from DB+template is trivial problem, trying
> to be aware of network state and move from arbitrary state A to
> arbitrary state B with minimal amount of changes is hard and
> unnecessary problem.

I tend to agree with you, Saku. What I've heard (from the vendors,
again) is that Ansible is not great because you don't inherently get
state confirmation feedback after posting the new configuration, and
that adding that intelligence into Ansible requires time and energy to
code. Okay, fair point, I'll bite. But also, we are network engineers -
we know what commands do when they run, and we've spent decades building
templates from as simple as a Windows Notepad text to as complex as a
MySQL database.

Then again, Terraform is meant to fix that downside of Ansible, but for
me, I don't really see that as a big issue. We aren't trying to
provision services across network domains (despite what MEF's LSO
architecture will have you believe), and even if we were, do I really
want you fiddling in my network. We each know our networks better than
outsiders know them, so what gives?


> If/when network becomes more cloudified, more as-a-service, where you
> use API to turn up your own active devices and circuits where you
> want, when you want, instead of owning anything and once those
> proprietary APIs get some subset standard APIs we'll probably start to
> see openstack, kubernetes type of complexity explosion in networks
> too.

MEF's LSO, which they've been pushing since about 2014. The concept is
sexy, but honestly, I've not heard much ado in 6 years re: real-world
deployment.

Also, while I'm wild enough to be one of the first maniacs to run a
network-wide Route Reflector on a VM on a server in 2014, you won't find
me deploying said RR's in AWS or Azure, so I can access them over some
API into an Openstack/Kubernetes/Docker enclosure. Life is too
interesting enough as it is :-).


>  But as long as we keep owning the network most will keep running
> it CLI jjockey network, touch when you must, but in many cases no one
> touches it for weeks or months.

Words of wisdom.

I just want to get back to building, operating and optimizing IP/MPLS
networks. I don't mind seeing the back of the last 10 years of SD-this
and 5G-that.

Mark.