[j-nsp] Automation - The Skinny (Was: Re: ACX5448 & ACX710)

[Mark Tinka]

On 23/Jan/20 12:59, Thomas Scott wrote:

> I had that conversation the again other day - someone said they were
> working on "automation" and when I probed deeper it revealed some
> (very useful, albeit not scalable) scripting. 'You keep using that
> word. I do not think it means what you think it means' seems to be the
> gist of most "automation" conversations, and I'm guilty often as not -
> 'automate it' seems to be a catch all for most of our operations issues.

You raise an important point.

The "automation" story is something most operators feel they need to be
talking about, and if they aren't doing anything yet or don't know how
to contribute to the discussion, feel better about not admitting that
they aren't doing anything about it nor have a plan yet that they are
really comfortable with.

When it all started going to hell with "SDN" back in 2011, skeptical-me
was immediately activated. I attended the MPLS/SDN/NFV/IPv6 congress in
Paris between 2013 - 2015, and realized I had little appetite for venues
where most presenters are trying to out-hype each other with the next
buzzword. I mean, you start to get a little "Uh huh" when someone comes
up to the stage and announces, "MPLS is dead and will no longer be seen
in networks from 2015".

Since about 2012, every time we've felt we've come close to finding an
"automation" solution that actually works and scales as good as it
sounds on the tin and from the community, we just get that niggling
feeling that things just aren't yet quite ready.

We don't have swaths of software developers lining our corridors ready
to write code for whatever we want. The few we do have are inundated
with other tasks that could have immediate but long-lasting solutions,
as we wade through the maze and sea that is "automation".

In 2020, after all the stories and buzzwords, we are honing in on
Ansible and Terraform, and even with those, we are being careful not to
waste time and resources we don't have. All other solution, IMNSHO, are
just a waste of time, in our view.

My 1+1 assessment of all of these issues is, I believe, down to the fact
that the industry wants to automate in an open standards manner, where
both vendors and operators are able to solve all implementation and
operational tasks with automation. If we did not have to have an open
standards mechanism for this, we'd all be automating - individually - to
our heart's content. I mean, we all know that the largest of largest
operators have had internal automation for decades, and despite all the
work going on at an industry level "to automate", they continue to have
and build their own internal automation tools that are bespoke and
proprietary to them. While they see the need to join the industry in
standardizing stuff they've already been doing for decades, they also
aren't feeling the pressure to push that harder than they could. After
all, the tools they've built are what has given them the edge, and they
aren't going to let that go anytime soon. And yes, even though we
sometimes get crumbs and drabs from the large content providers, when
they are in the mood, it's what they are willing to share with us, their
subjects. There's heaps more of interesting and exciting code they will
never share with the community, even though they are some of the loudest
faces you will find pushing for "industry-standard automation".

Since general consensus amongst those who haven't been "automating" for
a while now is that we should only automate if it is standardized, the
majority of the industry really interested in that agenda gets both
stuck in limbo and yo-yo, almost in an endless loop. Why, because unlike
large network operators who've been automating for decades, and unlike
large content providers who can throw 1,000 software developers at one
command line, the rest of the Internet community is not as gifted.

While I'm not suggesting that automating internally and independently of
the industry is the solution, my recommendation is to take a step back
from all the noise, as an individual operator, and assess your place in
the entire mix, and what this all means to you as a single network, and
to the industry that you love and want to support into the next 100 years.

For us, we still have a healthy dose of CLI communication with our
network to keep our engineers fresh and actually in tune with really
understanding what the network is actually doing. But we are also going
down the Ansible + Terraform path like Indiana Jones into a cave...
slowly and looking out for the mine fields laid not by Ansible or
Terraform, but by the politics automation has created in our industry.

It is also good to realize that if we are working from a "Day 1 to Day
End" position, that's never going to come. So we need to be deliberately
(and perhaps, selfishly) smarter about the paths we want to go down re:
"automation", as individual network operators.

Mark.