[c-nsp] new ASR9901 ios update problem

Wed Nov 6 03:16:23 EST 2019

I have hope that one day the Cisco gods will discover "apt update; apt upgrade" and all this sorcery that we need for an upgrade will become a thing of the past.

________________________________
From: cisco-nsp <cisco-nsp-bounces at puck.nether.net> on behalf of cnsp at marenda.net <cnsp at marenda.net>
Sent: Saturday, November 2, 2019 10:58 AM
To: 'Aaron Gould' <aaron1 at gvtc.com>; cnsp at marenda.net <cnsp at marenda.net>; cisco-nsp at puck.nether.net <cisco-nsp at puck.nether.net>
Subject: Re: [c-nsp] new ASR9901 ios update problem

Thanks fort he flowers,  Aaron!

Now i got stuck in those patches called SMU.

Not only the mentioned time-consuming (each reload takes 15..20 minutes) is boring,
But after installing most of the SMUs and ony 5..7 remaining from the bunch of 80+-5 SMUs,
the XXXXX device tells me on its console port, that the root filesystem is over 80% or more full.

LC/0/0/CPU0:Nov  2 12:47:56.505 CET: resmon[290]: %HA-HA_WD-3-DISK_ALARM_ALERT : A monitored device / ( rootfs:/ ) is above 80% utilization. Current utilization = 80. Please remove unwanted user files and configuration rollback points.

Googling for this i found

https://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/ios-xr-software/116332-maintain-ios-xr-smu-00.html
[...]
Bootflash is above 80% utilization

The following message may appear after SMU installation.
RP/0/RSP0/CPU0:Jul  9 17:40:37.959 : wdsysmon[447]: %HA-HA_WD-4-DISK_WARN : A monitored device /bootflash: is above 80% utilization. Current utilization = 89.  Please remove unwanted user files and configuration rollback points.
This message can be safely ignored.
As per design it is expected that IOS-XR will keep up to two MBIs on the bootflash following SMU install(s). At subsequent SMU install(s), if the bootflash space required by the new package(s) is not available, IOS-XR will clean up automatically old MBIs to make space for the new MBI package.
[...]

So i did continue and no it is 99-100% full, "install add source ..."
works but "install activate ..." aborts.

I do not have "userfiles" on it, i did but the ios,tar,smu's onto "harddisk:" .
I did not find any hint how to make space there,
i tried

"clear configuration commits oldest 100"

"install remove inactive all synchronous"

But this did not help.

#show install log 250 detail
Sat Nov  2 12:56:50.744 CET
Nov 02 09:56:57 Install operation 250 started by jm:
  install activate id 249
Nov 02 09:56:57 Package list:
Nov 02 09:56:57     asr9k-mgbl-x64-2.0.0.4-r653.CSCvr46090.x86_64
Nov 02 09:57:01 Action 1: install prepare action started
Nov 02 09:57:03 Install operation will continue in the background
Nov 02 09:57:03 The prepared software is set to be activated with process restart
Nov 02 09:57:47 Start preparing software for local installation
Nov 02 09:57:59 Action 1: install prepare action completed successfully
Nov 02 09:58:00 Action 2: install activate action started
Nov 02 09:58:00 The software will be activated with process restart
Nov 02 09:58:01 Activating XR packages
Nov 02 09:59:12 Node 0/RSP0/CPU0 encountered error(s) during operation. Please check 'show install log 250 detail' for error details
Nov 02 09:59:12
----------------------------------------------------------------------------
Error stack for location 0/RSP0/CPU0

        1# Available disk space(including additional buffer 104857600) 215699456 is not sufficient for rpm installation of archive size 110199132
        2# failed to load files from ldpath (new)

        Please collect 'show tech-support install one-showtech' from XR and 'show tech-support ctrace' from Admin and pass this information to your TAC representative for support.

----------------------------------------------------------------------------
Nov 02 09:59:12 Agent on the lead has err'ed during SWC_BEGIN Aborting the operation
Nov 02 09:59:12 Action 2: install activate action aborted
Nov 02 10:00:21 Install operation 250 aborted
Nov 02 10:00:21 Ending operation 250

I submitted the output from 'show tech-support install one-showtech' to my TAC case
But i have not found out how to move the "admin'show tech-support ctrace'" output
out of the box. Looks like admin-harddisk: is not the same as harddisk:
and also admin copy does not know ftp as destination (and i believe it will not work
with my mgmt-vrf, ip information is a stange 192.168.0.4 not my mgmt-ip, ....)
Very very strange ☹

BTW, When i was at the approx. 80% SMU installation point,
i got the hint from tac that i can untar the SMUs,
and bundle them (without the .txt files) in one tape-archive to get it   installed faster.
Way too late after 3 days of work....

Looks that the documentation on how to upgrade the box has never been tested
(and in/output captured)
and also, no-one had ever tried to add all recommended patches.

Any idea on what is blocking space on / and can be removed ?

Repartiion and install from scratch ?
RMA it and get a refurbished device with scratches
instead of this expensive brand new garbage ?

I am also a little bit afraid on using such a thing for production.,
Thought version 6.5 would be matured and procedures well documented
And the TAC people can give me concrete answers not generic blarney.

->sigh<-

Jürgen Marenda.

> -----Ursprüngliche Nachricht-----
> Von: Aaron Gould <aaron1 at gvtc.com>
> Btw, good job, and thanks Jürgen for the informative and detailed instruction
> on XR upgrade.

_______________________________________________
cisco-nsp mailing list  cisco-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

________________________________

CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files or previous e-mail messages attached to it may contain confidential information that is legally privileged. If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of any of the information contained in or attached to this transmission is STRICTLY PROHIBITED. If you have received this transmission in error please notify the sender immediately by replying to this e-mail. You must destroy the original transmission and its attachments without reading or saving in any manner. Thank you.