[c-nsp] new ASR9901 ios update problem

Sat Nov 2 11:58:51 EDT 2019

Thanks fort he flowers,  Aaron!

Now i got stuck in those patches called SMU.

Not only the mentioned time-consuming (each reload takes 15..20 minutes) is boring,
But after installing most of the SMUs and ony 5..7 remaining from the bunch of 80+-5 SMUs,
the XXXXX device tells me on its console port, that the root filesystem is over 80% or more full.

LC/0/0/CPU0:Nov  2 12:47:56.505 CET: resmon[290]: %HA-HA_WD-3-DISK_ALARM_ALERT : A monitored device / ( rootfs:/ ) is above 80% utilization. Current utilization = 80. Please remove unwanted user files and configuration rollback points.

Googling for this i found

https://www.cisco.com/c/en/us/support/docs/ios-nx-os-software/ios-xr-software/116332-maintain-ios-xr-smu-00.html
[...]
Bootflash is above 80% utilization

The following message may appear after SMU installation.
RP/0/RSP0/CPU0:Jul  9 17:40:37.959 : wdsysmon[447]: %HA-HA_WD-4-DISK_WARN : A monitored device /bootflash: is above 80% utilization. Current utilization = 89.  Please remove unwanted user files and configuration rollback points.
This message can be safely ignored.
As per design it is expected that IOS-XR will keep up to two MBIs on the bootflash following SMU install(s). At subsequent SMU install(s), if the bootflash space required by the new package(s) is not available, IOS-XR will clean up automatically old MBIs to make space for the new MBI package.
[...]

So i did continue and no it is 99-100% full, "install add source ..."
works but "install activate ..." aborts.

I do not have "userfiles" on it, i did but the ios,tar,smu's onto "harddisk:" .
I did not find any hint how to make space there,
i tried

"clear configuration commits oldest 100"

"install remove inactive all synchronous"

But this did not help.

#show install log 250 detail
Sat Nov  2 12:56:50.744 CET
Nov 02 09:56:57 Install operation 250 started by jm:
  install activate id 249 
Nov 02 09:56:57 Package list:
Nov 02 09:56:57     asr9k-mgbl-x64-2.0.0.4-r653.CSCvr46090.x86_64
Nov 02 09:57:01 Action 1: install prepare action started
Nov 02 09:57:03 Install operation will continue in the background
Nov 02 09:57:03 The prepared software is set to be activated with process restart
Nov 02 09:57:47 Start preparing software for local installation
Nov 02 09:57:59 Action 1: install prepare action completed successfully
Nov 02 09:58:00 Action 2: install activate action started
Nov 02 09:58:00 The software will be activated with process restart
Nov 02 09:58:01 Activating XR packages
Nov 02 09:59:12 Node 0/RSP0/CPU0 encountered error(s) during operation. Please check 'show install log 250 detail' for error details
Nov 02 09:59:12 
----------------------------------------------------------------------------
Error stack for location 0/RSP0/CPU0

        1# Available disk space(including additional buffer 104857600) 215699456 is not sufficient for rpm installation of archive size 110199132 
        2# failed to load files from ldpath (new)

        Please collect 'show tech-support install one-showtech' from XR and 'show tech-support ctrace' from Admin and pass this information to your TAC representative for support.

----------------------------------------------------------------------------
Nov 02 09:59:12 Agent on the lead has err'ed during SWC_BEGIN Aborting the operation
Nov 02 09:59:12 Action 2: install activate action aborted
Nov 02 10:00:21 Install operation 250 aborted
Nov 02 10:00:21 Ending operation 250

I submitted the output from 'show tech-support install one-showtech' to my TAC case
But i have not found out how to move the "admin'show tech-support ctrace'" output
out of the box. Looks like admin-harddisk: is not the same as harddisk:
and also admin copy does not know ftp as destination (and i believe it will not work
with my mgmt-vrf, ip information is a stange 192.168.0.4 not my mgmt-ip, ....)
Very very strange ☹

BTW, When i was at the approx. 80% SMU installation point,
i got the hint from tac that i can untar the SMUs,
and bundle them (without the .txt files) in one tape-archive to get it   installed faster.
Way too late after 3 days of work....

Looks that the documentation on how to upgrade the box has never been tested
(and in/output captured)
and also, no-one had ever tried to add all recommended patches.

Any idea on what is blocking space on / and can be removed ?

Repartiion and install from scratch ?
RMA it and get a refurbished device with scratches 
instead of this expensive brand new garbage ?

I am also a little bit afraid on using such a thing for production.,
Thought version 6.5 would be matured and procedures well documented
And the TAC people can give me concrete answers not generic blarney.

->sigh<-

Jürgen Marenda.

> -----Ursprüngliche Nachricht-----
> Von: Aaron Gould <aaron1 at gvtc.com>
> Btw, good job, and thanks Jürgen for the informative and detailed instruction
> on XR upgrade.