[cisco-voip] UCCX 8.5(1)SU2 Restore Woes

Tue Mar 14 06:31:57 EDT 2017

THrowing this out in case someone's seen anything similar.  Will be lodging
a TAC case in the morning.

Have a UCCX HA Cluster Setup running 8.5(1)SU2.  Currently trying to
restore the backup into an isolated network but keep hitting an issue with
the CCXCOMPONENT restore failing at 25%.

Running the COP02 and COP03 patches on both boxes (to get round the LDAP
Backup/Restore defect) and have a good backup from the cluster.

Restore Steps:
 - Have a restored copy of the CUCM Cluster up and running ok.
 - Built the CCX PUB node as 8.5(1)SU2 and did the initial setup using temp
username etc for the CTI and RMCM Users.
 - Installed COP02 and rebooted, followed by COP03 and a restart.
 - Attempted to run the restore of just the PUB.  All components restore
successfully until we get to CCXCOMPONENT.  This fails at 25% with the
error 32-Failed to start UCCX DB Service.

Log file extract is below.

Anyone seen similar?  Can't see anything in Bug DB.  Restore notes mention
that if you're on SU2 to complete the intial setup using temp users,
install COP files and then restore, so I think I'm doing this right.

========================================= Log file extract
==========================================

2017/03/14 18:18: Calling do_restore script of DB component
2017/03/14 18:18: Executing cmd:
/opt/cisco/uccx/drf/backup/db_do_restore.py /common/drf/restore.log
/common/drf/status.txt
2017-03-14 18:18:47,888 - uccxdb.bnr - DEBUG - ------------ Begin Unified
CCX DB DoRestore(db_do_restore.py) ------------
2017-03-14 18:18:47,892 - uccxdb.bnr - DEBUG - Executing
[/usr/local/cm/bin/controlcenter.sh list | grep "Cisco Unified CCX
Database" ] ...
2017-03-14 18:18:53,360 - uccxdb.bnr - DEBUG - Results of executing
[/usr/local/cm/bin/controlcenter.sh list | grep "Cisco Unified CCX
Database" ], returnCode :[0], Output :[log4j:ERROR Could not find value for
key log4j.appender.custom
log4j:ERROR Could not instantiate appender named "custom".
Cisco Unified CCX Database[STARTING] ]
2017-03-14 18:18:53,360 - uccxdb.bnr - DEBUG - Executing [cat
/common/drf/ccx_comps/uccx_db/uccx_version.txt] ...
2017-03-14 18:18:53,367 - uccxdb.bnr - DEBUG - Backup version of
UCCX:8.5.1.11002-22
2017-03-14 18:18:53,476 - uccxdb.bnr - DEBUG - Restore version of
UCCX:8.5.1.11002-22
2017-03-14 18:18:53,476 - uccxdb.bnr - DEBUG - UCCX Backup and Restore
versions are same
2017-03-14 18:18:53,476 - uccxdb.bnr - DEBUG - Copying DB configuration
files from backup
2017-03-14 18:18:53,477 - uccxdb.bnr - DEBUG - Copying
/common/drf/ccx_comps/uccx_db/onconfig.uccx
2017-03-14 18:18:53,477 - uccxdb.bnr - DEBUG - Copying
/common/drf/ccx_comps/uccx_db/sqlhosts.uccx
2017-03-14 18:18:53,478 - uccxdb.bnr - DEBUG - Copying
/common/drf/ccx_comps/uccx_db/uccx_ids.env
2017-03-14 18:18:53,478 - uccxdb.bnr - DEBUG - Copying
/common/drf/ccx_comps/uccx_db/services
2017-03-14 18:18:53,479 - uccxdb.bnr - DEBUG - Executing
[/usr/local/bin/base_scripts/xmlfoo
/usr/local/platform/conf/platformConfig.xml CcmFirstNode ParamValue] ...
2017-03-14 18:18:53,488 - uccxdb.bnr - DEBUG - Results of executing
[/usr/local/bin/base_scripts/xmlfoo
/usr/local/platform/conf/platformConfig.xml CcmFirstNode ParamValue],
returnCode :[0], Output :[yes]
2017-03-14 18:18:53,488 - uccxdb.bnr - DEBUG - UCCX DB Ontape Backup is
present in this node
2017-03-14 18:18:53,488 - uccxdb.bnr - DEBUG - Initiating UCCX DB Restore
operation...
2017-03-14 18:18:53,488 - uccxdb.bnr - DEBUG - Executing [/bin/chmod 664
/common/drf/ccx_comps/uccx_db/ontape_uccx_backup.gz] ...
2017-03-14 18:18:53,492 - uccxdb.bnr - DEBUG - Executing [source
/opt/cisco/uccx/bin/uccx_ids.env ; onstat -] ...
2017-03-14 18:18:53,522 - uccxdb.bnr - DEBUG - Results of executing [source
/opt/cisco/uccx/bin/uccx_ids.env ; onstat -], returnCode :[65280], Output
:[shared memory not initialized for INFORMIXSERVER 'ccxprd01_uccx']
2017-03-14 18:18:53,522 - uccxdb.bnr - DEBUG - Executing [source
/opt/cisco/uccx/bin/uccx_ids.env ; onmode -yuk] ...
2017-03-14 18:18:53,553 - uccxdb.bnr - DEBUG - Results of executing [source
/opt/cisco/uccx/bin/uccx_ids.env ; onmode -yuk], returnCode :[0], Output
:[shared memory not initialized for INFORMIXSERVER 'ccxprd01_uccx']
2017-03-14 18:18:53,554 - uccxdb.bnr - DEBUG - Unified CCX DB is not up and
running(not On-Line). Trying to start...
2017-03-14 18:18:53,554 - uccxdb.bnr - DEBUG - Executing [source
/opt/cisco/uccx/bin/uccx_ids.env ; oninit -vyw] ...
2017-03-14 18:18:58,130 - uccxdb.bnr - DEBUG - Results of executing [source
/opt/cisco/uccx/bin/uccx_ids.env ; oninit -vyw], returnCode :[256], Output
:[Checking group membership to determine server run mode...succeeded
Reading configuration file
'/usr/local/cm/db/informix/etc/onconfig.uccx'...succeeded
Creating /INFORMIXTMP/.infxdirs...succeeded
Checking config parameters...succeeded
Allocating and attaching to shared memory...succeeded
Creating resident pool 13960 kbytes...succeeded
Allocating 100016 kbytes for buffer pool of 2K page size...succeeded
Creating infos file
"/usr/local/cm/db/informix/etc/.infos.ccxprd01_uccx"...succeeded
Linking conf file
"/usr/local/cm/db/informix/etc/.conf.ccxprd01_uccx"...succeeded
Initializing rhead structure...succeeded
Writing to infos file...succeeded
Initialization of Encryption...succeeded
Initializing ASF...succeeded
Initializing Dictionary Cache and SPL Routine Cache...succeeded
Bringing up ADM VP...succeeded
Creating VP classes...succeeded
Onlining 0 additional cpu vps...succeeded

WARNING: server initialization failed, or possibly timed out (if -w was
used).
Check the message log, online.log, for errors.]
2017-03-14 18:18:58,130 - uccxdb.bnr - DEBUG - Failing to start Unified CCX
Database
2017-03-14 18:18:58,130 - uccxdb.bnr - DEBUG - ----- Exit Unified CCX DB
DoRestore(db_do_restore.py) with returnCode = 32 : Unsucccessful -----
2017/03/14 18:18: Results of executing
[/opt/cisco/uccx/drf/backup/db_do_restore.py /common/drf/restore.log
/common/drf/status.txt], returnCode :[32], Output :[]
2017/03/14 18:18: DO_Restore DB Script failed to work
2017/03/14 18:18: Re-setting UCCX Archive Flags
2017/03/14 18:18: Executing cmd:
/opt/cisco/uccx/drf/backup/restore_resetflags.py /common/drf/restore.log
2017/03/14 18:18: Re-Setting UCCX Archive Flags for Restore
2017/03/14 18:19: UCCX Archive flags has been reset
2017/03/14 18:19: Results of executing
[/opt/cisco/uccx/drf/backup/restore_resetflags.py /common/drf/restore.log],
returnCode :[0], Output :[]
2017/03/14 18:19: Re-Setting of Archive flags completed
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20170314/89497bc9/attachment.html>