[cisco-voip] CUCM Upgrade failure...
Jonathan Charles
jonvoip at gmail.com
Tue Nov 17 12:40:18 EST 2015
Yep, didn't work... same error. Let me check the other nodes...
On Tue, Nov 17, 2015 at 11:34 AM, Jonathan Charles <jonvoip at gmail.com>
wrote:
> Logged into OS Admin on the pub, and it didn't give me the option to
> assume control... can I run discovery/migration during production? Just
> want to make sure I don't pop anything....
>
>
>
> Jonathan
>
> On Tue, Nov 17, 2015 at 11:30 AM, Dave Goodwin <dave.goodwin at december.net>
> wrote:
>
>> I had a similar symptom a couple months ago - discovery timed out on one
>> or more nodes. I found a supportforum post from another individual having
>> that issue, and it appeared that PCD got "stuck" installing
>> the ciscocm.ucmap_platformconfig.cop file in the middle. The way to work
>> around it was to log into the Software Install/Upgrade page on the affected
>> node(s) and you can see you'll have the option to Assume Control of a
>> currently running install. Do that, and click through to complete the
>> install. Then try to do the discovery again. Unknown if there was an
>> existing bug causing this problem, but the above workaround steps solved my
>> problem.
>>
>> On Tue, Nov 17, 2015 at 12:25 PM, Jonathan Charles <jonvoip at gmail.com>
>> wrote:
>>
>>> I just posted the log... I am not sure from reading it...
>>>
>>> discovery node 1617 failed with errorcode 1-3
>>>
>>>
>>> Jonathan
>>>
>>> On Tue, Nov 17, 2015 at 11:23 AM, Ryan Huff <ryanhuff at outlook.com>
>>> wrote:
>>>
>>>> Jonathan,
>>>>
>>>> What timesout on the publisher? Are you referencing when PCD tries to
>>>> do the cluster discovery on the existing cluster?
>>>>
>>>> -Ryan
>>>>
>>>>
>>>> Sent from my T-Mobile 4G LTE Device
>>>>
>>>>
>>>> -------- Original message --------
>>>> From: Jonathan Charles
>>>> Date:11/17/2015 12:06 PM (GMT-05:00)
>>>> To: Anthony Holloway
>>>> Cc: cisco-voip at puck.nether.net
>>>> Subject: Re: [cisco-voip] CUCM Upgrade failure...
>>>>
>>>> Nope, not all good... new error... the upgrade failed so I deleted the
>>>> cluster and re-added... it finds all of the subscribers, but it times out
>>>> on the publisher....
>>>>
>>>> I have verified all services are running on the Pub and it looks
>>>> clean...
>>>>
>>>>
>>>>
>>>> Jonathan
>>>>
>>>> On Mon, Nov 16, 2015 at 10:20 AM, Anthony Holloway <
>>>> avholloway+cisco-voip at gmail.com> wrote:
>>>>
>>>>> Looks like you're all good now, but as a heads up to everyone else,
>>>>> don't stop at checking NTP with "utils ntp status". You will fail to
>>>>> upgrade if your NTP configuration has an FQHN for the NTP server which
>>>>> begins with a digit.
>>>>>
>>>>> E.g., 0.pool.ntp.org
>>>>>
>>>>> You will not see the hostname in the output of "utils ntp status", as
>>>>> it will only show you the resolved IP address. So, you will also need to
>>>>> issue a "utils ntp config" to see what value was entered by the
>>>>> administrator.
>>>>>
>>>>> This is the only defect reference I found, though my upgrade I hit it
>>>>> on was an 8.6 to 10.5 Refresh Upgrade (RU) (Not PCD).
>>>>>
>>>>> https://tools.cisco.com/bugsearch/bug/CSCtj07817
>>>>>
>>>>> On Sun, Nov 15, 2015 at 10:43 PM, Jonathan Charles <jonvoip at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> OK, a reboot of CPCD got it passed that error...
>>>>>>
>>>>>>
>>>>>> Jonathan
>>>>>>
>>>>>> On Sun, Nov 15, 2015 at 9:40 PM, Jonathan Charles <jonvoip at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Yeah, the error I am getting is:
>>>>>>>
>>>>>>> 1 nodes(s) in Export task action ID #1127... on the Publisher...
>>>>>>>
>>>>>>> I will try rebooting everything...
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Jonathan
>>>>>>>
>>>>>>> On Sun, Nov 15, 2015 at 9:35 PM, Ryan Huff <ryanhuff at outlook.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Looks healthy ...
>>>>>>>>
>>>>>>>> I recall trying PCD once and I hit really strange issues too. For
>>>>>>>> that upgrade, I ultimately abandoned PCD and built new VMs with the Answer
>>>>>>>> File Generator then a DRS backup/restore.
>>>>>>>>
>>>>>>>> Not sure where you are in your timeline or if it is that important
>>>>>>>> but it is definitely something I would consider. Sometimes you can spend
>>>>>>>> more time trying to get the silly tools to work, than to just do the work
>>>>>>>> yourself.
>>>>>>>>
>>>>>>>> Google is littered with PCD weirdness; great idea of an
>>>>>>>> application, just not there yet IMO.
>>>>>>>>
>>>>>>>> -Ryan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Sent from my iPad
>>>>>>>> On Nov 15, 2015, at 10:19 PM, Jonathan Charles <jonvoip at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Everything looks good....
>>>>>>>>
>>>>>>>>
>>>>>>>> admin:utils ntp status
>>>>>>>> ntpd (pid 19674) is running...
>>>>>>>>
>>>>>>>> remote refid st t when poll reach delay
>>>>>>>> offset jitter
>>>>>>>>
>>>>>>>> ==============================================================================
>>>>>>>> 127.127.1.0 .LOCL. 10 l 21 64 377 0.000
>>>>>>>> 0.000 0.001
>>>>>>>> 10.0.31.2 10.0.31.3 3 u 175 1024 377 0.635
>>>>>>>> -5.771 2.042
>>>>>>>> *10.0.31.3 129.6.15.29 2 u 970 1024 377 0.510
>>>>>>>> -11.340 0.449
>>>>>>>> 10.1.31.2 10.0.31.3 3 u 490 1024 377 0.850
>>>>>>>> -9.114 4.881
>>>>>>>> +10.1.31.3 129.6.15.29 2 u 184 1024 377 0.817
>>>>>>>> -4.085 5.355
>>>>>>>>
>>>>>>>>
>>>>>>>> synchronised to NTP server (10.0.31.3) at stratum 3
>>>>>>>> time correct to within 68 ms
>>>>>>>> polling server every 1024 s
>>>>>>>>
>>>>>>>> Current time in UTC is : Mon Nov 16 03:16:14 UTC 2015
>>>>>>>> Current time in America/Chicago is : Sun Nov 15 21:16:14 CST 2015
>>>>>>>> admin:
>>>>>>>>
>>>>>>>> admin:utils diagnose module validate_network
>>>>>>>>
>>>>>>>> Log file: platform/log/diag1.log
>>>>>>>>
>>>>>>>> Starting diagnostic test(s)
>>>>>>>> ===========================
>>>>>>>> test - validate_network : Passed
>>>>>>>>
>>>>>>>> Diagnostics Completed
>>>>>>>>
>>>>>>>> admin:#
>>>>>>>>
>>>>>>>> admin:utils dbreplication runtimestate
>>>>>>>>
>>>>>>>> DB and Replication Services: ALL RUNNING
>>>>>>>>
>>>>>>>> Cluster Replication State: Replication repair command started at:
>>>>>>>> 2014-06-20-23-22
>>>>>>>> Replication repair command COMPLETED 541 tables processed out
>>>>>>>> of 541
>>>>>>>> Errors or Mismatches Were Found:
>>>>>>>>
>>>>>>>> Use 'file view activelog
>>>>>>>> cm/trace/dbl/sdi/ReplicationRepair.2014_06_20_23_22_51.out' to see the
>>>>>>>> details
>>>>>>>>
>>>>>>>> DB Version: ccm8_6_2_20000_2
>>>>>>>> Number of replicated tables: 541
>>>>>>>>
>>>>>>>> Cluster Detailed View from PUB (5 Servers):
>>>>>>>>
>>>>>>>> PING REPLICATION
>>>>>>>> REPL. DBver& REPL. REPLICATION SETUP
>>>>>>>> SERVER-NAME IP ADDRESS (msec) RPC? STATUS
>>>>>>>> QUEUE TABLES LOOP? (RTMT) & details
>>>>>>>> ----------- ------------ ------ ---- -----------
>>>>>>>> ----- ------- ----- -----------------
>>>>>>>> IPTCMS02 10.0.126.12 0.196 Yes Connected 0
>>>>>>>> match Yes (2) Setup Completed
>>>>>>>> IPTCMS01 10.0.126.11 0.151 Yes Connected 0
>>>>>>>> match Yes (2) Setup Completed
>>>>>>>> IPTCMP 10.0.126.10 0.065 Yes Connected 0
>>>>>>>> match Yes (2) PUB Setup Completed
>>>>>>>> IPTCMS03 10.1.126.13 0.545 Yes Connected 0
>>>>>>>> match Yes (2) Setup Completed
>>>>>>>> IPTCMS04 10.1.126.14 0.527 Yes Connected 0
>>>>>>>> match Yes (2) Setup Completed
>>>>>>>>
>>>>>>>> admin:
>>>>>>>>
>>>>>>>> On Sun, Nov 15, 2015 at 9:14 PM, Ryan Huff <ryanhuff at outlook.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Also worth noting that if CCM NTP is synchronized to a Windows
>>>>>>>>> server (even if it shows Stratum 3 or better); that is a problem you'll
>>>>>>>>> need to correct as SNTP can play hell with UCOS and do some pretty weird
>>>>>>>>> stuff.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Ryan
>>>>>>>>>
>>>>>>>>> On Nov 15, 2015, at 10:06 PM, Ryan Huff <ryanhuff at outlook.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> If the FROM CCM version was unrestricted, it would say "
>>>>>>>>> *Unrestricted*" after the version number on the "*Active Master
>>>>>>>>> Version"* line. If it does not say "*Unrestricted*", then it is
>>>>>>>>> the more common restricted version.
>>>>>>>>>
>>>>>>>>> As to your original issue, I would start with all the usual
>>>>>>>>> suspects. Is the FROM CCM cluster healthy to start with; dns, ntp,
>>>>>>>>> replication ... etc?
>>>>>>>>>
>>>>>>>>> From CCM:
>>>>>>>>>
>>>>>>>>> 1.) #utils diagnose module validate_network
>>>>>>>>> (Should see *Passed*)
>>>>>>>>>
>>>>>>>>> 2.) #utils ntp status
>>>>>>>>> (Pub should be synchronized and Strata 3 or better)
>>>>>>>>>
>>>>>>>>> 3.) #utils dbreplication runtimestate
>>>>>>>>> (Should see *2 - Setup Completed* for all nodes)
>>>>>>>>>
>>>>>>>>> If PCD is moving the apps to a new platform/chassis, make sure the
>>>>>>>>> *target* environment can reach all the same network assets as the
>>>>>>>>> *from* environment.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Ryan
>>>>>>>>>
>>>>>>>>> On Nov 15, 2015, at 9:34 PM, Jonathan Charles <jonvoip at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> It just says the version number...
>>>>>>>>>
>>>>>>>>> admin:show version active
>>>>>>>>> Active Master Version: 8.6.2.20000-2
>>>>>>>>> Active Version Installed Software Options:
>>>>>>>>> cmterm-7942_7962-sccp.9-3-1ES27-rel.cop
>>>>>>>>> cmterm-devicepack8.6.2.24118-1.cop
>>>>>>>>> ciscocm.refresh_upgrade_v1.1.cop
>>>>>>>>> ciscocm.ucmap_platformconfig.cop
>>>>>>>>> ciscocm.migrate-export-v1.12.cop
>>>>>>>>> admin:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Jonahan
>>>>>>>>>
>>>>>>>>> On Sun, Nov 15, 2015 at 7:26 PM, Ryan Huff <ryanhuff at outlook.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Did you actually dump the logs to a serial interface (curious
>>>>>>>>>> what it shows)?
>>>>>>>>>>
>>>>>>>>>> On the FROM CCM, goto the CLI of pub (or sub) and do a "show
>>>>>>>>>> version active"; it will tell you if you have the UNREST.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>>
>>>>>>>>>> Ryan
>>>>>>>>>>
>>>>>>>>>> > On Nov 15, 2015, at 7:55 PM, Jonathan Charles <
>>>>>>>>>> jonvoip at gmail.com> wrote:
>>>>>>>>>> >
>>>>>>>>>> > Using PCD on CUCM 10.5.2.11901 got the following error:
>>>>>>>>>> >
>>>>>>>>>> > <image.png>
>>>>>>>>>> >
>>>>>>>>>> > It seems to imply I am not matching restricted vs.
>>>>>>>>>> unrestricted...
>>>>>>>>>> >
>>>>>>>>>> > Any easy way to find out?
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> >
>>>>>>>>>> > Jonathan
>>>>>>>>>> > _______________________________________________
>>>>>>>>>> > cisco-voip mailing list
>>>>>>>>>> > cisco-voip at puck.nether.net
>>>>>>>>>> > https://puck.nether.net/mailman/listinfo/cisco-voip
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> cisco-voip mailing list
>>>>>>>>> cisco-voip at puck.nether.net
>>>>>>>>> https://puck.nether.net/mailman/listinfo/cisco-voip
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> cisco-voip mailing list
>>>>>> cisco-voip at puck.nether.net
>>>>>> https://puck.nether.net/mailman/listinfo/cisco-voip
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> _______________________________________________
>>> cisco-voip mailing list
>>> cisco-voip at puck.nether.net
>>> https://puck.nether.net/mailman/listinfo/cisco-voip
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://puck.nether.net/pipermail/cisco-voip/attachments/20151117/395eaea0/attachment.html>
More information about the cisco-voip
mailing list