[cisco-voip] CUCM 8.5.1 SU1 to SU3 update failure on CAR db

Erick B. erickbee at gmail.com
Thu May 17 11:57:37 EDT 2012 

Wes,

Backups are good and include CAR Data, all successful and I did a
manual backup fine with all options.

Haven't ran the recovery disc yet ever on this node, I don't recall
instance where the fs went read-only on this in the past. There was
SAN issue and CCM/CTI services Core dumped and restarted at same time.
The upgrade to SU3 is to get correction for Disk I/O timeout increase
from 30sec to 180 sec actually (CSCtj97676).

I was going to purge the CAR database and see if that helped.  Is
there a way to recreate the CAR db on the linux platform without
lengthy TAC involvement? I knew I could do it on Windows.

Thanks Wes.

On Thu, May 17, 2012 at 10:25 AM, Wes Sisk <wsisk at cisco.com> wrote:
> Looks like a couple of things going on.  Migrating CAR data took significant time and likely timed out.  That will slow the upgrade. Toss the CAR data unless you really need it.
>
> Then you hit:
> "Primary and Mirror chunks are bad". We have several of these on record where the consensus seems to be: "Database restore failed because of bad chunks in active partition."
>
> Has this sever ever suffered an unexpected reboot?  Look for it using tips from:
> CSCtr88859    add alarm and alert for unexpected reboot
>
> Has recovery disk been used with filesystem repair? Note that fs repair only truncates bad files. Thus:
> CSCth60800    Recovery Disc warning to rebuild system after file system repair
>
>
> I am not clear if the existing in-place database is bad or if it is an export or newly created database that is bad. If the former that will be a big problem. If the later then it suggests a problem with the filesystem and potentially hardware.  Checking and repairing those may be sufficient.
>
> To get a better idea check recent backups. Are backups successful? Backups have to dump the entire database. Does the backup include the CAR database? Can you take a full backup?
>
> Regards,
> Wes
>
>
>
> On May 17, 2012, at 10:08 AM, Erick B. wrote:
>
> Does anyone know of any update issues from 8.5.1 SU1 to SU3?  I've
> done some looking, hot RSS feeds, etc and not finding anything yet and
> am working with TAC on this but just thought I'd ask here.
>
> Upgrade patch ran fine for about an hour, then gave error below. I did
> the update via SFTP and the file size and MD5 matched.
>
> Error encountered: An unknown error occurred while accessing the upgrade file
>
> This is snippet from install logs, it is fine before this and appears
> to have an issue with the CAR database.
>
>
> 05/17/2012 00:33:36 component_install|(CAPTURE) run command:
> /usr/local/cm/bin/pryftool --system cm --hive dbl --set
> current_informixserver="cm1_car8_5_1_13900_5"|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) Successfully added
> current_informixserver in dbl prefs.SET
> prior_informixserver|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) Inside Linux|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:35,721 WARN  [main] migration.MigrationTool
> (MigrationTool.java:529) - Exception in cm_car_post_time
> script|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:35,740 WARN  [main] migration.MigrationTool
> (MigrationTool.java:530) - Following things might have
> happened|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:35,746 WARN  [main] migration.MigrationTool
> (MigrationTool.java:531) - 1.User might have set the time to 60
> mins|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:35,747 WARN  [main] migration.MigrationTool
> (MigrationTool.java:532) - 2.DMABackupInfo.inf file or CARMigraionTime
> entry may not be present |<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:35,747 WARN  [main] migration.MigrationTool
> (MigrationTool.java:533) - Hence setting the default migration time
> |<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:35,748 INFO  [main] migration.MigrationTool
> (MigrationTool.java:108) - main()-->|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) Thu May 17 00:33:35
> CDT 2012:main:connect() SQLException|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) Thu May 17 00:33:35
> CDT 2012:main:|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:35,996 WARN  [main] dbl.Connector (Connector.java:1184) -
> Connector(NOT CONNECTED:
> Driver=com.informix.jdbc.IfxDriver;)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) java.sql.SQLException:
> Cannot open system catalog (systables).|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.a(IfxSqli.java:3457)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.E(IfxSqli.java:3774)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.dispatchMsg(IfxSqli.java:2580)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.receiveMessage(IfxSqli.java:2496)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.executeOpenDatabase(IfxSqli.java:2021)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqliConnect.<init>(IfxSqliConnect.java:1622)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxDriver.connect(IfxDriver.java:254)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> java.sql.DriverManager.getConnection(DriverManager.java:582)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> java.sql.DriverManager.getConnection(DriverManager.java:207)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.getConn(Connector.java:769)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.connect(Connector.java:677)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.connect(Connector.java:619)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.prepareStatement(Connector.java:978)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.prepareStatement(Connector.java:964)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.beans.EBSystemPreferences.loadCurrentData(EBSystemPreferences.java:171)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.beans.EBSystemPreferences.<init>(EBSystemPreferences.java:98)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.database.SchemaCreator.<clinit>(SchemaCreator.java:111)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.migration.MigrationTool.main(MigrationTool.java:110)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) Caused by:
> java.sql.SQLException: ISAM error: Primary and Mirror chunks are
> bad|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.util.IfxErrMsg.getSQLException(IfxErrMsg.java:408)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.E(IfxSqli.java:3779)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         ... 20 more|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) Thu May 17 00:33:36
> CDT 2012:main:handleConnectionException: ErrorCode is
> -311|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:36,007 WARN  [main] dbl.Connector (Connector.java:1187) -
> handleConnectionException: ErrorCode is -311|<LVL::Debug>
>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:36,007 ERROR [main] beans.EBSystemPreferences
> (EBSystemPreferences.java:199) - loadCurrentData(): Error while
> loading Sys Pref data|<LVL::Debug>
>
> 05/17/2012 00:33:36 component_install|(CAPTURE) java.sql.SQLException:
> Cannot open system catalog (systables).|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.a(IfxSqli.java:3457)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.E(IfxSqli.java:3774)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.dispatchMsg(IfxSqli.java:2580)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.receiveMessage(IfxSqli.java:2496)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.executeOpenDatabase(IfxSqli.java:2021)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqliConnect.<init>(IfxSqliConnect.java:1622)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxDriver.connect(IfxDriver.java:254)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> java.sql.DriverManager.getConnection(DriverManager.java:582)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> java.sql.DriverManager.getConnection(DriverManager.java:207)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.getConn(Connector.java:769)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.connect(Connector.java:677)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.connect(Connector.java:619)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.prepareStatement(Connector.java:978)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.dbl.Connector.prepareStatement(Connector.java:964)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.beans.EBSystemPreferences.loadCurrentData(EBSystemPreferences.java:171)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.beans.EBSystemPreferences.<init>(EBSystemPreferences.java:98)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.database.SchemaCreator.<clinit>(SchemaCreator.java:111)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.cisco.ccm.car.migration.MigrationTool.main(MigrationTool.java:110)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE) Caused by:
> java.sql.SQLException: ISAM error: Primary and Mirror chunks are
> bad|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.util.IfxErrMsg.getSQLException(IfxErrMsg.java:408)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         at
> com.informix.jdbc.IfxSqli.E(IfxSqli.java:3779)|<LVL::Debug>
> 05/17/2012 00:33:36 component_install|(CAPTURE)         ... 20 more|<LVL::Debug>
>
> 05/17/2012 00:33:36 component_install|(CAPTURE) 2012-05-17
> 00:33:36,008 ERROR [main] database.SchemaCreator
> (SchemaCreator.java:128) - error while setting the manual purge status
> to 0 |<LVL::Debug>
> 05/17/2012
> _______________________________________________
> cisco-voip mailing list
> cisco-voip at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-voip
>
>

More information about the cisco-voip mailing list