[cisco-voip] CallProcessingNodeCpuPegging

[Kevin Thorngren]

It is possible that could be the cause of either the files ending up in 
the BAD files folder and/or the CPU spike.  I guess it would depend on 
what your application does and how you are using the CDR Insert 
service.

Hope the reboot helps!!

Kevin

On Jul 10, 2006, at 1:07 PM, Voll, Scott wrote:

> I see all my calls in CDR, so I believe everything is working 
> correctly.
>  
> We are using a cron job to move CDR files to a offbox SQL server to do 
> billing.  Could that cause it?  It was done in house.
>  
> The BAD folder has multiple files per day from july last year though 
> last month. 
>  
> Thanks
>  
> Scott
>  
> PS… I’m still rebooting tonight.
>  
>
> From: Kevin Thorngren [mailto:kthorngr at cisco.com]
> Sent: Monday, July 10, 2006 9:56 AM
> To: Voll, Scott
> Cc: cisco-voip at puck.nether.net
> Subject: Re: [cisco-voip] CallProcessingNodeCpuPegging
>  
> Well, 3000 files shouldn't cause a CPU spike. Obviously the 2695 files 
> in the BAD folder are causing the event log message you are seeing. I 
> think the threshold is 200. It has been a long time since I have had 
> to troubleshoot CDRs moving to the BAD folder. CDR Insert traces would 
> help but they may need to be set to detailed when the next CDR is 
> moved to the BAD folder to find the problem.
>  
> Does it seem like these files are all recent and that none of your 
> CDRs are being inserted?
>  
> I have seen issues were customers using third party CDR tools run into 
> issues with the SQL trigger from the third party. This could cause the 
> CDRs to move to the BAD folder.
>  
> When I was in TAC I never got away with asking the customer to reboot 
> ;-)
>  
> Based on the info you provided I am not convinced that the CPU spike 
> is a result of the CDR Insert issue (although I won't rule anything 
> out). The only reason I mentioned troubleshooting before rebooting is 
> if CDRs were not being inserted and staying in the CDR folder then 
> there would be a possibility of fixing the CDR Insert problem without 
> a reboot. Then you could see if the CPU spike problem was resolved.
>  
> Kevin
> On Jul 10, 2006, at 12:15 PM, Voll, Scott wrote:
>>  
>> Kevin—
>>  
>> Thanks for the reply….. here is what I know.
>>  
>> Pub:
>>  CDR directory – 0 files / folders
>>  CMR Dirctory – 0 Files / Folders
>>  Bad directory – 2695 files
>>  
>> Sub:
>>  CDR directory – 0 files / folders
>>  CMR Dirctory – 0 Files / Folders
>>  
>> I just restarted the CDR insert service on the PUB.
>>  
>> Why do I want to trouble shoot further before rebooting?  Seems as 
>> it’s windows it might be a better solution.
>>  
>> Scott
>>  
>> From: Kevin Thorngren [mailto:kthorngr at cisco.com]
>> Sent: Monday, July 10, 2006 8:59 AM
>> To: Voll, Scott
>> Cc: cisco-voip at puck.nether.net
>> Subject: Re: [cisco-voip] CallProcessingNodeCpuPegging
>>  
>> This may be a separate issue.
>>  
>> If you are getting a build up of thousands of files in one of the 
>> following directories then you would see the System process spike the 
>> CPU each time the folder is accessed.
>>  
>> C:\Program Files\Cisco\CallDetail\CDR
>> C:\Program Files\Cisco\CallDetail\CMR
>> C:\Program Files\Cisco\CallDetail\BAD
>>  
>> But I would suspect that you would see these spikes throughout the 
>> day, each time a file is copied over from one of the Subscribers. I 
>> would recommend waiting until after hours before checking to see how 
>> many files are in these folders as you run the possibility of spiking 
>> the CPU if there are many thousands of files.
>>  
>> Maybe a restart of the CDR Insert service would resolve the issue. 
>> Again I would wait for after hours in case of a CPU spike. You might 
>> want to enable the CDR Insert traces to troubleshoot the problem 
>> before doing any of the restarts/reboots.
>>  
>> I would recommend starting a perfmon log of Processor usage to find 
>> out which process is spiking the CPU.
>>  
>> Kevin
>>  
>> On Jul 10, 2006, at 11:35 AM, Voll, Scott wrote:
>>>  
>>> I see this in the event log:
>>>  
>>> Event ID 3
>>> Source:  Cisco Database Layer
>>>  
>>> Error: kErrorCDRFilesBackingUp - CDR flat files are backing up.
>>>   App ID: Cisco Database Layer Monitor
>>>   Cluster ID: CMPUB-Cluster
>>>   Node ID: CMPUB
>>> Explanation: CDR flat files are not being removed.  On the primary 
>>> CDR server, verify that the InsertCDR service is running and 
>>> properly configured. On a server not the primary, verify that the 
>>> location for collecting CDR files is accessible via the network.
>>> Recommended Action: Set trace for InsertCDR service to detailed and 
>>> look for errors in the trace.  Check enterprise CDR parameters for 
>>> accuracy..
>>>  
>>> I went into the Services and all look to be running.  So I’m not 
>>> sure what’s up.  I will be rebooting it tonight.
>>>  
>>> Scott
>>>  
>>>  
>>>  
>>> From: Kevin Thorngren [mailto:kthorngr at cisco.com]
>>> Sent: Monday, July 10, 2006 8:01 AM
>>> To: Voll, Scott
>>> Cc: cisco-voip at puck.nether.net
>>> Subject: Re: [cisco-voip] CallProcessingNodeCpuPegging
>>>  
>>> Do you know which process is pegging the CPU?
>>>  
>>> Kevin
>>> On Jul 10, 2006, at 10:52 AM, Voll, Scott wrote:
>>>>  
>>>> I installed OS 4.2.sr8 and since then I have been getting CPU 
>>>> pegging out around the midnight hour.  I believe it’s the CDR flat 
>>>> not getting truncated.
>>>>  
>>>> Has anyone else seen this?
>>>>  
>>>> Scott
>>>> _______________________________________________
>>>> cisco-voip mailing list
>>>> cisco-voip at puck.nether.net
>>>> https://puck.nether.net/mailman/listinfo/cisco-voip
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 31463 bytes
Desc: not available
Url : https://puck.nether.net/pipermail/cisco-voip/attachments/20060710/2b5e7a1a/attachment-0001.bin