<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Jumping in on the thread, what is your CDR retention set to? Do you redirect to a 3<sup>rd</sup> party CDR such as ISI? We have only a 30 day retention on our
servers I believe. We never use the CDR from the server.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">We had a lot of IO when our CDR was set to a few months, The IO was from the deletion of the old CDR at the end of the month. It has been a few years since
we changed the settings.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="mso-element:para-border-div;border:none;border-bottom:solid windowtext 1.5pt;padding:0in 0in 1.0pt 0in">
<p class="MsoNormal" style="border:none;padding:0in"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Neal Haas<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> cisco-voip-bounces@puck.nether.net [mailto:cisco-voip-bounces@puck.nether.net]
<b>On Behalf Of </b>Erick Wellnitz<br>
<b>Sent:</b> Friday, April 05, 2013 8:21 AM<br>
<b>To:</b> Tom Piscitell (tpiscite)<br>
<b>Cc:</b> cisco-voip<br>
<b>Subject:</b> Re: [cisco-voip] high I/O Wait on one core<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">caroninit seems to be the biggest offender (by about 50x) in both disk writes and cpu usage. Am I correct in assuming this has something to do with call detail records?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Fri, Apr 5, 2013 at 9:02 AM, Tom Piscitell (tpiscite) <<a href="mailto:tpiscite@cisco.com" target="_blank">tpiscite@cisco.com</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Erick,<br>
<br>
You can use the FIOR utility from the CLI to identify which processes are writing to the disk.<br>
<br>
admin:utils fior<br>
utils fior disable<br>
utils fior enable<br>
utils fior list<br>
utils fior start<br>
utils fior status<br>
utils fior stop<br>
utils fior top<br>
<br>
Here is a typical use case:<br>
<br>
1. Enable the FIOR utility before/during a time of High IO Wait<br>
admin:utils fior enable<br>
File I/O Statistics has been enabled.<br>
admin:utils fior start<br>
Loading fiostats module: ok<br>
Enabling fiostats : ok<br>
File I/O Statistics has been started.<br>
<br>
2. Wait a couple minutes. FIOR will poll for data every 5 seconds I believe. Then use utils fior top to see whats hitting the CPU the hardest:<br>
<br>
admin:utils fior top ?<br>
Syntax:<br>
utils fior top n sort_by [start=date-time] [stop=date-time]<br>
<br>
n: number of processes<br>
sort_by: read, write, read-rate, write-rate<br>
date-time: of the form %H:%M, %H:%M:%S<br>
%a,%H:%M, %a,%H:%M:%S<br>
%Y-%m-%d,%H:%M, %Y-%m-%d %H:%M:%S<br>
Example:<br>
admin:utils fior top 10 write start=2010-04-20 10:00:00 stop=2010-04-20 10:30:00<br>
<br>
This of course won't tell you *why* a process is hitting the disk, but it will at least show you who has the most read/writes. To answer the why question you would need to look at traces for the offending process/service.<br>
<br>
HTH,<br>
-Tom<br>
<br>
On Apr 4, 2013, at 5:43 PM, Erick Wellnitz <<a href="mailto:ewellnitzvoip@gmail.com">ewellnitzvoip@gmail.com</a>> wrote:<br>
<br>
> Hello all!<br>
><br>
> I have a dual 4 core IBM 7835I3 which is my publisher. One one core of the first CPU the I/O Wait is through the roof. RTMT shows that writes to the hard drives are at between 600 and 700 MB/s which is exponentially higher than the subscriber on the same
model of hardware.<br>
><br>
> Short of calling TAC is there any way to figure out what is causing the extremely high volume of writes to the drives? I already stopped most traces and looking at the processes doesn't give any clues.<br>
><br>
> Thanks again!<br>
><br>
><br>
> _______________________________________________<br>
> cisco-voip mailing list<br>
> <a href="mailto:cisco-voip@puck.nether.net">cisco-voip@puck.nether.net</a><br>
> <a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank">https://puck.nether.net/mailman/listinfo/cisco-voip</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>