<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
The 4 arrays on BE7K also have an impact on IOPS for the volume, though the servers are primarily built that way for array rebuild times as has been noted previously in this thread.
<div class=""><br class="">
</div>
<div class="">On that server especially while one volume may be fine from a space perspective note that the amount of IOPS for DAS is derived from the number of disk spindles in the array. </div>
<div class="">This means it is entirely possible to cause IO starvation problems on a BE7K by putting all the VMs on a single storage volume. </div>
<div class="">Spreading your VMs across the arrays will provide you protection against multiple disk failures (because they are RAID 5) AND spread the IOPS load across them.</div>
<div class=""><br class="">
</div>
<div class="">With respect to rebuilding a TRC’s RAID5 into RAID6 or RAID10 that server is no longer classified as a TRC and thus Cisco cannot guarantee the same level of performance we would otherwise.</div>
<div class="">You may be perfectly comfortable with making such a change and willing to take on the risk, but guaranteeing the server meets the IOPS requirements of the apps running on it is your responsibility, not ours.</div>
<div class=""><br class="">
</div>
<div class="">If things go sideways you can expect TAC to ask you to rebuild it to a RAID5 (or at least one volume) the way it shipped as part of the troubleshooting process.<br class="">
<div class=""><br class="">
<div class="">-Ryan </div>
<br class="">
<div>
<div class="">On Nov 14, 2017, at 11:33 AM, Lelio Fulgenzi <<a href="mailto:lelio@uoguelph.ca" class="">lelio@uoguelph.ca</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Yeah, I hear ya. I mean, it’s not like there are not advantages, but, still, the managing of which array to put things in. ugh.<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="Apple-converted-space"> </span>Charles Goldsmith [<a href="mailto:wokka@justfamily.org" class="">mailto:wokka@justfamily.org</a>]<span class="Apple-converted-space"> </span><br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>Tuesday, November 14, 2017 10:48 AM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>Lelio Fulgenzi <<a href="mailto:lelio@uoguelph.ca" class="">lelio@uoguelph.ca</a>><br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span>Ryan Huff <<a href="mailto:ryanhuff@outlook.com" class="">ryanhuff@outlook.com</a>>;
<a href="mailto:cisco-voip@puck.nether.net" class="">cisco-voip@puck.nether.net</a><br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [cisco-voip] UCS C210 Replace 146 GB Disk in RAID5 with 300 GB Disk<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Lelio, if you are using those be7k's for UC apps, your limiting factor is always the CPU cores, you won't run out of drive space. Also, the be7K-h is a workhorse of a server, fastest one I've ever built a cluster on. 4 independent arrays means you can easily
do maintenance on one app without affecting the performance of another, assuming you separate your apps between arrays and stagger them out. I did 4 simultaneous installs on one, each app on it's own partition and they were all done in under 2 hours.<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Sadly, the pricepoint on the H isn't there for most customers. the M is nice, but only has 2 of the arrays, but I'll take an H for an install any day!<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Tue, Nov 14, 2017 at 9:34 AM, Lelio Fulgenzi <<a href="mailto:lelio@uoguelph.ca" target="_blank" style="color: purple; text-decoration: underline;" class="">lelio@uoguelph.ca</a>> wrote:<o:p class=""></o:p></div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;" class="">
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The idea of RAID drives and managing the different volumes always had me on the fence on how to do things. In a perfect world, I’d stick with one big RAID 6 array with a spare on the shelf.<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
The BE7K servers I ordered were delivered with 4 RAID 5 arrays. Personally, while I can appreciate separating the arrays, I don’t like losing that extra space and managing which volume to put images on is a pain.<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I’ll admit, I looked at RAID 10 (when I was first reading the TRC specs) and was confused to heck. I did finally understand things after referring to a colleague, but it was a lot of drawing out.<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I will say this, RAID isn’t gonna protect you if you don’t have platform monitoring on. You need to know the second a drive fails so you can proceed accordingly.<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Also, if the ever do construction in your computer room, do yourself a favour, go to the hardware store, buy a 9.99 loose fibre furnace filter and stick it in front of your air intakes.<span class="Apple-converted-space"> </span><o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<b class="">From:</b><span class="Apple-converted-space"> </span>cisco-voip [mailto:<a href="mailto:cisco-voip-bounces@puck.nether.net" target="_blank" style="color: purple; text-decoration: underline;" class="">cisco-voip-bounces@puck.nether.net</a>]<span class="Apple-converted-space"> </span><b class="">On
Behalf Of<span class="Apple-converted-space"> </span></b>Charles Goldsmith<br class="">
<b class="">Sent:</b><span class="Apple-converted-space"> </span>Tuesday, November 14, 2017 9:48 AM<br class="">
<b class="">To:</b><span class="Apple-converted-space"> </span>Ryan Huff <<a href="mailto:ryanhuff@outlook.com" target="_blank" style="color: purple; text-decoration: underline;" class="">ryanhuff@outlook.com</a>><br class="">
<b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:cisco-voip@puck.nether.net" target="_blank" style="color: purple; text-decoration: underline;" class="">cisco-voip@puck.nether.net</a><br class="">
<b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [cisco-voip] UCS C210 Replace 146 GB Disk in RAID5 with 300 GB Disk<o:p class=""></o:p></div>
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I've seen one URE fail in a raid 5 resilvering process, years ago on a DG system. Had to rebuild and restore from backup, fun times.<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I agree Ryan, on a TRC system and RMA a drive, you stick with it. <o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
From my reading on TRC, you can rebuild as a RAID 10 and get faster speeds, but you lose some space in the process.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On my personal systems, I'm using RAID 10 everywhere.<o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Tue, Nov 14, 2017 at 8:17 AM, Ryan Huff <<a href="mailto:ryanhuff@outlook.com" target="_blank" style="color: purple; text-decoration: underline;" class="">ryanhuff@outlook.com</a>> wrote:<o:p class=""></o:p></div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt;" class="">
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
As I’ve read and understood; it isn’t due to actual functionality though. It is as you say, due mostly to longer rebuild times (indexing a physically larger geometry than the rest of the array members, for a smaller logical geometry) and the risk (rare IMO)
to the rest of the array (as a rebuild will stress the array and could cause other, near-death disks to fail thereby causing the array to fail). It also wastes the extra horsepower of the disk since the existing RAID can’t capitalize on the resources of the
larger disk.<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
So in a case of, would you go out and buy a new disk that way .... I’d say no; but if that is the result of a covered RMA, I’d say go for it.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 11pt; font-family: Calibri, sans-serif;">
I’m no diskologist though ... just based on my own experiences of what has worked for me for the last couple of decades ... and I’ve never lost a server ... outside of that one time when my pants pocket snagged the release on the 2nd disk in a R5 on my way
out the door ... bad memories.<o:p class=""></o:p></p>
<div id="m_3193799730819117075m_3660877515088355866AppleMailSignature" class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<span style="color: rgb(136, 136, 136);" class="">-Ryan</span><o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<br class="">
On Nov 14, 2017, at 9:03 AM, Charles Goldsmith <<a href="mailto:wokka@justfamily.org" target="_blank" style="color: purple; text-decoration: underline;" class="">wokka@justfamily.org</a>> wrote:<o:p class=""></o:p></p>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="">
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Keep in mind, RAID 5 is ok for smaller disks, but larger disks it's no longer recommended, but sadly, the best article about it is from Dell: <a href="http://en.community.dell.com/techcenter/b/techcenter/archive/2012/08/14/new-equallogic-raid-tech-report-considerations-and-best-practices-released" target="_blank" style="color: purple; text-decoration: underline;" class="">http://en.community.dell.com/techcenter/b/techcenter/archive/2012/08/14/new-equallogic-raid-tech-report-considerations-and-best-practices-released</a><o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
With bigger disks, it's even said that RAID 6 is no longer good enough, due to large rebuild times in case of a failure. <a href="http://www.zdnet.com/blog/storage/why-raid-6-stops-working-in-2019/805" target="_blank" style="color: purple; text-decoration: underline;" class="">http://www.zdnet.com/blog/storage/why-raid-6-stops-working-in-2019/805</a> <o:p class=""></o:p></div>
</div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
On Tue, Nov 14, 2017 at 7:41 AM, Ryan Huff <<a href="mailto:ryanhuff@outlook.com" target="_blank" style="color: purple; text-decoration: underline;" class="">ryanhuff@outlook.com</a>> wrote:<o:p class=""></o:p></div>
<blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin: 5pt 0in 5pt 4.8pt;" class="">
<div class="">
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 11pt; font-family: Calibri, sans-serif;">
Reto,<o:p class=""></o:p></p>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Seek/rpm speeds and media type (flash, sata ... etc) are usually what matter the most for RAID disks. If your only difference is total storage capacity, the bigger disk will usually work just fine, your just gonna waste the additional 154GB of space (because
the RAID will only provision 146GB of that 300GB disk).<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Just remember on a RAID 5, don’t pull/lose more that 1 disk at a time .... painful lesson long ago I share over beer every now and then.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
-Ryan<o:p class=""></o:p></div>
</div>
<div class="">
<div class="">
<div class="">
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<br class="">
On Nov 14, 2017, at 8:23 AM, Reto Gassmann <<a href="mailto:voip@mrga.ch" target="_blank" style="color: purple; text-decoration: underline;" class="">voip@mrga.ch</a>> wrote:<o:p class=""></o:p></p>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="">
<div class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Hallo <o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
We have a UCS C210 Server with 10x146 GB Disks. One of the Disks failed and I got a 300 GB replacement Disk from Cisco.<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Is that a problem if I replace the defect 146 Disk in the RAID 5 with a 300 GB Disk?<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Thanks for help<o:p class=""></o:p></div>
</div>
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Regards Reto<o:p class=""></o:p></div>
</div>
</div>
</blockquote>
</div>
</div>
<blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class="">
<div class="">
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
_______________________________________________<br class="">
cisco-voip mailing list<br class="">
<a href="mailto:cisco-voip@puck.nether.net" target="_blank" style="color: purple; text-decoration: underline;" class="">cisco-voip@puck.nether.net</a><br class="">
<a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank" style="color: purple; text-decoration: underline;" class="">https://puck.nether.net/mailman/listinfo/cisco-voip</a><o:p class=""></o:p></div>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin: 0in 0in 12pt; font-size: 11pt; font-family: Calibri, sans-serif;">
<br class="">
_______________________________________________<br class="">
cisco-voip mailing list<br class="">
<a href="mailto:cisco-voip@puck.nether.net" target="_blank" style="color: purple; text-decoration: underline;" class="">cisco-voip@puck.nether.net</a><br class="">
<a href="https://puck.nether.net/mailman/listinfo/cisco-voip" target="_blank" style="color: purple; text-decoration: underline;" class="">https://puck.nether.net/mailman/listinfo/cisco-voip</a><o:p class=""></o:p></p>
</blockquote>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""></o:p></div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
</div>
<span style="font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">_______________________________________________</span><br style="font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">cisco-voip
mailing list</span><br style="font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="mailto:cisco-voip@puck.nether.net" class="">cisco-voip@puck.nether.net</a></span><br style="font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">
<span style="font-family: LucidaGrande; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""><a href="https://puck.nether.net/mailman/listinfo/cisco-voip" class="">https://puck.nether.net/mailman/listinfo/cisco-voip</a></span></div>
</div>
<br class="">
</div>
</div>
</body>
</html>