<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Just a general ‘watch out’ with regards to 5.5e I’d like to share, as we’ve had very bad results with 5.5 (including 5.5e)
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">with regards to IPv6 in combination with IS-IS
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">(though TAC also mentioned it was on OSPF as well, but don’t see that in the release notes)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal">DEFECT000500944<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Router-A has for example a path towards eg. the IPv6 loopback address of Router-B via eth1/4<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Now eth1/4 on Router-A goes down.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><br>
IS-IS picks for example eth 3/4 as the new best path towards the loopback.<br>
Now, a ‘show ipv6 route <loopback-of-Router-B>’ correctly shows the new interface (eth 3/4)...
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Yet the IPv6 cache keeps a stale entry towards eth 1/4... and traffic from Router-A towards
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Router-B’s loopback is now blackholed.... as it still tries to send it out eth 1/4 (even when it’s down)<br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">This affected in our case for example IPv6 iBGP sessions.... so far for redundant links, etc.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Physical or ve does not matter.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Also, if I recall correctly, nasty stuff also happened when you simply made a metric change of the IS-IS path.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">(Eg. when eth 3/4 is towards Router-C, and Router-C has a best path towards Router-B’s loopback via the link to Router-A)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I believe it was only problematic for ‘local’ traffic from Router-A, and not for transit traffic - but not 100% sure anymore.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Fixed in 5.6.something, not sure if they ever fixed it in 5.5<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Simplest workaround once you are affected is executing the ‘hidden’ command (</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">DEFECT000503937
) : </span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">clear ipv6 cache<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Best regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><br>
Wouter<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:windowtext"> foundry-nsp [mailto:foundry-nsp-bounces@puck.nether.net]
<b>On Behalf Of </b>Jeroen Wunnink | Hibernia Networks<br>
<b>Sent:</b> Wednesday, February 18, 2015 17:32<br>
<b>To:</b> Brad Fleming; Frank Bulk<br>
<b>Cc:</b> foundry-nsp@puck.nether.net<br>
<b>Subject:</b> Re: [f-nsp] MLX throughput issues<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Try physically pulling the SFM's one by one rather then just powercycling them. Yes there is a difference there :-)<br>
<br>
Also, 5400d is fairly buggy, there were some major issues with CRC checksums in hSFM's and on 100G cards. We have fairly good results with 5500e<br>
<br>
<br>
<br>
On 18/02/15 16:50, Brad Fleming wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">TAC replaced hSFMs and line cards the first couple times but we’ve seen this issue at least once on every node in the network. The ones where we replaced every module (SFM, mgmt, port cards, even PSUs) have still had at least one event.
So I’m not even sure what hardware we’d replace at this point. That lead us to thinking a config problem since each box uses the same template but after a lengthy audit with TAC nobody could find anything. It happens infrequently enough that we grew to just
live with it. <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Feb 18, 2015, at 12:45 AM, Frank Bulk <<a href="mailto:frnkblk@iname.com">frnkblk@iname.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">So don’t errors like this suggest replacing the hardware?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Frank</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"> </span><o:p></o:p></p>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span class="apple-converted-space"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span></span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">foundry-nsp
[<a href="mailto:foundry-nsp-bounces@puck.nether.net">mailto:foundry-nsp-bounces@puck.nether.net</a>]<span class="apple-converted-space"> </span><b>On Behalf Of<span class="apple-converted-space"> </span></b>Brad Fleming<br>
<b>Sent:</b><span class="apple-converted-space"> </span>Tuesday, February 17, 2015 3:10 PM<br>
<b>To:</b><span class="apple-converted-space"> </span>Josh Galvez<br>
<b>Cc:</b><span class="apple-converted-space"> </span><a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [f-nsp] MLX throughput issues</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">The common symptoms for us are alarms of TM errors / resets. We’ve been told on multiple TAC cases that logs indicating transmit TM errors are likely caused by problems in one of the SFM links / lanes. We’ve been told that resetting the
SFMs one at a time will clear the issue.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Symptoms during the issue is that 1/3rd of the traffic moving from one TM to another TM will simply get dropped. So we see TCP globally start to throttle like crazy and if enough errors count up the TM will simply reset. After the TM reset
is seems a 50/50 chance the box will remain stable or go back to dropping packets within ~20mins. So when we see a TM reset we simply do the SFM Dance no matter what.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">On Feb 16, 2015, at 10:08 PM, Josh Galvez <<a href="mailto:josh@zevlag.com"><span style="color:purple">josh@zevlag.com</span></a>> wrote:<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Why kind of wigout? And how do you diagnose the corruption? I'm intrigued.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">On Mon, Feb 16, 2015 at 8:43 AM, Brad Fleming <<a href="mailto:bdflemin@gmail.com" target="_blank"><span style="color:purple">bdflemin@gmail.com</span></a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">We’ve seen it since installing the high-capacity switch fabrics into our XMR4000 chassis roughly 4 years ago. We saw it through IronWare 5.4.00d. I’m not sure what software we were using when they were first installed; probably whatever
would have been stable/popular around December 2010.<br>
<br>
Command is simply “power-off snm [1-3]” then “power-on snm [1-3]”.<br>
<br>
Note that the power-on process causes your management session to hang for a few seconds. The device isn’t broken and packets aren’t getting dropped; it’s just going through checks and echoing back status.<br>
<br>
-brad<o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><br>
<br>
> On Feb 16, 2015, at 7:07 AM, Jethro R Binks <<a href="mailto:jethro.binks@strath.ac.uk"><span style="color:purple">jethro.binks@strath.ac.uk</span></a>> wrote:<br>
><br>
> On Fri, 13 Feb 2015, Brad Fleming wrote:<br>
><br>
>> Over the years we’ve seen odd issues where one of the<br>
>> switch-fabric-links will “wigout” and some of the data moving between<br>
>> cards will get corrupted. When this happens we power cycle each switch<br>
>> fab one at a time using this process:<br>
>><br>
>> 1) Shutdown SFM #3<br>
>> 2) Wait 1 minute<br>
>> 3) Power SFM #3 on again<br>
>> 4) Verify all SFM links are up to SFM#3<br>
>> 5) Wait 1 minute<br>
>> 6) Perform steps 1-5 for SFM #2<br>
>> 7) Perform steps 1-5 for SFM #3<br>
>><br>
>> Not sure you’re seeing the same issue that we see but the “SFM Dance”<br>
>> (as we call it) is a once-every-four-months thing somewhere across our<br>
>> 16 XMR4000 boxes. It can be done with little to no impact if you are<br>
>> patient verify status before moving to the next SFM.<br>
><br>
> That's all interesting. What code versions is this? Also, how do you<br>
> shutdown the SFMs? I don't recall seeing documentation for that.<br>
><br>
> Jethro.<br>
><br>
><br>
>><br>
>>> On Feb 13, 2015, at 11:41 AM,<span class="apple-converted-space"> </span><a href="mailto:nethub@gmail.com"><span style="color:purple">nethub@gmail.com</span></a><span class="apple-converted-space"> </span>wrote:<br>
>>><br>
>>> We have three switch fabrics installed, all are under 1% utilized.<br>
>>><br>
>>><br>
>>> From: Jeroen Wunnink | Hibernia Networks [mailto:<a href="mailto:jeroen.wunnink@atrato.com"><span style="color:purple">jeroen.wunnink@atrato.com</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:jeroen.wunnink@atrato.com"><span style="color:purple">jeroen.wunnink@atrato.com</span></a>>]<br>
>>> Sent: Friday, February 13, 2015 12:27 PM<br>
>>> To:<span class="apple-converted-space"> </span><a href="mailto:nethub@gmail.com"><span style="color:purple">nethub@gmail.com</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:nethub@gmail.com"><span style="color:purple">nethub@gmail.com</span></a>>;
'Jeroen Wunnink | Hibernia Networks'<br>
>>> Subject: Re: [f-nsp] MLX throughput issues<br>
>>><br>
>>> How many switchfabrics do you have in that MLX and how high is the utilization on them<br>
>>><br>
>>> On 13/02/15 18:12,<span class="apple-converted-space"> </span><a href="mailto:nethub@gmail.com"><span style="color:purple">nethub@gmail.com</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:nethub@gmail.com"><span style="color:purple">nethub@gmail.com</span></a>>
wrote:<br>
>>>> We also tested with a spare Quanta LB4M we have and are seeing about the same speeds as we are seeing with the FLS648 (around 20MB/s or 160Mbps).<br>
>>>><br>
>>>> I also reduced the number of routes we are accepting down to about 189K and that did not make a difference.<br>
>>>><br>
>>>><br>
>>>> From: foundry-nsp [mailto:<a href="mailto:foundry-nsp-bounces@puck.nether.net"><span style="color:purple">foundry-nsp-bounces@puck.nether.net</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:foundry-nsp-bounces@puck.nether.net"><span style="color:purple">foundry-nsp-bounces@puck.nether.net</span></a>>]
On Behalf Of Jeroen Wunnink | Hibernia Networks<br>
>>>> Sent: Friday, February 13, 2015 3:35 AM<br>
>>>> To:<span class="apple-converted-space"> </span><a href="mailto:foundry-nsp@puck.nether.net"><span style="color:purple">foundry-nsp@puck.nether.net</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:foundry-nsp@puck.nether.net"><span style="color:purple">foundry-nsp@puck.nether.net</span></a>><br>
>>>> Subject: Re: [f-nsp] MLX throughput issues<br>
>>>><br>
>>>> The FLS switches do something weird with packets. I've noticed they somehow interfere with changing the MSS window size dynamically, resulting in destinations further away having very poor speed results compared to destinations close by.<br>
>>>><br>
>>>> We got rid of those a while ago.<br>
>>>><br>
>>>><br>
>>>> On 12/02/15 17:37,<span class="apple-converted-space"> </span><a href="mailto:nethub@gmail.com"><span style="color:purple">nethub@gmail.com</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:nethub@gmail.com"><span style="color:purple">nethub@gmail.com</span></a>>
wrote:<br>
>>>>> We are having a strange issue on our MLX running code 5.6.00c. We are encountering some throughput issues that seem to be randomly impacting specific networks.<br>
>>>>><br>
>>>>> We use the MLX to handle both external BGP and internal VLAN routing. Each FLS648 is used for Layer 2 VLANs only.<br>
>>>>><br>
>>>>> From a server connected by 1 Gbps uplink to a Foundry FLS648 switch, which is then connected to the MLX on a 10 Gbps port, running a speed test to an external network is getting 20MB/s.<br>
>>>>><br>
>>>>> Connecting the same server directly to the MLX is getting 70MB/s.<br>
>>>>><br>
>>>>> Connecting the same server to one of my customer's Juniper EX3200 (which BGP peers with the MLX) also gets 70MB/s.<br>
>>>>><br>
>>>>> Testing to another external network, all three scenarios get 110MB/s.<br>
>>>>><br>
>>>>> The path to both test network locations goes through the same IP transit provider.<br>
>>>>><br>
>>>>> We are running NI-MLX-MR with 2GB RAM, NI-MLX-10Gx4 connect to the Foundry FLS648 by XFP-10G-LR, NI-MLX-1Gx20-GC was used for directly connecting the server. A separate NI-MLX-10Gx4 connects to our upstream BGP providers. Customer’s Juniper EX3200 connects
to the same NI-MLX-10Gx4 as the FLS648. We take default routes plus full tables from three providers by BGP, but filter out most of the routes.<br>
>>>>><br>
>>>>> The fiber and optics on everything look fine. CPU usage is less than 10% on the MLX and all line cards and CPU usage at 1% on the FLS648. ARP table on the MLX is about 12K, and BGP table is about 308K routes.<br>
>>>>><br>
>>>>> Any assistance would be appreciated. I suspect there is a setting that we’re missing on the MLX that is causing this issue.<br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>> _______________________________________________<br>
>>>>> foundry-nsp mailing list<br>
>>>>><span class="apple-converted-space"> </span><a href="mailto:foundry-nsp@puck.nether.net"><span style="color:purple">foundry-nsp@puck.nether.net</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:foundry-nsp@puck.nether.net"><span style="color:purple">foundry-nsp@puck.nether.net</span></a>><br>
>>>>><span class="apple-converted-space"> </span><a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank"><span style="color:purple">http://puck.nether.net/mailman/listinfo/foundry-nsp</span></a><span class="apple-converted-space"> </span><<a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank"><span style="color:purple">http://puck.nether.net/mailman/listinfo/foundry-nsp</span></a>><br>
>>>><br>
>>>><br>
>>>><br>
>>>> --<br>
>>>><br>
>>>> Jeroen Wunnink<br>
>>>> IP NOC Manager - Hibernia Networks<br>
>>>> Main numbers (Ext: 1011): USA<span class="apple-converted-space"> </span><a href="tel:%2B1.908.516.4200"><span style="color:purple">+1.908.516.4200</span></a><span class="apple-converted-space"> </span>| UK<span class="apple-converted-space"> </span><a href="tel:%2B44.1704.322.300"><span style="color:purple">+44.1704.322.300</span></a><br>
>>>> Netherlands<span class="apple-converted-space"> </span><a href="tel:%2B31.208.200.622"><span style="color:purple">+31.208.200.622</span></a><span class="apple-converted-space"> </span>| 24/7 IP NOC Phone:<span class="apple-converted-space"> </span><a href="tel:%2B31.20.82.00.623"><span style="color:purple">+31.20.82.00.623</span></a><br>
>>>><span class="apple-converted-space"> </span><a href="mailto:jeroen.wunnink@hibernianetworks.com"><span style="color:purple">jeroen.wunnink@hibernianetworks.com</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:jeroen.wunnink@hibernianetworks.com"><span style="color:purple">jeroen.wunnink@hibernianetworks.com</span></a>><br>
>>>><span class="apple-converted-space"> </span><a href="http://www.hibernianetworks.com/" target="_blank"><span style="color:purple">www.hibernianetworks.com</span></a><span class="apple-converted-space"> </span><<a href="http://www.hibernianetworks.com/" target="_blank"><span style="color:purple">http://www.hibernianetworks.com/</span></a>><br>
>>><br>
>>><br>
>>> --<br>
>>><br>
>>> Jeroen Wunnink<br>
>>> IP NOC Manager - Hibernia Networks<br>
>>> Main numbers (Ext: 1011): USA<span class="apple-converted-space"> </span><a href="tel:%2B1.908.516.4200"><span style="color:purple">+1.908.516.4200</span></a><span class="apple-converted-space"> </span>| UK<span class="apple-converted-space"> </span><a href="tel:%2B44.1704.322.300"><span style="color:purple">+44.1704.322.300</span></a><br>
>>> Netherlands<span class="apple-converted-space"> </span><a href="tel:%2B31.208.200.622"><span style="color:purple">+31.208.200.622</span></a><span class="apple-converted-space"> </span>| 24/7 IP NOC Phone:<span class="apple-converted-space"> </span><a href="tel:%2B31.20.82.00.623"><span style="color:purple">+31.20.82.00.623</span></a><br>
>>><span class="apple-converted-space"> </span><a href="mailto:jeroen.wunnink@hibernianetworks.com"><span style="color:purple">jeroen.wunnink@hibernianetworks.com</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:jeroen.wunnink@hibernianetworks.com"><span style="color:purple">jeroen.wunnink@hibernianetworks.com</span></a>><br>
>>><span class="apple-converted-space"> </span><a href="http://www.hibernianetworks.com/" target="_blank"><span style="color:purple">www.hibernianetworks.com</span></a><span class="apple-converted-space"> </span><<a href="http://www.hibernianetworks.com/" target="_blank"><span style="color:purple">http://www.hibernianetworks.com/</span></a>>_______________________________________________<br>
>>> foundry-nsp mailing list<br>
>>><span class="apple-converted-space"> </span><a href="mailto:foundry-nsp@puck.nether.net"><span style="color:purple">foundry-nsp@puck.nether.net</span></a><span class="apple-converted-space"> </span><mailto:<a href="mailto:foundry-nsp@puck.nether.net"><span style="color:purple">foundry-nsp@puck.nether.net</span></a>><br>
>>><span class="apple-converted-space"> </span><a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank"><span style="color:purple">http://puck.nether.net/mailman/listinfo/foundry-nsp</span></a><span class="apple-converted-space"> </span><<a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank"><span style="color:purple">http://puck.nether.net/mailman/listinfo/foundry-nsp</span></a>><br>
>><br>
><br>
> . . . . . . . . . . . . . . . . . . . . . . . . .<br>
> Jethro R Binks, Network Manager,<br>
> Information Services Directorate, University Of Strathclyde, Glasgow, UK<br>
><br>
> The University of Strathclyde is a charitable body, registered in<br>
> Scotland, number SC015263.<br>
<br>
<br>
_______________________________________________<br>
foundry-nsp mailing list<br>
<a href="mailto:foundry-nsp@puck.nether.net"><span style="color:purple">foundry-nsp@puck.nether.net</span></a><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><a href="http://puck.nether.net/mailman/listinfo/foundry-nsp" target="_blank"><span style="color:purple">http://puck.nether.net/mailman/listinfo/foundry-nsp</span></a><o:p></o:p></p>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>foundry-nsp mailing list<o:p></o:p></pre>
<pre><a href="mailto:foundry-nsp@puck.nether.net">foundry-nsp@puck.nether.net</a><o:p></o:p></pre>
<pre><a href="http://puck.nether.net/mailman/listinfo/foundry-nsp">http://puck.nether.net/mailman/listinfo/foundry-nsp</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Jeroen Wunnink<o:p></o:p></pre>
<pre>IP NOC Manager - Hibernia Networks<o:p></o:p></pre>
<pre>Main numbers (Ext: 1011): USA +1.908.516.4200 | UK +44.1704.322.300 <o:p></o:p></pre>
<pre>Netherlands +31.208.200.622 | 24/7 IP NOC Phone: +31.20.82.00.623<o:p></o:p></pre>
<pre><a href="mailto:jeroen.wunnink@hibernianetworks.com">jeroen.wunnink@hibernianetworks.com</a><o:p></o:p></pre>
<pre><a href="http://www.hibernianetworks.com">www.hibernianetworks.com</a><o:p></o:p></pre>
</div>
</body>
</html>