<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I had a recent issue that was similar to this. In that case it was a DDOS signature update on a specific vendors' DDOS scrubber at the host site that was the problem. The specific tunnel src/dst flow would be dropped, however all other traffic between endpoints
was allowed. Because all the traffic for that specific flow hit a threshold above the rule and the IP's in question were flagged low enough in reputation to be fully inspected, it was blocked. After whitelisting the IP's in the DDOS solution, all VPN traffic
worked fine.<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Just a thought.<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Outages <outages-bounces@outages.org> on behalf of Biddle, Josh via Outages <outages@outages.org><br>
<b>Sent:</b> Monday, October 21, 2019 12:39 PM<br>
<b>To:</b> Marty Adkins <marty@martyadkins.com>; outages@outages.org <outages@outages.org><br>
<b>Cc:</b> D L <route2null0@yahoo.com>; Cullis, Ben <BCullis@ntst.com>; Cochran, Brian <BCochran@ntst.com><br>
<b>Subject:</b> Re: [outages] VPN issues over Spectrum L3 boundaries</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.x_MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
p.x_msonormal0, li.x_msonormal0, div.x_msonormal0
{margin-right:0in;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
span.x_EmailStyle18
{font-family:"Calibri",sans-serif;
color:windowtext}
span.x_EmailStyle19
{font-family:"Calibri",sans-serif;
color:windowtext}
span.x_st4cursor
{}
span.x_EmailStyle21
{font-family:"Calibri",sans-serif;
color:windowtext}
span.x_EmailStyle22
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
div.x_WordSection1
{}
-->
</style>
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="x_WordSection1">
<p class="x_MsoNormal">Our issues have recently magically resolved (last Thursday 10/17) due to L3 vanishing from the hop list. Our traffic now disappears into ntt.net and we are seeing two way IPsec traffic without any issues.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">Marty, did your issues resolve?</p>
<p class="x_MsoNormal"></p>
<p class="x_MsoNormal"><img width="617" height="386" id="x_Picture_x0020_1" style="width:6.427in; height:4.0208in" data-outlook-trace="F:1|T:1" src="cid:image001.png@01D58815.06155610"></p>
<p class="x_MsoNormal"> </p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b>From:</b> Marty Adkins <marty@martyadkins.com> <br>
<b>Sent:</b> Friday, October 18, 2019 4:01 PM<br>
<b>To:</b> Biddle, Josh <JBiddle@ntst.com>; outages@outages.org<br>
<b>Subject:</b> Re: [outages] VPN issues over Spectrum L3 boundaries</p>
</div>
</div>
<p class="x_MsoNormal"> </p>
<div>
<p class="x_MsoNormal">A customer of mine has had the same issue with a TW-connected site in Ohio and another in PA. One VPN tunnel works fine, the other has one-way ISAKMP traffic to the other head-end, which connects to Level3. A traceroute shows the failing
path includes 66.109.7.162. The failing direction is from the PA/OH sites toward the L3 head end. Full-size pings work fine. It's the UDP/500 that vanishes. I.E., it has nothing to do with MTU.<br>
<br>
Both started having the issue around Aug 26. Mysteriously, every week to 10 days, the broken path will start working for a while. This is usually shortly after midnight EDT; they go back down 1-3 hours later and stay down. The log entries for the two sites
match within seconds.<br>
<br>
The customer's contract is with Comcast Business so it's been difficult to get to someone clueful about this symptom in TW.<br>
<br>
-Marty<br>
<br>
On 10/15/2019 8:04 AM, Biddle, Josh via Outages wrote:</p>
</div>
<blockquote style="margin-top:5.0pt; margin-bottom:5.0pt">
<p class="x_MsoNormal">Found a thread in the Spectrum forums talking about the issue finally – it was marked as resolved so I started a new one.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__forums.timewarnercable.com_t5_Connectivity_Traffic-2Dissues-2Dat-2D66-2D109-2D7-2D162_m-2Dp_164091-23M53497&d=DwMD-g&c=-7HNwxqfpkdcRXCW8HB54Q&r=svX1Si7sopSBMitBL3bFwQ&m=pXHvd5iI_J5DVYGMDKBSdUZl1iyilZUvR3oyvE3BJ0E&s=_90gCNY2Ln1XdqWtMNguWIRejhsdUxoLyvgTpo-R5jM&e=">https://forums.timewarnercable.com/t5/Connectivity/Traffic-issues-at-66-109-7-162/m-p/164091#M53497</a></p>
<p class="x_MsoNormal"> </p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_MsoNormal"><b>From:</b> Outages <a href="mailto:outages-bounces@outages.org">
<outages-bounces@outages.org></a> <b>On Behalf Of </b>Biddle, Josh via Outages<br>
<b>Sent:</b> Sunday, October 13, 2019 12:00 PM<br>
<b>To:</b> <a href="mailto:outages@outages.org">outages@outages.org</a><br>
<b>Subject:</b> [outages] VPN issues over Spectrum L3 boundaries</p>
</div>
</div>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">We have several offices over the Ohio and Pennsylvania area that are experiencing issues passing traffic over VPN tunnels (specifically, there is always a Spectrum >< Level 3 interconnect). It is a very strange issue. The VPN tunnel will
actually establish, and if you source your ping from inside the internal network across the VPN tunnel to the destination, the traffic gets there and replies, but the replies never make it back to the original sending point.</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">Anyone else experiencing any similar issues like this?</p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal">Best Regards,</p>
<p class="x_MsoNormal"><b><span style="font-size:10.0pt; font-family:"Arial",sans-serif; color:#005480">Josh</span></b></p>
<p class="x_MsoNormal"> </p>
<p class="x_MsoNormal"> </p>
</blockquote>
</div>
This email and its attachments may contain privileged and confidential information and/or protected health information (PHI) intended solely for the use of Netsmart Technologies and the recipient(s) named above. If you are not the recipient, or the employee
or agent responsible for delivering this message to the intended recipient, you are hereby notified that any review, dissemination, distribution, printing or copying of this email message and/or any attachments is strictly prohibited. If you have received
this transmission in error, please email compliance@NTST.com immediately and permanently delete this email and any attachments.
</div>
</body>
</html>