[c-nsp] 10gig Link Bouncing Consistently

[Justin M. Streiner]

On Tue, 2 Apr 2013, Justin Krejci wrote:

1. What types of optics are being used on these links?
2. What type of multimode fiber (OM3/OM4?  hopefully not OM1...) is being 
used?
3. What is the distance between the devices, in terms of cable feet?

jms

> We've a customer with two separate 10gig links to a pair of our
> 6509-E's. Both of our 6509-E's have dual Sup720-3BXL and the 6704-10GE
> line cards.
>
> Link 1: 6509 Router#1 routed port to Netgear#1. IPv4 only. Multimode
> fiber
> Link 2: 6509 Router#2 routed port to Netgear#2. IPv4 only. Multimode
> fiber
> The two Netgears are in a virtual stack. The 6509s are not.
> Netgear model: GSM7352Sv2
>
> Beginning several days ago both links began flapping every 5 and a half
> hours, both links have this happen within a couple of minutes of each
> other.
> They will flap for several minutes and then return to normal operation
> requiring no user action and the ports never enter err-disable or show
> any other symptoms.
> Our Ciscos log between 5 and 10 interface drops in the time frame.
> The Netgear switches log hundreds of interface drops in the same time
> frame.
>
>
> Further Details:
>
> We've replaced the transceivers on all endpoints.
> Light power levels are great at all end points.
> All other 10gig ports on the Ciscos are operating without issue, though
> they are all using single mode/LR optics.
> No other related log entries or other observed/reported issues anywhere
> else.
> On the Cisco's there are a some old interface errors but the error
> counters have not increased over the past several days.
> On the Netgears there are a few thousand FCS errors during the the few
> minutes of the problem occurring otherwise no errors ever.
> Sup720's are not failing over
>
> Beyond the very basic things the Ciscos are also doing OSPF and full BGP
> tables. IPv4 and IPv6 is in use. HSRP used in a few places, including
> these particular 10gig ports.
> Management is in a VRF.
>
> The peculiar thing is the regularity of 5.5 hour cycle which is obvious
> in the logs and in the traffic graphs.
>
> Both cisco ports are identically configured except hsrp priority and ip
> address.
>
> interface TenGigabitEthernet9/2
> ip address XXXX YYYY
> ip flow ingress
> ip flow egress
> standby 9 ip XXX
> standby 9 priority 105
> standby 9 preempt delay minimum 120
> standby 9 authentication md5 key-string 7 XXXX
> standby 9 track 1 decrement 20
> no cdp enable
>
>
> Is there some known issue or any suggestions at all at this point?
> Compatibility issues?
> An IOS upgrade seems to be another thing to try but in reading through
> the IOS SXI and SXJ release notes I don't see anything obviously
> related.
> Or maybe even just a reboot might help.
>
>
> Thanks for any advice.
>
> _______________________________________________
> cisco-nsp mailing list  cisco-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
>