[j-nsp] [c-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
Saku Ytti
saku at ytti.fi
Sun Feb 11 08:10:38 EST 2024
Hey James,
You shared this off-list, I think it's sufficiently material to share.
2024 Feb 9 16:39:36 NEXUS1
%ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN: Interface
port-channel101 is down (No operational members)
2024 Feb 9 16:39:36 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_DOWN:
port-channel101: Ethernet1/44 is down
Feb 9 16:39:35.813 2024 MX1 lacpd[31632]: LACPD_TIMEOUT: et-0/1/5:
lacp current while timer expired current Receive State: CURRENT
Feb 9 16:39:35.813 2024 MX1 lacpd[31632]: LACP_INTF_DOWN: ae49:
Interface marked down due to lacp timeout on member et-0/1/5
We can't know the order of events here, due to no subsecond precision
enabled on Cisco end.
But if failure would start from interface down, it would take 3seconds
for Juniper to realise LACP failure. However we can see that it
happens in less than 1s, so we can determine the interface was not
down first, the first problem was Juniper not receiving 3 consecutive
LACP PDUs, 1s apart, prior to noticing any type of interface state
related problems.
Is this always the order of events? Does it always happen with Juniper
noticing problems receiving LACP PDU first?
On Sun, 11 Feb 2024 at 14:55, james list via juniper-nsp
<juniper-nsp at puck.nether.net> wrote:
>
> Hi
>
> 1) cable has been replaced with a brand new one, they said that to check an
> MPO 100 Gbs cable is not that easy
>
> 3) no errors reported on both side
>
> 2) here the output of cisco and juniper
>
> NEXUS1# sh interface eth1/44 transceiver details
> Ethernet1/44
> transceiver is present
> type is QSFP-100G-SR4
> name is CISCO-INNOLIGHT
> part number is TR-FC85S-NC3
> revision is 2C
> serial number is INL27050TVT
> nominal bitrate is 25500 MBit/sec
> Link length supported for 50/125um OM3 fiber is 70 m
> cisco id is 17
> cisco extended id number is 220
> cisco part number is 10-3142-03
> cisco product id is QSFP-100G-SR4-S
> cisco version id is V03
>
> Lane Number:1 Network Lane
> SFP Detail Diagnostics Information (internal calibration)
>
> ----------------------------------------------------------------------------
> Current Alarms Warnings
> Measurement High Low High Low
>
> ----------------------------------------------------------------------------
> Temperature 30.51 C 75.00 C -5.00 C 70.00 C 0.00 C
> Voltage 3.28 V 3.63 V 2.97 V 3.46 V 3.13 V
> Current 6.40 mA 12.45 mA 3.25 mA 12.45 mA 3.25
> mA
> Tx Power 0.98 dBm 5.39 dBm -12.44 dBm 2.39 dBm -8.41
> dBm
> Rx Power -1.60 dBm 5.39 dBm -14.31 dBm 2.39 dBm -10.31
> dBm
> Transmit Fault Count = 0
>
> ----------------------------------------------------------------------------
> Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning
>
> Lane Number:2 Network Lane
> SFP Detail Diagnostics Information (internal calibration)
>
> ----------------------------------------------------------------------------
> Current Alarms Warnings
> Measurement High Low High Low
>
> ----------------------------------------------------------------------------
> Temperature 30.51 C 75.00 C -5.00 C 70.00 C 0.00 C
> Voltage 3.28 V 3.63 V 2.97 V 3.46 V 3.13 V
> Current 6.40 mA 12.45 mA 3.25 mA 12.45 mA 3.25
> mA
> Tx Power 0.62 dBm 5.39 dBm -12.44 dBm 2.39 dBm -8.41
> dBm
> Rx Power -1.18 dBm 5.39 dBm -14.31 dBm 2.39 dBm -10.31
> dBm
> Transmit Fault Count = 0
>
> ----------------------------------------------------------------------------
> Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning
>
> Lane Number:3 Network Lane
> SFP Detail Diagnostics Information (internal calibration)
>
> ----------------------------------------------------------------------------
> Current Alarms Warnings
> Measurement High Low High Low
>
> ----------------------------------------------------------------------------
> Temperature 30.51 C 75.00 C -5.00 C 70.00 C 0.00 C
> Voltage 3.28 V 3.63 V 2.97 V 3.46 V 3.13 V
> Current 6.40 mA 12.45 mA 3.25 mA 12.45 mA 3.25
> mA
> Tx Power 0.87 dBm 5.39 dBm -12.44 dBm 2.39 dBm -8.41
> dBm
> Rx Power 0.01 dBm 5.39 dBm -14.31 dBm 2.39 dBm -10.31
> dBm
> Transmit Fault Count = 0
>
> ----------------------------------------------------------------------------
> Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning
>
> Lane Number:4 Network Lane
> SFP Detail Diagnostics Information (internal calibration)
>
> ----------------------------------------------------------------------------
> Current Alarms Warnings
> Measurement High Low High Low
>
> ----------------------------------------------------------------------------
> Temperature 30.51 C 75.00 C -5.00 C 70.00 C 0.00 C
> Voltage 3.28 V 3.63 V 2.97 V 3.46 V 3.13 V
> Current 6.40 mA 12.45 mA 3.25 mA 12.45 mA 3.25
> mA
> Tx Power 0.67 dBm 5.39 dBm -12.44 dBm 2.39 dBm -8.41
> dBm
> Rx Power 0.11 dBm 5.39 dBm -14.31 dBm 2.39 dBm -10.31
> dBm
> Transmit Fault Count = 0
>
> ----------------------------------------------------------------------------
> Note: ++ high-alarm; + high-warning; -- low-alarm; - low-warning
>
>
>
> MX1> show interfaces diagnostics optics et-1/0/5
> Physical interface: et-1/0/5
> Module temperature : 38 degrees C / 100 degrees
> F
> Module voltage : 3.2740 V
> Module temperature high alarm : Off
> Module temperature low alarm : Off
> Module temperature high warning : Off
> Module temperature low warning : Off
> Module voltage high alarm : Off
> Module voltage low alarm : Off
> Module voltage high warning : Off
> Module voltage low warning : Off
> Module temperature high alarm threshold : 78 degrees C / 172 degrees
> F
> Module temperature low alarm threshold : -5 degrees C / 23 degrees F
> Module temperature high warning threshold : 75 degrees C / 167 degrees
> F
> Module temperature low warning threshold : 0 degrees C / 32 degrees F
> Module voltage high alarm threshold : 3.6300 V
> Module voltage low alarm threshold : 2.9700 V
> Module voltage high warning threshold : 3.4640 V
> Module voltage low warning threshold : 3.1340 V
> Laser bias current high alarm threshold : 104.999 mA
> Laser bias current low alarm threshold : 7.999 mA
> Laser bias current high warning threshold : 104.999 mA
> Laser bias current low warning threshold : 9.999 mA
> Laser output power high alarm threshold : 5.3703 mW / 7.30 dBm
> Laser output power low alarm threshold : 0.0794 mW / -11.00 dBm
> Laser output power high warning threshold : 3.1623 mW / 5.00 dBm
> Laser output power low warning threshold : 0.1995 mW / -7.00 dBm
> Laser rx power high alarm threshold : 4.4668 mW / 6.50 dBm
> Laser rx power low alarm threshold : 0.0251 mW / -16.00 dBm
> Laser rx power high warning threshold : 3.5481 mW / 5.50 dBm
> Laser rx power low warning threshold : 0.0630 mW / -12.01 dBm
> Lane 0
> Laser bias current : 41.588 mA
> Laser output power : 1.702 mW / 2.31 dBm
> Laser receiver power : 1.102 mW / 0.42 dBm
> Laser bias current high alarm : Off
> Laser bias current low alarm : Off
> Laser bias current high warning : Off
> Laser bias current low warning : Off
> Laser receiver power high alarm : Off
> Laser receiver power low alarm : Off
> Laser receiver power high warning : Off
> Laser receiver power low warning : Off
> Tx loss of signal functionality alarm : Off
> Rx loss of signal alarm : Off
> Tx laser disabled alarm : Off
> Lane 1
> Laser bias current : 42.324 mA
> Laser output power : 1.376 mW / 1.39 dBm
> Laser receiver power : 2.001 mW / 3.01 dBm
> Laser bias current high alarm : Off
> Laser bias current low alarm : Off
> Laser bias current high warning : Off
> Laser bias current low warning : Off
> Laser receiver power high alarm : Off
> Laser receiver power low alarm : Off
> Laser receiver power high warning : Off
> Laser receiver power low warning : Off
> Tx loss of signal functionality alarm : Off
> Rx loss of signal alarm : Off
> Tx laser disabled alarm : Off
> Lane 2
> Laser bias current : 41.066 mA
> Laser output power : 1.659 mW / 2.20 dBm
> Laser receiver power : 1.328 mW / 1.23 dBm
> Laser bias current high alarm : Off
> Laser bias current low alarm : Off
> Laser bias current high warning : Off
> Laser bias current low warning : Off
> Laser receiver power high alarm : Off
> Laser receiver power low alarm : Off
> Laser receiver power high warning : Off
> Laser receiver power low warning : Off
> Tx loss of signal functionality alarm : Off
> Rx loss of signal alarm : Off
> Tx laser disabled alarm : Off
> Lane 3
> Laser bias current : 37.970 mA
> Laser output power : 1.304 mW / 1.15 dBm
> Laser receiver power : 1.370 mW / 1.37 dBm
> Laser bias current high alarm : Off
> Laser bias current low alarm : Off
> Laser bias current high warning : Off
> Laser bias current low warning : Off
> Laser receiver power high alarm : Off
> Laser receiver power low alarm : Off
> Laser receiver power high warning : Off
> Laser receiver power low warning : Off
> Tx loss of signal functionality alarm : Off
> Rx loss of signal alarm : Off
> Tx laser disabled alarm : Off
>
> Il giorno dom 11 feb 2024 alle ore 13:12 Havard Eidnes <he at uninett.no> ha
> scritto:
>
> > > DC technicians states cable are the same in both DCs and
> > > direct, no patch panel
> >
> > Things I would look at:
> >
> > * Has all the connectors been verified clean via microscope?
> >
> > * Optical levels relative to threshold values (may relate to the
> > first).
> >
> > * Any end seeing any input errors? (May relate to the above
> > two.) On the Juniper you can see some of this via PCS
> > ("Physical Coding Sublayer") unexpected events independently
> > of whether you have payload traffic, not sure you can do the
> > same on the Nexus boxes.
> >
> > Regards,
> >
> > - Håvard
> >
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
--
++ytti
More information about the juniper-nsp
mailing list