[c-nsp] Stange issue on 100 Gbs interconnection Juniper - Cisco
james list
jameslist72 at gmail.com
Sun Feb 11 07:49:55 EST 2024
Hi
there are no errors on both interfaces (Cisco and Juniper).
here following logs of one event on both side, config and LACP stats.
LOGS of one event time 16:39:
CISCO
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN:
Interface port-channel101 is down (No operational members)
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_DOWN_PARENT_DOWN: Interface
port-channel101.2303 is down (Parent interface is down)
2024 Feb 9 16:39:36 NEXUS1 %BGP-5-ADJCHANGE: bgp- [xxx] (xxx) neighbor
172.16.6.17 Down - sent: other configuration change
2024 Feb 9 16:39:36 NEXUS1 %ETH_PORT_CHANNEL-5-FOP_CHANGED:
port-channel101: first operational port changed from Ethernet1/44 to none
2024 Feb 9 16:39:36 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_DOWN: port-channel101:
Ethernet1/44 is down
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface
port-channel101,bandwidth changed to 100000 Kbit
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_DOWN_INITIALIZING: Interface
Ethernet1/44 is down (Initializing)
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN:
Interface port-channel101 is down (No operational members)
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-SPEED: Interface port-channel101,
operational speed changed to 100 Gbps
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_DUPLEX: Interface
port-channel101, operational duplex mode changed to Full
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_RX_FLOW_CONTROL: Interface
port-channel101, operational Receive Flow Control state changed to off
2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_TX_FLOW_CONTROL: Interface
port-channel101, operational Transmit Flow Control state changed to off
2024 Feb 9 16:39:39 NEXUS1 %ETH_PORT_CHANNEL-5-PORT_UP: port-channel101:
Ethernet1/44 is up
2024 Feb 9 16:39:39 NEXUS1 %ETH_PORT_CHANNEL-5-FOP_CHANGED:
port-channel101: first operational port changed from none to Ethernet1/44
2024 Feb 9 16:39:39 NEXUS1 %ETHPORT-5-IF_BANDWIDTH_CHANGE: Interface
port-channel101,bandwidth changed to 100000000 Kbit
2024 Feb 9 16:39:39 NEXUS1 %ETHPORT-5-IF_UP: Interface Ethernet1/44 is up
in Layer3
2024 Feb 9 16:39:39 NEXUS1 %ETHPORT-5-IF_UP: Interface port-channel101 is
up in Layer3
2024 Feb 9 16:39:39 NEXUS1 %ETHPORT-5-IF_UP: Interface
port-channel101.2303 is up in Layer3
2024 Feb 9 16:39:43 NEXUS1 %BGP-5-ADJCHANGE: bgp- [xxx] (xxx) neighbor
172.16.6.17 Up
Feb 9 16:39:35.813 2024 MX1 lacpd[31632]: LACPD_TIMEOUT: et-0/1/5: lacp
current while timer expired current Receive State: CURRENT
Feb 9 16:39:35.813 2024 MX1 lacpd[31632]: LACP_INTF_DOWN: ae49: Interface
marked down due to lacp timeout on member et-0/1/5
Feb 9 16:39:35.819 2024 MX1 kernel: lag_bundlestate_ifd_change: bundle
ae49: bundle IFD minimum bandwidth or minimum links not met, Bandwidth
(Current : Required) 0 : 100000000000 Number of links (Current : Required)
0 : 1
Feb 9 16:39:35.815 2024 MX1 lacpd[31632]: LACP_INTF_MUX_STATE_CHANGED:
ae49: et-0/1/5: Lacp state changed from COLLECTING_DISTRIBUTING to
ATTACHED, actor port state : |EXP|-|-|-|IN_SYNC|AGG|SHORT|ACT|, partner
port state : |-|-|DIS|COL|OUT_OF_SYNC|AGG|SHORT|ACT|
Feb 9 16:39:35.869 2024 MX1 rpd[31866]: bgp_ifachange_group:10697:
NOTIFICATION sent to 172.16.6.18 (External AS xxx): code 6 (Cease) subcode
6 (Other Configuration Change), Reason: Interface change for the peer-group
Feb 9 16:39:35.909 2024 MX1 mib2d[31909]: SNMP_TRAP_LINK_DOWN: ifIndex
684, ifAdminStatus up(1), ifOperStatus down(2), ifName ae49
Feb 9 16:39:36.083 2024 MX1 lacpd[31632]: LACP_INTF_MUX_STATE_CHANGED:
ae49: et-0/1/5: Lacp state changed from ATTACHED to
COLLECTING_DISTRIBUTING, actor port state :
|-|-|DIS|COL|IN_SYNC|AGG|SHORT|ACT|, partner port state :
|-|-|DIS|COL|IN_SYNC|AGG|SHORT|ACT|
Feb 9 16:39:36.089 2024 MX1 kernel: lag_bundlestate_ifd_change: bundle
ae49 is now Up. uplinks 1 >= min_links 1
Feb 9 16:39:36.089 2024 MX1 kernel: lag_bundlestate_ifd_change: bundle
ae49: bundle IFD minimum bandwidth or minimum links not met, Bandwidth
(Current : Required) 0 : 100000000000 Number of links (Current : Required)
0 : 1
Feb 9 16:39:36.085 2024 MX1 lacpd[31632]: LACP_INTF_MUX_STATE_CHANGED:
ae49: et-0/1/5: Lacp state changed from COLLECTING_DISTRIBUTING to
ATTACHED, actor port state : |-|-|-|-|IN_SYNC|AGG|SHORT|ACT|, partner port
state : |-|-|-|-|OUT_OF_SYNC|AGG|SHORT|ACT|
Feb 9 16:39:39.095 2024 MX1 lacpd[31632]: LACP_INTF_MUX_STATE_CHANGED:
ae49: et-0/1/5: Lacp state changed from ATTACHED to
COLLECTING_DISTRIBUTING, actor port state :
|-|-|DIS|COL|IN_SYNC|AGG|SHORT|ACT|, partner port state :
|-|-|-|-|IN_SYNC|AGG|SHORT|ACT|
Feb 9 16:39:39.101 2024 MX1 kernel: lag_bundlestate_ifd_change: bundle
ae49 is now Up. uplinks 1 >= min_links 1
Feb 9 16:39:39.109 2024 MX1 mib2d[31909]: SNMP_TRAP_LINK_UP: ifIndex 684,
ifAdminStatus up(1), ifOperStatus up(1), ifName ae49
Feb 9 16:39:41.190 2024 MX1 rpd[31866]: bgp_recv: read from peer
172.16.6.18 (External AS xxx) failed: Unknown error: 48110976
CONFIG:
CISCO
NEXUS1# sh run int port-channel 101
interface port-channel101
description <[To MX1|Et-0/1/5]>
mtu 9216
no ip redirects
NEXUS01# sh run int port-channel 101.2303
interface port-channel101.2303
description <[To MX1|Et-0/1/5]>
mtu 9216
encapsulation dot1q 2303
vrf member SIA
bfd ipv4 interval 250 min_rx 250 multiplier 3
no ip redirects
ip address 172.16.6.18/30
no shutdown
JUNIPER
MX1> show configuration interfaces ae49
description "link to NEXUS01";
flexible-vlan-tagging;
mtu 9192;
encapsulation flexible-ethernet-services;
aggregated-ether-options {
lacp {
active;
periodic fast;
}
}
unit 2303 {
vlan-id 2303;
family inet {
mtu 1500;
address 172.16.6.17/30;
}
}
LACP counters:
CISCO
NEXUS01# sh lacp counters
NOTE: Clear lacp counters to get accurate statistics
------------------------------------------------------------------------------
LACPDUs Markers/Resp
LACPDUs
Port Sent Recv Recv Sent Pkts
Err
------------------------------------------------------------------------------
port-channel101
Ethernet1/44 6123011 6118981 0 0
0
NEXUS1# sh lacp interface eth1/44
Interface Ethernet1/44 is up
Channel group is 101 port channel is Po101
PDUs sent: 6123014
PDUs rcvd: 6118984
Markers sent: 0
Markers rcvd: 0
Marker response sent: 0
Marker response rcvd: 0
Unknown packets rcvd: 0
Illegal packets rcvd: 0
Lag Id: [ [(7f, c4-9-b7-64-30-38, 32, 7f, 18), (8000, b0-8b-cf-83-49-5b,
64, 8000, 1ad)] ]
Operational as aggregated link since Fri Feb 9 16:39:39 2024
Local Port: Eth1/44 MAC Address= b0-8b-cf-83-49-5b
System Identifier=0x8000, Port Identifier=0x8000,0x1ad
Operational key=100
LACP_Activity=active
LACP_Timeout=Short Timeout (1s)
Synchronization=IN_SYNC
Collecting=true
Distributing=true
Partner information refresh timeout=Short Timeout (3s)
Actor Admin State=63
Actor Oper State=63
Neighbor: 0x18
MAC Address= c4-9-b7-64-30-38
System Identifier=0x7f, Port Identifier=0x7f,0x18
Operational key=50
LACP_Activity=active
LACP_Timeout=short Timeout (1s)
Synchronization=IN_SYNC
Collecting=true
Distributing=true
Partner Admin State=63
Partner Oper State=63
Aggregate or Individual(True=1)= 1
JUNIPER
MX1> show lacp interfaces ae49 extensive
Aggregated interface: ae49
LACP state: Role Exp Def Dist Col Syn Aggr Timeout
Activity
et-0/1/5 Actor No No Yes Yes Yes Yes Fast
Active
et-0/1/5 Partner No No Yes Yes Yes Yes Fast
Active
LACP protocol: Receive State Transmit State Mux State
et-0/1/5 Current Fast periodic Collecting
distributing
LACP info: Role System System Port
Port Port
priority identifier priority
number key
et-0/1/5 Actor 127 c4:09:b7:64:30:38 127
24 50
et-0/1/5 Partner 32768 b0:8b:cf:83:49:5b 32768
429 100
Il giorno dom 11 feb 2024 alle ore 13:07 Gert Doering <gert at greenie.muc.de>
ha scritto:
> HI,
>
> On Sun, Feb 11, 2024 at 12:50:32PM +0100, james list wrote:
> > 2024 Feb 9 16:39:36 NEXUS1 %ETHPORT-5-IF_DOWN_PORT_CHANNEL_MEMBERS_DOWN:
> > Interface port-channel101 is down (No operational members)
>
> So there is no *BGP* problem here, but a lower layer issue.
>
> Let me repeat that part about "error counters on the interface"...
>
> gert
> --
> "If was one thing all people took for granted, was conviction that if you
> feed honest figures into a computer, honest figures come out. Never
> doubted
> it myself till I met a computer with a sense of humor."
> Robert A. Heinlein, The Moon is a Harsh
> Mistress
>
> Gert Doering - Munich, Germany
> gert at greenie.muc.de
>
More information about the cisco-nsp
mailing list