[c-nsp] cisco VPC problem on nx 3064
BASSAGET Cédric
cedric.bassaget.ml at gmail.com
Wed Oct 9 10:13:59 EDT 2019
Hello,
I'm facing something I can't explain and I don't understand what is the
problem. I have a lab setup with two n3k switches and 2 linux hosts. If
somebody can tell me if there's something wrong with my setup, it would be
very helpfull.
Here's my setup :
on each n3k :
vpc domain 1, consistency OK, peer-link port-channel 4091, keepalive-link
port-channel 4090
int eth1/13 = port-channel 13 = vpc 13
int eth1/14 = port-channel 14 = vpc 14
on each linux host : iface bond1, mode 802.3ad
n3k-1 iface enp130s0f1 <-> n3k-1 eth1/13
n3k-1 iface enp130s0f0 <-> n3k-2 eth1/13
n3k-2 iface enp130s0f1 <-> n3k-1 eth1/14
n3k-2 iface enp130s0f0 <-> n3k-2 eth1/14
test : ping linux1 (3c:fd:fe:a7:9c:58) -> linux2 (3c:fd:fe:10:02:a0)
If I unplug one of the two n3k, everything continue to work as expected.
if I unplug n3k-1 eth1/13 : linux hosts can't see each other.
### before unplugging n3k-1 eth1/13 :
################ n3k-1 : mac-address table vlan 40 :
+ 40 3cfd.fe10.02a0 dynamic 0 F F Po14
+ 40 3cfd.fea7.9c58 dynamic 0 F F Po13
################ n3k-2 : mac-address table vlan 40 :
+ 40 3cfd.fe10.02a0 dynamic 0 F F Po14
+ 40 3cfd.fea7.9c58 dynamic 0 F F Po13
### after unplugging n3k-1 eth1/13
################ n3k-1 : sh mac-address table vlan 40
+ 40 3cfd.fe10.02a0 dynamic 0 F F Po14
40 3cfd.fea7.9c58 dynamic 0 F F Po4091
################ n3k-2 : sh mac-address table vlan 40
+ 40 3cfd.fe10.02a0 dynamic 0 F F Po14
+ 40 3cfd.fea7.9c58 dynamic 0 F F Po13
ICMP echo request :
linux1 (3c:fd:fe:a7:9c:58) -> n3k-2 eth1/13 -> n3k-2 eth1/14 -> linux2
(3c:fd:fe:10:02:a0)
ICMP echo reply :
linux2 (3c:fd:fe:10:02:a0) -> n3k-1 eth1/14 -> ????
ICMP echo reply is sent on the interface which is still UP on n3k-1. I
guess n3k-1 sends it to n3k-2 via VPC peer-link (Po4091). Then I don't know
what happens, but ICMP reply is never received by linux1 host.
################################
################ Configs
################################
################ n3k-1 :
vpc domain 1
peer-switch
role priority 32000
peer-keepalive destination 10.10.10.2 source 10.10.10.1 vrf vpc_keepalive
delay restore 250
auto-recovery reload-delay 300
interface port-channel4090
no switchport
vrf member vpc_keepalive
ip address 10.10.10.1/30
interface port-channel4091
switchport mode trunk
spanning-tree port type network
vpc peer-link
interface port-channel13
switchport mode trunk
switchport trunk allowed vlan 40
vpc 13
interface port-channel14
switchport mode trunk
switchport trunk allowed vlan 40
vpc 14
interface Ethernet1/13
switchport mode trunk
switchport trunk allowed vlan 40
channel-group 13 mode active
no shutdown
interface Ethernet1/14
switchport mode trunk
switchport trunk allowed vlan 40
channel-group 14 mode active
no shutdown
################ n3k-2 :
vpc domain 1
peer-switch
role priority 33000
peer-keepalive destination 10.10.10.1 source 10.10.10.2 vrf vpc_keepalive
delay restore 250
auto-recovery reload-delay 300
interface port-channel4090
no switchport
vrf member vpc_keepalive
ip address 10.10.10.2/30
interface port-channel4091
switchport mode trunk
spanning-tree port type network
vpc peer-link
interface port-channel13
switchport mode trunk
switchport trunk allowed vlan 40
vpc 13
interface port-channel14
switchport mode trunk
switchport trunk allowed vlan 40
vpc 14
interface Ethernet1/13
switchport mode trunk
switchport trunk allowed vlan 40
channel-group 13 mode active
no shutdown
interface Ethernet1/14
switchport mode trunk
switchport trunk allowed vlan 40
channel-group 14 mode active
no shutdown
################ linux 1 & linux 2 :
#cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
802.3ad info
LACP rate: slow
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 3c:fd:fe:a7:9c:58
Active Aggregator Info:
Aggregator ID: 2
Number of ports: 2
Actor Key: 15
Partner Key: 32781
Partner Mac Address: 00:23:04:ee:be:01
Slave Interface: enp130s0f0
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 2
Permanent HW addr: 3c:fd:fe:a7:9c:58
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: 3c:fd:fe:a7:9c:58
port key: 15
port priority: 255
port number: 1
port state: 61
details partner lacp pdu:
system priority: 32667
system mac address: 00:23:04:ee:be:01
oper key: 32781
port priority: 32768
port number: 269
port state: 61
Slave Interface: enp130s0f1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 16
Permanent HW addr: 3c:fd:fe:a7:9c:59
Slave queue ID: 0
Aggregator ID: 2
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
system priority: 65535
system mac address: 3c:fd:fe:a7:9c:58
port key: 15
port priority: 255
port number: 2
port state: 61
details partner lacp pdu:
system priority: 32667
system mac address: 00:23:04:ee:be:01
oper key: 32781
port priority: 32768
port number: 16653
port state: 61
################ linux 1 :
# ifconfig bond1.40
bond1.40: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet x.x.x.x netmask 255.255.255.224 broadcast x.x.x.x
inet6 fe80::3efd:feff:fea7:9c58 prefixlen 64 scopeid 0x20<link>
ether 3c:fd:fe:a7:9c:58 txqueuelen 1000 (Ethernet)
################ linux 2 :
# ifconfig bond1.40
bond1.40: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet x.x.x.x netmask 255.255.255.224 broadcast x.x.x.x
inet6 fe80::3efd:feff:fe10:2a0 prefixlen 64 scopeid 0x20<link>
ether 3c:fd:fe:10:02:a0 txqueuelen 1000 (Ethernet)
Thanks for your help.
Regards,
Cédric
More information about the cisco-nsp
mailing list