[j-nsp] QFX10002 Inline Flow

[Milton Ngan]

One of the other issues we ran across was XL710/X710 NIC wasn't being initialized properly in the hypervisor and required a full power cycle to correct it after the upgrade.


This is what you see on a non-functional system


root at bg2-okc1:RE:0% vhclient -sroot at bg2-okc1:~# lspci -vvt
-[0000:00]-+-00.0  Intel Corporation 3rd Gen Core processor DRAM Controller
           +-01.0-[01-06]--+-00.0  Intel Corporation DH89XXCC Series QAT
           |               +-00.1  Intel Corporation DH8900CC Series Gigabit Fiber Network Connection
           |               +-00.2  Intel Corporation DH8900CC Series Gigabit Fiber Network Connection
           |               \-00.3  Intel Corporation DH8900CC Series Gigabit Fiber Network Connection
           +-01.1-[30-32]--
           +-06.0-[08-11]----00.0-[09-11]--+-01.0-[0a]--
           |                               +-02.0-[0b]--
           |                               +-05.0-[0c]----00.0  Intel Corporation Device 005e
           |                               +-07.0-[0d]--
           |                               +-09.0-[0e]----00.0  Intel Corporation Device 005f
           |                               +-0a.0-[0f]----00.0  Intel Corporation Device 005e
           |                               +-0c.0-[10]----00.0  Intel Corporation Device 005e
           |                               \-0e.0-[11]--
           +-1c.0-[12-17]----00.0  Juniper Networks Device 0076
           +-1c.1-[18-1d]----00.0  Juniper Networks Device 0077
           +-1c.2-[1e-23]--+-00.0  Broadcom Corporation Device b041
           |               \-00.1  Broadcom Corporation Device b041
           +-1c.3-[24-29]----00.0  Xilinx Corporation Device 0505
           +-1d.0  Intel Corporation DH89xxCC USB2 Enhanced Host Controller #1
           +-1f.0  Intel Corporation DH89xxCC LPC Controller
           +-1f.2  Intel Corporation DH89xxCC 4 Port SATA AHCI Controller
           +-1f.3  Intel Corporation DH89xxCC SMBus Controller
           \-1f.7  Intel Corporation DH89xxCC Watchdog Timer


And this is what you see on a functional one. Notice the presence of the


root at bg2-waw1:RE:0% vhclient -sroot at bg2-waw1:~# lspci -vvt
-[0000:00]-+-00.0  Intel Corporation 3rd Gen Core processor DRAM Controller
           +-01.0-[01-06]--+-00.0  Intel Corporation DH89XXCC Series QAT
           |               +-00.1  Intel Corporation DH8900CC Series Gigabit Fiber Network Connection
           |               +-00.2  Intel Corporation DH8900CC Series Gigabit Fiber Network Connection
           |               \-00.3  Intel Corporation DH8900CC Series Gigabit Fiber Network Connection
           +-01.1-[30-32]--+-[0000:31]-+-02.0  Intel Corporation XL710/X710 Virtual Function
           |               |           +-02.1  Intel Corporation XL710/X710 Virtual Function
           |               |           +-02.2  Intel Corporation XL710/X710 Virtual Function
           |               |           +-02.3  Intel Corporation XL710/X710 Virtual Function
           |               |           +-0a.0  Intel Corporation XL710/X710 Virtual Function
           |               |           +-0a.1  Intel Corporation XL710/X710 Virtual Function
           |               |           +-0a.2  Intel Corporation XL710/X710 Virtual Function
           |               |           \-0a.3  Intel Corporation XL710/X710 Virtual Function
           |               \-[0000:30]-+-00.0  Intel Corporation Ethernet Controller X710 for 10GbE backplane
           |                           \-00.1  Intel Corporation Ethernet Controller X710 for 10GbE backplane
           +-06.0-[08-11]----00.0-[09-11]--+-01.0-[0a]--
           |                               +-02.0-[0b]--
           |                               +-05.0-[0c]----00.0  Intel Corporation Device 005e
           |                               +-07.0-[0d]--
           |                               +-09.0-[0e]----00.0  Intel Corporation Device 005f
           |                               +-0a.0-[0f]----00.0  Intel Corporation Device 005e
           |                               +-0c.0-[10]----00.0  Intel Corporation Device 005e
           |                               \-0e.0-[11]--
           +-1c.0-[12-17]----00.0  Juniper Networks Device 0076
           +-1c.1-[18-1d]----00.0  Juniper Networks Device 0077
           +-1c.2-[1e-23]--+-00.0  Broadcom Corporation Device b041
           |               \-00.1  Broadcom Corporation Device b041
           +-1c.3-[24-29]----00.0  Xilinx Corporation Device 0505
           +-1d.0  Intel Corporation DH89xxCC USB2 Enhanced Host Controller #1
           +-1f.0  Intel Corporation DH89xxCC LPC Controller
           +-1f.2  Intel Corporation DH89xxCC 4 Port SATA AHCI Controller
           +-1f.3  Intel Corporation DH89xxCC SMBus Controller
           \-1f.7  Intel Corporation DH89xxCC Watchdog Timer


________________________________
From: juniper-nsp <juniper-nsp-bounces at puck.nether.net> on behalf of Nikolas Geyer <nik at neko.id.au>
Sent: Thursday, December 3, 2020 5:18:04 AM
To: Brendan Mannella; juniper-nsp at puck.nether.net
Subject: Re: [j-nsp] QFX10002 Inline Flow

What version did you upgrade from? Check out https://lkhill.com/juniper-qfx10k-ipfix/ as there were some things changed in Junos 17 that resulted in broken IPFIX.

Sent from my iPhone

On Dec 1, 2020, at 9:51 PM, Brendan Mannella <bmannella at teraswitch.com> wrote:

Curious if anyone else has completely broken Inline flow on QFX10002 in any
of the recent recommended versions. It was running fine with the current
configuration, then we upgraded two different sets and both ended up with
broken flow.

We are running --- JUNOS 19.1R3-S3.2 Kernel 64-bit and --- JUNOS 20.2R2.11
Kernel 64-bit

Is anyone else seeing this?
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp at puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp