[j-nsp] Link establishment issues with 1Gbps SX/LX SFPs on QFX5110

[Timothy Creswick]

For anyone interested in this, JTAC released to us 3.5R1-S4.1 for the satellite devices (I don't believe this will be generally available until tomorrow). This addresses the issue and confirms that there were 2 or 3 different PRs relating to the Broadcom chipset being incorrectly programmed by Junos.

Some output, testing with a variety of 1Gbps optics (coded SX, LX, uncoded BX, LX and SX).

> show chassis hardware satellite
Hardware inventory:
Item             Version  Part number  Serial number     Description
FPC 100          REV 27   650-061152   WS37183----X      QFX5110-48S-4C
  PIC 0          REV 27   650-061152   WS37183----X      48x10G-4x100G
    Xcvr 0       850nm    740-011782   F----------X      SFP-SX
    Xcvr 2       1310nm   740-011783   F----------X      SFP-LX10
    Xcvr 4                NON-JNPR     VS51208----X      SFP-1000BASE-BX10-D
    Xcvr 6                NON-JNPR     VS60930----X      SFP-LX10
    Xcvr 8                NON-JNPR     VS50930----X      SFP-SX

> show interfaces ge-100/0/[02468] terse
Interface               Admin Link Proto    Local                 Remote
ge-100/0/0              up    up
ge-100/0/2              up    up
ge-100/0/4              up    up
ge-100/0/6              up    up
ge-100/0/8              up    up

> show configuration interfaces
ge-100/0/0 {
    unit 0;
}
ge-100/0/2 {
    unit 0;
}
ge-100/0/4 {
    unit 0;
}
ge-100/0/6 {
    unit 0;
}
ge-100/0/8 {
    unit 0;
}

# bcmshell ps 1,3,5,7,9

HW (unit 0)
                 ena/    speed/ link auto    STP                  lrn  inter   max  loop
           port  link    duplex scan neg?   state   pause  discrd ops   face frame  back
       ge0(  1)  up      1G  FD   HW  Yes  Forward          None    F   GMII  1518
       ge1(  3)  up      1G  FD   HW  Yes  Forward          None    F   GMII  1518
       ge2(  5)  up      1G  FD   HW  Yes  Forward          None    F   GMII  1518
       ge3(  7)  up      1G  FD   HW  Yes  Forward          None    F   GMII  1518
       ge4(  9)  up      1G  FD   HW  Yes  Forward          None    F   GMII  1518

We note that now changing the auto-negotiate state in Junos correctly updates the underlying chipset and link negotiation works correctly. Interface is shown as GMII which we understand to be correct also.

All-in, pretty terrible that - as far as we can tell - 1Gbps ports have *never worked* on QFX5110 properly in all the four basic combinations (i.e. SX and LX at both auto- and no-auto negotiation) until this release. Quite how Juniper have been shipping the hardware for so long in that state is a mystery to me.


Tim