[j-nsp] 1000BASE-T SFP sets interface status "up" once inserted into EX4200-24F

Thu Nov 4 22:40:03 EDT 2010

On Fri, Nov 05, 2010 at 02:31:43AM +0200, Martin T wrote:
> I have few ProLabs GLC-T-C 1000BASE-T SFP's. If I insert this SFP into 
> EX4200-24F switch, both Admin and Link status are immediately "up". 
> Admin status should be "up" by default, but I find it odd, that Link 
> status stays "up" as well even if there is no cable connected to the 
> SFP.
> 
> When I insert the SFP, "show log messages" shows following 
> error/warning: Oct 9 17:42:36 chas[727]: IFCM: no handler for command 
> subtype 226
> 
> JUNOS version is 10.3R1.9 and switch model is EX4200-24F.
> 
> Any ideas, what might cause such odd behavior, where interface Link 
> status is set "up" even if there is no cable connected to the port? 

We've seen the same issue with Cisco branded copper SFPs in EX4200-24F, 
and have a case open on it. The latest update I heard from Juniper was 
that this behavior had been added intentionally (if you check older 9.x 
code it doesn't happen there), as part of an effort to start enforcing a 
"Juniper branded optics only" policy on the EX. I was going to wait for 
an official confirmation before I said anything publicly, as I'm still 
holding out a small shred of hope that Juniper isn't actually this 
stupid, but since you brought it up... :)

Ironically we don't see the issue on EX8200 48F linecards, even with the 
same code and optics, just the 4200-24F. You can also pop on the 
chassism interface and look at the phy, which clearly shows that link is 
detected as down, so I'm not sure what they're doing to end up thinking 
that the link is still up. From the shell as root:

root at router:RE:0% lcdd 0 chassism
Telnet escape character is '^D'.
Trying 128.0.0.16...
Connected to 128.0.0.16.
Escape character is '^D'.
chassism<0>#show phy ge-0/0/8
devNum:1, portNum:8
SFP-T PHY i2c device = 105
Copper GMII Control:                (0-0x00) 0x1140
Copper GMII Status:                 (0-0x01) 0x7949
Fiber GMII Control:                 (1-0x00) 0x1140
Fiber GMII Status:                  (1-0x01) 0x0169
PHY Identifier:                     (0-0x02) 0x0141
PHY Identifier:                     (0-0x03) 0x0cc2
Copper GMII Autoneg Advertise:      (0-0x04) 0x01e1
Copper GMII Link Partner Ability:   (0-0x05) 0x0000
Copper GMII Autoneg Expansion:      (0-0x06) 0x0004
Copper GMII Next Page Tx:           (0-0x07) 0x2001
Copper GMII Link Partner Next Page: (0-0x08) 0x0000
Fiber GMII Autoneg Advertise:       (1-0x04) 0x0801
Fiber GMII Link Partner Ability:    (1-0x05) 0x40a0
Fiber GMII Autoneg Expansion:       (1-0x06) 0x0007
Fiber GMII Next Page Tx:            (1-0x07) 0x2001
Fiber GMII Link Partner Next Page:  (1-0x08) 0x0000
GMII 1000BASE-T Control:            (0-0x09) 0x0e00
GMII 1000BASE-T Status:             (0-0x0a) 0x4000
Reserved Register:                  (0-0x0b) 0x0000
Reserved Register:                  (0-0x0c) 0x0000
Reserved Register:                  (0-0x0d) 0x0000
Reserved Register:                  (0-0x0e) 0x0000
GMII Extended Status:               (0-0x0f) 0xf000
Control Register 1:                 (0-0x10) 0x0078
Copper Status 1:                    (0-0x11) 0x8140
Copper Interrupt Enable:            (0-0x12) 0x0000
Copper Interrupt Status:            (0-0x13) 0x0040
Fiber Status 1:                     (1-0x11) 0x9c10
Fiber Interrupt Enable:             (1-0x12) 0x0000
Fiber Interrupt Status:             (1-0x13) 0x1c00
Control Register 3:                 (0-0x14) 0x0c68
Receive Error Counter:              (0-0x15) 0x0000
Page:                               (0-0x16) 0x0000
Global Status:                      (0-0x17) 0x0000
LED Control:                        (0-0x18) 0x4100
Manual LED Override:                (0-0x19) 0x0000
Control Register 2:                 (0-0x1a) 0x000a
       -                            (0-0x1b) 0x9084
       -                            (0-0x1c) 0x0000
       -                            (0-0x1d) 0x0000
       -                            (0-0x1e) 0x0000
Reserved Register:                  (0-0x1f) 0x0000
PHY type:          88E1111
Auto-neg enable:   YES
Auto-neg complete: NO
Speed/duplex:      NOT RESOLVED
Link status:       DOWN   MDIX      <---------------------

ras at router> show interfaces terse | match ge-0/0/8 
ge-0/0/8                up    up  

We've also seen some really goofy behavior with Cisco branded fiber 
optics on EX4200-24F. For example, if you insert Cisco SX SFPs into 
ports 0, 1, and 2, the TX doesn't turn on correctly. The port can 
receive light, if you turn auto-neg off it will go link up, but the 
laser/led is simple OFF. If you later insert an SFP into port 3, the TX 
on ports 0-2 suddenly turns on. If you then pull one of those ports, all 
of the other ports turn off again. The strange thing is it doesn't 
always happen in blocks of 4, and loading the chassis full of optics 
doesn't guarantee that all ports will turn on either. We saw this 
behavior on multiple versions of 10.1 and 10.2 across multiple 
EX4200-24F chassis, and across the entire range of ports. Also if you 
reboot the box enough times, eventually you may get all of the ports to 
power on the TX following the reboot, but if you pull and reinsert the 
SFP it won't always re-light. We have a case open on this as well, no 
updates yet.

I also suspect there is some kind of initialization issue too, as even 
Juniper branded optics don't always light correctly after you pull and 
reinsert them. We made a small video clip demonstrating this with SX 
SFPs, pardon the lack of focus on the crackberry camera. :)

http://www.youtube.com/watch?v=T-_CtbKinpE

Will try to get a video made showing the "other ports turn on after you 
insert another unrelated optics" issue at some point too.

-- 
Richard A Steenbergen <ras at e-gerbil.net>       http://www.e-gerbil.net/ras
GPG Key ID: 0xF8B12CBC (7535 7F59 8204 ED1F CC1C 53AF 4C41 5ECA F8B1 2CBC)