[j-nsp] Juniper M10i PIC problems

Sat Mar 31 10:44:22 EDT 2012

Really sounds like somehow the chassis/backplane is toast, or that RE
is toast.  I'd open a JTAC case.  I don't know of any JunOS versions
that would cause the symptoms you're showing from the syslog at all.
The RE is clearly having trouble communicating with the chassis
components, whether thats a PCI bus issue, or, the internal switch/MAC
complex I don't know at all.

On Sat, Mar 31, 2012 at 7:41 AM, Emmanuel Kwarteng
<emml.kwarteng at gmail.com> wrote:
> Hello,
>
> I have a Juniper M10i with one 2x STM-1 interface card and 1xGE card in FEC
> 0. It's been working for more than a year now.
>
> It started with all the lights on the box going off. After a reboot, the
> STM-1 interface came up but the GE interface didn't come up.
> The RE active light also takes a long time to come up.
>
> After swapping the slots I realized it's not localized to the slot.
> Now both STM-1 and GE cards dont boot up at all.
>
> Any help would be very much appreciated.
>
> Please find some logs below:
>
>
> -----
> *Mar 30 09:14:25  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x4c, index 0xaa)*
> *Mar 30 09:14:25  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x4c,
> off 0xaa, group 0x4, flags 0xd, len 0x2*
> *Mar 30 09:14:25  bbn-perimeter-rtr4 cfmd[1374]: ------ Unable to fetch cfm
> system mac address. Setting up for retry*
> *Mar 30 09:14:25  bbn-perimeter-rtr4 cfmd[1374]:
> cfmd_get_system_mac_timer_expiry:329 Error fetching reserved mac address.
> Starting timer for retry*
> *Mar 30 09:14:26  bbn-perimeter-rtr4 cfmd[1374]: ------ Unable to fetch cfm
> system mac address. Setting up for retry*
> *Mar 30 09:14:26  bbn-perimeter-rtr4 cfmd[1374]:
> cfmd_get_system_mac_timer_expiry:329 Error fetching reserved mac address.
> Starting timer for retry*
> *Mar 30 09:14:27  bbn-perimeter-rtr4 cfmd[1374]: ------ Unable to fetch cfm
> system mac address. Setting up for retry*
> *Mar 30 09:14:27  bbn-perimeter-rtr4 cfmd[1374]:
> cfmd_get_system_mac_timer_expiry:329 Error fetching reserved mac address.
> Starting timer for retry*
> *Mar 30 09:14:28  bbn-perimeter-rtr4 cfmd[1374]: ------ Unable to fetch cfm
> system mac address. Setting up for retry*
> *Mar 30 09:14:28  bbn-perimeter-rtr4 cfmd[1374]:
> cfmd_get_system_mac_timer_expiry:329 Error fetching reserved mac address.
> Starting timer for retry*
> *Mar 30 09:14:29  bbn-perimeter-rtr4 cfmd[1374]: ------ Unable to fetch cfm
> system mac address. Setting up for retry*
> *Mar 30 09:14:29  bbn-perimeter-rtr4 cfmd[1374]:
> cfmd_get_system_mac_timer_expiry:329 Error fetching reserved mac address.
> Starting timer for retry*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x4a, index 0xaa)*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x4a,
> off 0xaa, group 0x4, flags 0xd, len 0x2*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x4b, index 0xaa)*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x4b,
> off 0xaa, group 0x4, flags 0xd, len 0x2*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x4c, index 0xaa)*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x4c,
> off 0xaa, group 0x4, flags 0xd, len 0x2*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x4a, index 0xaa)*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x4a,
> off 0xaa, group 0x4, flags 0xd, len 0x2*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x4b, index 0xaa)*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x4b,
> off 0xaa, group 0x4, flags 0xd, len 0x2*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x4c, index 0xaa)*
> *Mar 30 09:14:30  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x4c,
> off 0xaa, group 0x4, flags 0xd, len 0x2*
> *
> *
> *
> ar 30 08:59:47  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 09:00:02  bbn-perimeter-rtr4 /kernel: b_op_ok : master perms not
> sufficient..
> Mar 30 08:59:47  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 09:00:01  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:48  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 09:00:01  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:49  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:49  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:50  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:50  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:51  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:51  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:52  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:52  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:53  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: hcm_am_master: unable to read HCM mastership
> configuration
> Mar 30 08:59:53  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:54  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:55  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:55  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:56  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:56  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:59:57  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:59:58  bbn-perimeter-rtr4 chassisd[1413]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output erro
>
> Mar 30 08:15:00 bbn-perimeter-rtr4 newsyslog[1447]: logfile turned over due
> to size>1024K
> Mar 30 08:14:59  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:15:01  bbn-perimeter-rtr4 /kernel: b_op_ok : master perms not
> sufficient..
> Mar 30 08:14:59  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:15:01  bbn-perimeter-rtr4 ksyncd[1405]:
> ksyncd_select_control_plane_proto: rhost_sysctlbyname_get: No such file or
> directory
> Mar 30 08:15:00  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:15:00  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:15:01  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x10)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x10)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB bus in an unstable
> condition. Some device failed on the SMB bus
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x58, off
> 0x10, group 0x0, flags 0xd, len 0x2
> Mar 30 08:15:02  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x16)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x16)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB bus in an unstable
> condition. Some device failed on the SMB bus
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x58, off
> 0x16, group 0x0, flags 0xd, len 0x2
> Mar 30 08:15:02  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
>
>
> ar 30 08:15:00 bbn-perimeter-rtr4 newsyslog[1447]: logfile turned over due
> to size>1024K
> Mar 30 08:14:59  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:15:01  bbn-perimeter-rtr4 /kernel: b_op_ok : master perms not
> sufficient..
> Mar 30 08:14:59  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:15:01  bbn-perimeter-rtr4 ksyncd[1405]:
> ksyncd_select_control_plane_proto: rhost_sysctlbyname_get: No such file or
> directory
> Mar 30 08:15:00  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:15:00  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:15:01  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x10)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x10)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB bus in an unstable
> condition. Some device failed on the SMB bus
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x58, off
> 0x10, group 0x0, flags 0xd, len 0x2
> Mar 30 08:15:02  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_FIC_PRESENCE_READ: fic_presence_masks: FIC unable to get
> presence masks (Input/output error)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x1e)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x16)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB failed transaction
> (bus_addr 0x58, index 0x16)
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB busy timeout
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB bus in an unstable
> condition. Some device failed on the SMB bus
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB read failed addr 0x58, off
> 0x16, group 0x0, flags 0xd, len 0x2
> Mar 30 08:15:02  bbn-perimeter-rtr4 chassisd[1419]:
> CHASSISD_I2C_GENERIC_ERROR: smb_cnh_re_offline_req: unable to read Routing
> Engine remove request
> Mar 30 08:15:02  bbn-perimeter-rtr4 /kernel: SMB device error (bus_addr
> 0x58, index 0x1e)
> *
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp

-- 

"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler