[j-nsp] J2300/J4300 FPCs cannot go online

[Mircho Mirchev]

Hi all,
We have a lot of J2300/J4300 routers in educational labs.
Suddenly (this weekend) on all of them all the interfaces (both embedded
and on line cards) disappeared.
Researching the issue come with that the FPCs (even FPC0, which is the
build-in) cannot be onlined and are in status " Chassis connection dropped"

root at J4300> show chassis fpc detail
Slot 0 information:
  State                               Offline
  Reason                              Chassis connection dropped
  Total CPU DRAM                    0 MB

Further investigations revealed the following in chassisd log:
.....
Mar 27 11:51:24 fpc 0 clean, bringing online
Mar 27 11:51:24 IXP id from HW 0x707 slot 0
Mar 27 11:51:24 send: fpc 0 online cmd
*Mar 27 11:51:25 Failed to verify signature*
......
Mar 27 11:51:25 CHASSISD_PIPE_WRITE_ERROR: Pipe write error: Broken pipe
Mar 27 11:51:25 CHASSISD_IPC_FLUSH_ERROR: ch_flush_fru_pipe: flush
operation failed for FPC 0
Mar 27 11:51:25 CHASSISD_SIGPIPE: SIGPIPE received
Mar 27 11:51:25 rcv: ch_ipc_dispatch() null ipc read for args 0x89cc560
pipe 0x89d2080, fru FPC 0
Mar 27 11:51:25 fpc_disconnect_generic: fpc 0 state Online cargs 0x89cc560
clean_shutdown 0
Mar 27 11:51:25 -- FPC 0, last request 132, state Online
Mar 27 11:51:25 send: fwdd, fpc 0 pic 0 detached
Mar 27 11:51:25 CHASSISD_IPC_CONNECTION_DROPPED: Dropped IPC connection for
FPC 0
Mar 27 11:51:25 CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(0)
Mar 27 11:51:25 fpc_offline_now - slot 0, reason: None, error Chassis
connection dropped
Mar 27 11:51:25 hwdb: entry for fpc 1799 at slot 0 deleted
.....
And the FPC0 goes offline.

The interesting thing is, that setting the router date to several years
earlier (i.e. 2010), the FPCs go online without any problem.
This seems to us as an expired HW certificate, or something similar, but it
is acting as a time bomb :)

Does anyone had a similar issue and found a workaround besides bringing the
router date backwards?

Any help is appreciated.

Regards,
Mircho