[j-nsp] M20 stopped routing
Paul Davies
p.davies at coreix.net
Thu May 28 07:42:16 EDT 2009
Hi all,
We have a Juniper M20 with RE-3.0 (1SSB/1RE) running JunOS 8.5R1.14.
Basically it seems a power supply died, we proposed to change this supply this morning however before we got a chance the router experienced some issues. The router basically lost all BGP sessions/OSPF instances et cetera, and although the BGP sessions came back online and the router was accepting prefixes it could not actually route the traffic. This basically caused an outage on this particular router. The other core router on the site (also an M20, same specifications), could not take over all the traffic immediately since it did think the other router was forwarding traffic (based on the fact it had active prefixes). There were multiple small outages on this router as a result basically. The unit eventually came back online correctly while we were investigating. We have a full log of the event naturally (however we have taken out a lot of information since the log is about 2MB).
The router has been taken offline basically, so we can debug this issue. As a rule most errors we can debug, this one looks a little nasty! We have our suspicious and theories, however just wanted some other opinions / suggestions / rants ;)
The abridged log shows the following:
-------------------------------------------------------------
May 28 04:29:36 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 2, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr Power Supply B, jnxOperatingState/Temp 6)
May 28 04:32:05 core1.sta.lon2 smartd[4206]: atastandbyarmset: ioctl: Inappropriate ioctl for device
May 28 04:32:05 core1.sta.lon2 smartd[4206]: standby_request: Error: atastandbyarmset(TRUE): Inappropriate ioctl for device
May 28 04:56:31 core1.sta.lon2 /kernel: fxp1: link DOWN 100Mb / full-duplex
May 28 04:56:31 core1.sta.lon2 eventd: sendto: Network is down
May 28 04:56:31 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 2, ifAdminStatus up(1), ifOperStatus down(2), ifName fxp1
May 28 04:56:31 core1.sta.lon2 snmpd[4317]: SNMPD_SEND_FAILURE: trap_io_send_trap_now: send to (193.26.222.140) failure: Network is down
May 28 04:56:31 core1.sta.lon2 snmpd[4317]: SNMPD_SEND_FAILURE: trap_io_send_trap_now: send to (193.26.222.150) failure: Network is down
May 28 04:56:31 core1.sta.lon2 /kernel: rdp retransmit error: Network is down (50) src 0x00000001:1020 dest 0x00000012:28672
May 28 04:56:31 core1.sta.lon2 vrrpd[4288]: VRRPD_NEW_MASTER: Interface ge-0/1/0.900 (local address 193.26.222.194) became VRRP master for group 0
May 28 04:56:31 core1.sta.lon2 rpd[4285]: RPD_OSPF_NBRDOWN: OSPF neighbor fe80::217:fff:feac:6a80 (ge-0/0/0.56) state changed from Full to Down due to KillNbr (event reason: interface went down)
May 28 04:56:31 core1.sta.lon2 rpd[4285]: RPD_OSPF_NBRDOWN: OSPF neighbor fe80::217:fff:feac:6a40 (ge-0/1/0.54) state changed from Full to Down due to KillNbr (event reason: interface went down)
May 28 04:56:32 core1.sta.lon2 /kernel: fxp1: link UP 100Mb / full-duplex
May 28 04:56:33 core1.sta.lon2 eventd: sendto: Network is down
May 28 04:56:33 core1.sta.lon2 /kernel: rdp keepalive expired, connection dropped - src 0x00000001:1020 dest 0x00000012:28672
May 28 04:56:33 core1.sta.lon2 /kernel: rdp keepalive expired, connection dropped - src 0x00000001:1020 dest 0x00000011:18432
May 28 04:56:33 core1.sta.lon2 /kernel: rdp keepalive expired, connection dropped - src 0x00000001:1020 dest 0x00000010:20480
May 28 04:56:33 core1.sta.lon2 /kernel: rdp keepalive expired, connection dropped - src 0x00000001:1020 dest 0x00000002:1025
May 28 04:56:33 core1.sta.lon2 /kernel: rdp keepalive expired, connection dropped - src 0x00000001:1021 dest 0x00000002:1024
May 28 04:56:33 core1.sta.lon2 /kernel: pfe_listener_disconnect: conn dropped: listener idx=3, tnpaddr=0x12, reason: socket error
May 28 04:56:33 core1.sta.lon2 /kernel: pfe_listener_disconnect: conn dropped: listener idx=2, tnpaddr=0x11, reason: socket error
May 28 04:56:33 core1.sta.lon2 /kernel: pfe_listener_disconnect: conn dropped: listener idx=1, tnpaddr=0x10, reason: socket error
May 28 04:56:33 core1.sta.lon2 /kernel: pfe_listener_disconnect: conn dropped: listener idx=0, tnpaddr=0x2, reason: socket error
May 28 04:56:33 core1.sta.lon2 chassisd[4196]: CHASSISD_SHUTDOWN_NOTICE: Shutdown reason: SSB connection lost
May 28 04:56:33 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(0)
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 70, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-0/0/0
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 71, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-0/1/0
May 28 04:56:33 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(1)
May 28 04:56:33 core1.sta.lon2 /kernel: bundle as0.0: bundle IFL state changed to DOWN
May 28 04:56:33 core1.sta.lon2 rpd[4285]: RPD_OSPF_NBRDOWN: OSPF neighbor 85.13.192.22 (ge-0/0/0.56) state changed from Full to Down due to KillNbr (event reason: interface went down)
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 53, ifAdminStatus up(1), ifOperStatus down(2), ifName so-1/0/0
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 72, ifAdminStatus up(1), ifOperStatus down(2), ifName so-1/0/1
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 79, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-0/0/0.56
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 73, ifAdminStatus up(1), ifOperStatus down(2), ifName so-1/0/2
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 80, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-0/0/0.902
May 28 04:56:33 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(2)
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 83, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-0/0/0.908
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 84, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-0/0/0.32767
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 77, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-2/3/0
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 75, ifAdminStatus up(1), ifOperStatus down(2), ifName ge-1/2/0
May 28 04:56:33 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_ALL_PSEUDO: ifdev_detach(pseudo devices: all)
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 106, ifAdminStatus up(1), ifOperStatus down(2), ifName as0
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 85, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-0/1/0.54
May 28 04:56:33 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 86, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-0/1/0.91
May 28 04:56:33 core1.sta.lon2 /kernel: BAD_PAGE_FAULT: pid 4288 (vrrpd), uid 0: pc 0x807274a got a read fault at 0x5c, x86 fault flags = 0x4
May 28 04:56:33 core1.sta.lon2 /kernel: Trapframe Register Dump:
May 28 04:56:33 core1.sta.lon2 /kernel: eax: 00000002 ecx: 00026368 edx: 00000000 ebx: 0820e800
May 28 04:56:33 core1.sta.lon2 /kernel: esp: bfbedd2c ebp: bfbeddc8 esi: bfbedd90 edi: 0820d000
May 28 04:56:33 core1.sta.lon2 /kernel: eip: 0807274a eflags: 00010282
May 28 04:56:33 core1.sta.lon2 /kernel: cs: 0033 ss: 003b ds: bfbe003b es: 81f003b
May 28 04:56:33 core1.sta.lon2 /kernel: fs: 003b trapno: 0000000c err: 00000004
May 28 04:56:33 core1.sta.lon2 /kernel: Page table info for PC address 0x807274a: PDE = 0x51c80067, PTE = 54bc3425
May 28 04:56:33 core1.sta.lon2 /kernel: Dumping 16 bytes starting at PC address 0x807274a:
May 28 04:56:33 core1.sta.lon2 /kernel: ff 72 5c 8d 45 a8 50 ff 72 58 e8 5b 0b ff ff b1
May 28 04:56:33 core1.sta.lon2 /kernel: aggr_link_op: link so-1/0/0.0 (lidx=0) detached from bundle as0.0
May 28 04:56:33 core1.sta.lon2 /kernel: so-1/0/0.0 leaves as0.0
May 28 04:56:33 core1.sta.lon2 /kernel: aggr_link_op: link so-1/0/1.0 (lidx=1) detached from bundle as0.0
May 28 04:56:33 core1.sta.lon2 /kernel: so-1/0/1.0 leaves as0.0
May 28 04:56:34 core1.sta.lon2 /kernel: aggr_link_op: link so-1/0/2.0 (lidx=2) detached from bundle as0.0
May 28 04:56:34 core1.sta.lon2 /kernel: so-1/0/2.0 leaves as0.0
May 28 04:56:34 core1.sta.lon2 /kernel: aggr_link_op: link so-1/0/3.0 (lidx=3) detached from bundle as0.0
May 28 04:56:34 core1.sta.lon2 /kernel: so-1/0/3.0 leaves as0.0
May 28 04:56:34 core1.sta.lon2 alarmd[4197]: shutting down chassisd connection: chassisd ipc pipe read error
May 28 04:56:34 core1.sta.lon2 alarmd[4197]: chassisd connection succeeded after 0 retries
May 28 04:56:34 core1.sta.lon2 alarmd[4197]: resending alarm state
May 28 04:56:34 core1.sta.lon2 alarmd[4197]: Alarm set: Pwr supply color=RED, class=CHASSIS, reason=Power Supply B not providing power
May 28 04:56:34 core1.sta.lon2 craftd[4198]: craftd_user_conn_shutdown: socket 5, errno = 0
May 28 04:56:34 core1.sta.lon2 craftd[4198]: chassisd connection succeeded after 0 retries
May 28 04:56:34 core1.sta.lon2 craftd[4198]: forwarding display request to chassisd: type = 4, subtype = 43
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 92, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.9
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 93, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.10
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 94, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.24
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 95, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.30
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 96, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.35
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 97, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.40
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 102, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.63
May 28 04:56:34 core1.sta.lon2 /kernel: ae_link_op: link ge-2/2/0.58 (lidx=0) detached from bundle ae0.58
May 28 04:56:34 core1.sta.lon2 /kernel: ge-2/2/0.58 leaves ae0.58
May 28 04:56:34 core1.sta.lon2 /kernel: ae_link_op: link ge-2/2/0.32767 (lidx=0) detached from bundle ae0.32767
May 28 04:56:34 core1.sta.lon2 /kernel: ge-2/2/0.32767 leaves ae0.32767
May 28 04:56:34 core1.sta.lon2 /kernel: ae_link_op: link ge-2/3/0.58 (lidx=1) detached from bundle ae0.58
May 28 04:56:34 core1.sta.lon2 /kernel: ge-2/3/0.58 leaves ae0.58
May 28 04:56:34 core1.sta.lon2 /kernel: ae_link_op: link ge-2/3/0.32767 (lidx=1) detached from bundle ae0.32767
May 28 04:56:34 core1.sta.lon2 /kernel: ge-2/3/0.32767 leaves ae0.32767
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 104, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.96
May 28 04:56:34 core1.sta.lon2 init: vrrp (PID 4288) terminated by signal number 11. Core dumped!
May 28 04:56:34 core1.sta.lon2 init: vrrp (PID 6057) started
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 103, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-1/2/0.65
May 28 04:56:34 core1.sta.lon2 mib2d[4284]: SNMP_TRAP_LINK_DOWN: ifIndex 131, ifAdminStatus down(2), ifOperStatus down(2), ifName ge-2/2/0.58
May 28 04:56:34 core1.sta.lon2 vrrpd[6057]: snmp_ipc_try_connect: opened socket 15
May 28 04:56:34 core1.sta.lon2 vrrpd[6057]: SNMP_SUBAGENT_IPC_REG_ROWS: snmp_mib_register: registering 4 rows
May 28 04:56:34 core1.sta.lon2 vrrpd[6057]: SNMP_SUBAGENT_IPC_REG_OK: reg_comp: received all responses, canceling timer
May 28 04:56:35 core1.sta.lon2 rpd[4285]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 2001:668:0:3::4000:211 (External AS 3257) changed state from Established to Idle (event Stop)
May 28 04:56:35 core1.sta.lon2 rpd[4285]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 77.67.65.225 (External AS 3257) changed state from Established to Idle (event Stop)
May 28 04:56:35 core1.sta.lon2 rpd[4285]: RPD_OSPF_NBRDOWN: OSPF neighbor 193.26.222.14 (ge-1/2/0.10) state changed from Full to Down due to KillNbr (event reason: interface went down)
May 28 04:56:35 core1.sta.lon2 rpd[4285]: RPD_OSPF_NBRDOWN: OSPF neighbor fe80::280:42ff:fe13:4051 (as0.0) state changed from Full to Down due to KillNbr (event reason: interface went down)
May 28 04:56:35 core1.sta.lon2 rpd[4285]: RPD_OSPF_NBRDOWN: OSPF neighbor 193.26.222.10 (as0.0) state changed from Full to Down due to KillNbr (event reason: interface went down)
May 28 04:56:35 core1.sta.lon2 rpd[4285]: cannot perform nh operation ADDANDGET nhop (null) type unilist index 0 errno 2
May 28 04:56:35 core1.sta.lon2 last message repeated 2 times
May 28 04:56:35 core1.sta.lon2 rpd[4285]: Indirect next hop operation 2 on protocol next hop 85.13.192.30 table id 16777216 tag 0 index 0 (0x14f3a618) attempts=3 failed, error "No such file or directory"
May 28 04:56:35 core1.sta.lon2 rpd[4285]: cannot perform nh operation ADDANDGET nhop (null) type unilist index 0 errno 2
May 28 04:56:36 core1.sta.lon2 mgd[6078]: UI_CHILD_EXITED: Child exited: PID 6092, status 69, command '/usr/sbin/httpd'
May 28 04:56:37 core1.sta.lon2 rpd[4285]: Indirect next hop operation 2 on protocol next hop 195.50.118.25 table id 16777216 tag 0 index 0 (0xa8167ec) attempts=3 failed, error "No such file or directory"
May 28 04:56:37 core1.sta.lon2 rpd[4285]: cannot perform nh operation ADDANDGET nhop 2001:7f8:4::2f09:1 type unicast index 0 errno 2
May 28 04:56:37 core1.sta.lon2 rpd[4285]: cannot perform nh operation ADDANDGET nhop 2001:7f8:4::5394:1 type unicast index 0 errno
May 28 04:57:01 core1.sta.lon2 snmpd[4317]: SNMPD_SEND_FAILURE: trap_io_send_trap_now: send to (193.26.222.140) failure: No route to host
May 28 04:57:01 core1.sta.lon2 snmpd[4317]: SNMPD_SEND_FAILURE: trap_io_send_trap_now: send to (193.26.222.150) failure: No route to host
May 28 04:57:02 core1.sta.lon2 rpd[4285]: Indirect next hop operation 2 on protocol next hop 193.203.5.227 table id 16777216 tag 0 index 0 (0x14f3a09c) attempts=403 failed, error "No such file or directory"
May 28 04:57:02 core1.sta.lon2 rpd[4285]: Indirect next hop operation 2 on protocol next hop 193.203.5.110 table id 16777216 tag 0 index 0 (0x13e6d57c) attempts=603 failed, error "No such file or directory"
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 6, jnxFruL1Index 1, jnxFruL2Index 0, jnxFruL3Index 0, jnxFruName SSB 0, jnxFruType 5, jnxFruSlot 1, jnxFruOfflineReason 2, jnxFruLastPowerOff 13145130, jnxFruLastPowerOn 13145130)
May 28 04:57:04 core1.sta.lon2 ssb CM(0): ALARM SET: (Major) Power Supply B not providing power
May 28 04:57:04 core1.sta.lon2 /kernel: scb_peer_init:overriding peertype_module
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(0)
May 28 04:57:04 core1.sta.lon2 alarmd[4197]: Alarm cleared: Pwr supply color=RED, class=CHASSIS, reason=Power Supply B not providing power
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(1)
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(2)
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_FPC: ifdev_detach(3)
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_DETACH_ALL_PSEUDO: ifdev_detach(pseudo devices: all)
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for ae0
May 28 04:57:04 core1.sta.lon2 craftd[4198]: Major alarm cleared, Power Supply B not providing power
May 28 04:57:04 core1.sta.lon2 craftd[4198]: forwarding display request to chassisd: type = 4, subtype = 44
May 28 04:57:04 core1.sta.lon2 craftd[4198]: attempt to delete alarm not in list
May 28 04:57:04 core1.sta.lon2 craftd[4198]: forwarding display request to chassisd: type = 4, subtype = 44
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as0
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as1
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as2
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as3
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as4
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as5
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as6
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as7
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as8
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as9
May 28 04:57:04 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as10
May 28 04:57:05 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as11
May 28 04:57:05 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as12
May 28 04:57:05 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as13
May 28 04:57:05 core1.sta.lon2 chassisd[4196]: CHASSISD_IFDEV_CREATE_NOTICE: create_pseudos: created pseudo interface device for as14
May 28 04:57:07 core1.sta.lon2 ssb CM(0): ALARM CLEAR: Power Supply B not providing power
May 28 04:57:09 core1.sta.lon2 alarmd[4197]: Alarm set: Pwr supply color=RED, class=CHASSIS, reason=Power Supply B not providing power
May 28 04:57:09 core1.sta.lon2 craftd[4198]: Major alarm set, Power Supply B not providing power
May 28 04:57:09 core1.sta.lon2 craftd[4198]: forwarding display request to chassisd: type = 4, subtype = 43
May 28 04:57:11 core1.sta.lon2 ssb CM(0): ALARM SET: (Major) Power Supply B not providing power
May 28 04:57:20 core1.sta.lon2 dumpd: Core and context for vrrpd saved in /var/tmp/vrrpd.core-tarball.0.tgz
May 28 04:57:31 core1.sta.lon2 xntpd[4283]: sendto(212.178.119.179): No route to host
May 28 04:57:31 core1.sta.lon2 rpd[4285]: bgp_hold_timeout: NOTIFICATION sent to 85.13.192.36 (Internal AS 31708): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 85.13.192.36 (Internal AS 31708), socket buffer sndcc: 161 rcvcc: 0 TCP state: 4, snd_una: 2440140547 snd_nxt: 2440140708 snd_wnd: 16061 rcv_nxt: 2060406974 rcv_adv: 2060423358, hold timer 0
May 28 04:57:31 core1.sta.lon2 rpd[4285]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 85.13.192.36 (Internal AS 31708) changed state from Established to Idle (event HoldTime)
May 28 04:57:32 core1.sta.lon2 rpd[4285]: bgp_hold_timeout: NOTIFICATION sent to 2a01:c0::39 (Internal AS 31708): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 2a01:c0::39 (Internal AS 31708), socket buffer sndcc: 38 rcvcc: 0 TCP state: 4, snd_una: 3240634558 snd_nxt: 3240634596 snd_wnd: 15909 rcv_nxt: 3378616059 rcv_adv: 3378632443, hold timer 0
May 28 04:57:32 core1.sta.lon2 rpd[4285]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 2a01:c0::39 (Internal AS 31708) changed state from Established to Idle (event HoldTime)
May 28 04:57:32 core1.sta.lon2 /kernel: fpc40_peer_init:overriding peertype_module
May 28 04:57:32 core1.sta.lon2 chassisd[4196]: CHASSISD_FRU_EVENT: fpc_m40_recv_restart: restarted FPC 0
May 28 04:57:33 core1.sta.lon2 xntpd[4283]: sendto(212.178.119.179): No route to host
May 28 04:57:33 core1.sta.lon2 rpd[4285]: bgp_hold_timeout: NOTIFICATION sent to 2a01:c0::38 (Internal AS 31708): code 4 (Hold Timer Expired Error), Reason: holdtime expired for 2a01:c0::38 (Internal AS 31708), socket buffer sndcc: 57 rcvcc: 0 TCP state: 4, snd_una: 4188158518 snd_nxt: 4188158575 snd_wnd: 15966 rcv_nxt: 3734871778 rcv_adv: 3734888162, hold timer 0
May 28 04:57:33 core1.sta.lon2 rpd[4285]: RPD_BGP_NEIGHBOR_STATE_CHANGED: BGP peer 2a01:c0::38 (Internal AS 31708) changed state from Established to Idle (event HoldTime)
May 28 04:57:34 core1.sta.lon2 /kernel: fpc40_peer_init:overriding peertype_module
May 28 04:57:34 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 8, jnxFruL1Index 1, jnxFruL2Index 1, jnxFruL3Index 0, jnxFruName PIC: 1x G/E, 1000 BASE-SX @ 0/0/*, jnxFruType 11, jnxFruSlot 1, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)
May 28 04:57:34 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 8, jnxFruL1Index 1, jnxFruL2Index 2, jnxFruL3Index 0, jnxFruName PIC: 1x G/E, 1000 BASE-SX @ 0/1/*, jnxFruType 11, jnxFruSlot 1, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)
May 28 04:57:34 core1.sta.lon2 chassisd[4196]: CHASSISD_FRU_EVENT: fpc_m40_recv_restart: restarted FPC 1
May 28 04:57:35 core1.sta.lon2 /kernel: fpc40_peer_init:overriding peertype_module
May 28 04:57:35 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 8, jnxFruL1Index 2, jnxFruL2Index 1, jnxFruL3Index 0, jnxFruName PIC: 4x OC-3 SONET, SMIR @ 1/0/*, jnxFruType 11, jnxFruSlot 2, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)
May 28 04:57:35 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 8, jnxFruL1Index 2, jnxFruL2Index 3, jnxFruL3Index 0, jnxFruName PIC: 1x G/E, 1000 BASE-SX @ 1/2/*, jnxFruType 11, jnxFruSlot 2, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)
May 28 04:57:35 core1.sta.lon2 xntpd[4283]: sendto(212.178.119.179): No route to host
May 28 04:57:35 core1.sta.lon2 chassisd[4196]: CHASSISD_FRU_EVENT: fpc_m40_recv_restart: restarted FPC 2
May 28 04:57:35 core1.sta.lon2 chassisd[4196]: CHASSISD_FRU_EVENT: scb_recv_slot_attach: attached FPC 0
May 28 04:57:36 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 8, jnxFruL1Index 3, jnxFruL2Index 3, jnxFruL3Index 0, jnxFruName PIC: 1x G/E, 1000 BASE-SX @ 2/2/*, jnxFruType 11, jnxFruSlot 3, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)
May 28 04:57:36 core1.sta.lon2 chassisd[4196]: CHASSISD_SNMP_TRAP10: SNMP trap generated: FRU power on (jnxFruContentsIndex 8, jnxFruL1Index 3, jnxFruL2Index 4, jnxFruL3Index 0, jnxFruName PIC: 1x G/E, 1000 BASE-SX @ 2/3/*, jnxFruType 11, jnxFruSlot 3, jnxFruOfflineReason 2, jnxFruLastPowerOff 0, jnxFruLastPowerOn 0)
May 28 04:57:37 core1.sta.lon2 xntpd[4283]: sendto(212.178.119.179): No route to host
May 28 04:57:41 core1.sta.lon2 last message repeated 2 times
May 28 04:57:41 core1.sta.lon2 chassisd[4196]: CHASSISD_FRU_EVENT: scb_recv_slot_attach: attached FPC 1
May 28 04:57:43 core1.sta.lon2 xntpd[4283]: sendto(212.178.119.179): No route to host
May 28 04:57:45 core1.sta.lon2 xntpd[4283]: sendto(212.178.119.179): No route to host
May 28 04:57:47 core1.sta.lon2 chassisd[4196]: CHASSISD_FRU_EVENT: scb_recv_slot_attach: attached FPC 2
-------------------------------------------------------------
Wondering if anyone can help regarding this matter (any guidance is appreciated) - Please advise if you need any further information.
Best regards,
Paul Davies
Coreix Limited
DDI: (+44) 0207 183 1725 ext 94
Mobile: (+44) 07950 040 999
Fax: (+44) 0208 53 44 111
*********************************************************************
This message is intended solely for the use of the individual or organisation to whom it is addressed. It may contain privileged or confidential information. If you are not the intended recipient, you should not use, copy, alter, or disclose the contents of this message
*********************************************************************
__________ Information from ESET NOD32 Antivirus, version of virus signature database 4111 (20090528) __________
The message was checked by ESET NOD32 Antivirus.
http://www.eset.com
More information about the juniper-nsp
mailing list