[j-nsp] MX960 power supply stopped during ISSU

[Tim Warnock]

Power supplies have firmware on them ;)

Regardless - I don't know much about the MX960 arch but do you have enough power supplies to maintain N+1  at full tilt?

> -----Original Message-----
> From: juniper-nsp [mailto:juniper-nsp-bounces at puck.nether.net] On Behalf
> Of Aaron Gould
> Sent: Wednesday, 30 January 2019 5:12 AM
> To: juniper-nsp at puck.nether.net
> Subject: [j-nsp] MX960 power supply stopped during ISSU
> 
> Last night I had a successful ISSU upgrade. BUT. "show chassis alarm" showed
> me that PEM0 power supply had issues.  Searching logs didn't turn up any
> previous issues so I think that this happened during the ISSU process.
> Anyway ever seen something like that before?  I would've thought that a
> software upgrade wouldn't do much with power, but I'm wondering now.
> 
> 
> 
> 17.4R1-S2.2 - old
> 
> 17.4R2-S1.2 - new
> 
> 
> 
> agould at blvr-960> show chassis alarms
> 
> 3 alarms currently active
> 
> Alarm time               Class  Description
> 
> 2019-01-29 00:33:12 CST  Major  PEM 0 Input Failure
> 
> 2019-01-29 00:33:12 CST  Major  PEM 0 Not OK
> 
> 2019-01-29 00:32:27 CST  Minor  Backup RE Active
> 
> 
> 
> .this morning CO Tech went on site and said power feeds to PEM0 were fine,
> and no tripped fuzes or anything.  "show chassis power" showed 2 feeds
> expected and connected, and good power but not putting anything out.
> 
> He removed the bad PEM0 and put it into lab MX960, and it works!
> 
> 
> 
> Some messages seen were. I wonder what "bump volt" means ?  .wondering
> if
> that is an action to actually hit the voltage of each PEM, and if so, wonder
> if that would've tripped on offline.
> 
> 
> 
> Jan 29 00:32:20  hwdb: entry for cbd 2988 at slot 2 inserted
> 
> Jan 29 00:32:20  acb_add: CB 2, initializing SGLS SGLINk type 2 Local ACB
> type 4
> 
> Jan 29 00:32:20  acb_sglink_init: GE 8374 PHY PMC ctrl 2 : 0xa300 at slot 2
> 
> Jan 29 00:32:20  acb_sglink_init: GE 8374 PHY PMC ctrl 2: set TXCLK4 at slot
> 2
> 
> Jan 29 00:32:20  acb_sglink_init: GE 8354 PHY Auto Neg Status 2: 0x2a for
> slot 2
> 
> Jan 29 00:32:20  acb_sglink_init: GE 8354 PHY is byte aligned for slot 2
> 
> Jan 29 00:32:21  acb_sglink_init: GE 8374 PHY Auto Neg Status 2: 0xa0 at
> slot 2
> 
> Jan 29 00:32:21  acb_sglink_init: GE 8374 PHY is byte aligned at slot 2
> 
> Jan 29 00:32:21  acb_sglink_init: CB slot 2 SGLS version 0
> 
> Jan 29 00:32:21  acb_sglink_init: CB slot 2 SGLS type 2, acb type 4
> 
> Jan 29 00:32:21  acb_add: CB 2, initializing PCIe hub
> 
> Jan 29 00:32:21  acb_add: setting CB 2 cache type and i2c 0xbac
> 
> Jan 29 00:32:21  ch_probe_frus: Routing Engine 1 added
> 
> Jan 29 00:32:21  reading RE 1 initial state
> 
> Jan 29 00:32:21  reading host processor dimms
> 
> Jan 29 00:32:22  hwdb: entry for re 3087 at slot 1 inserted
> 
> Jan 29 00:32:22  ch_probe_frus: PEM 0 added
> 
> Jan 29 00:32:22  reading PEM 0 initial state
> 
> Jan 29 00:32:22  Bump volt: reset structure for pem 0 during add
> 
> Jan 29 00:32:22  ch_probe_frus: PEM 1 added
> 
> Jan 29 00:32:22  reading PEM 1 initial state
> 
> Jan 29 00:32:22  Bump volt: reset structure for pem 1 during add
> 
> Jan 29 00:32:22  ch_probe_frus: PEM 2 added
> 
> Jan 29 00:32:22  reading PEM 2 initial state
> 
> Jan 29 00:32:22  Bump volt: reset structure for pem 2 during add
> 
> Jan 29 00:32:22  ch_probe_frus: PEM 3 added
> 
> Jan 29 00:32:22  reading PEM 3 initial state
> 
> Jan 29 00:32:22  Bump volt: reset structure for pem 3 during add
> 
> Jan 29 00:32:22  ch_probe_frus: FPM 0 added
> 
> Jan 29 00:32:22  reading FPM 0 initial state
> 
> Jan 29 00:32:22  check_and_carp_on_i2cs_version I2CS version=0x29
> 
> 
> 
> Jan 29 00:33:12  blvr-960 alarmd[16028]: Alarm set: Pwr supply color=RED,
> class=CHASSIS, reason=PEM 0 Not OK
> 
> Jan 29 00:33:12  blvr-960 craftd[13352]:  Major alarm set, PEM 0 Not OK
> 
> Jan 29 00:33:12  blvr-960 chassisd[13337]: CHASSISD_PEM_INPUT_BAD:
> status
> failure for power supply 0 (status bits: 0x2); check circuit breaker
> 
> Jan 29 00:33:12  blvr-960 alarmd[16028]: Alarm set: Pwr supply color=RED,
> class=CHASSIS, reason=PEM 0 Input Failure
> 
> Jan 29 00:33:12  blvr-960 craftd[13352]:  Major alarm set, PEM 0 Input
> Failure
> 
> Jan 29 00:33:12  blvr-960 chassisd[13337]: CHASSISD_PEM_INPUT_BAD: Input
> failure for power supply 0 (status bits: 0x2); check circuit breaker
> 
> Jan 29 00:33:17  blvr-960 chassisd[13337]: CHASSISD_PEM_INPUT_BAD:
> status
> failure for power supply 0 (status bits: 0x2); check circuit breaker
> 
> 
> 
> Jan 29 00:33:12  send: red alarm set, device PEM 0, reason PEM 0 Not OK
> 
> Jan 29 00:33:12 CHASSISD_PEM_INPUT_BAD: status failure for power supply
> 0
> (status bits: 0x2); check circuit breaker
> 
> Jan 29 00:33:12  send: red alarm set, device PEM 0, reason PEM 0 Input
> Failure
> 
> Jan 29 00:33:12 CHASSISD_PEM_INPUT_BAD: Input failure for power supply 0
> (status bits: 0x2); check circuit breaker
> 
> 
> 
> -Aaron
> 
> 
> 
> _______________________________________________
> juniper-nsp mailing list juniper-nsp at puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp