3par 8400 cannot boot to whack

[DUNGPQ]

Hi all,

I have problem while install new controller for 3par 8400.  I cannot boot to whack, error occur after boot is S1 dnodef1481007 NEMOE (HARDWARE FAILURE).

This is boot log:

S1 dnodef1481007 NEMOE (HARDWARE FAILURE) -> Loading PEIM at 0x000FFFD92A0 CpuPeim.efi
Loading PEIM at 0x000FFFCC2A0 PcatSingleSegmentPciCfg2Pei.efi
Loading PEIM at 0x000FFFCDC20 PciCfgOnPciCfg2Thunk.efi
Loading PEIM at 0x000FFFD7220 PlatformInfo.efi
Board ID = BoardIdTornadoNode
Waited 3500 ms for NEMOE boot
Board Revision : 06
BIOS Version: HPE SPI Image 05.05.07. Release version. 14:14:20 Nov 01 2019.

Loading PEIM at 0x000FFFDA5A0 PeiVariable.efi
Loading PEIM at 0x000FFFDBE20 ReadOnlyVariableOnReadOnlyVariable2Thunk.efi
Loading PEIM at 0x000FFFCE1A0 PchInitPeim.efi
Loading PEIM at 0x000FFFDCF40 PlatformEarlyInit.efi
Loading PEIM at 0x000FFFCA820 CpuIoPei.efi
Loading PEIM at 0x000FFFD61C0 IioDmiInitPeim.efi
Loading PEIM at 0x000FFFE1120 CapsulePei.efi
Loading PEIM at 0x000FFFE4F20 PchSmbusArpDisabled.efi
Loading PEIM at 0x000FFFE5FA0 PchSpiPeim.efi
Loading PEIM at 0x000FFFE8740 TxtPei.efi
Loading PEIM at 0x000FFF00100 UncoreInit.efi
Loading PEIM at 0x000FFFD92A0 CpuPeim.efi
Loading PEIM at 0x000FFFCC2A0 PcatSingleSegmentPciCfg2Pei.efi
Loading PEIM at 0x000FFFCDC20 PciCfgOnPciCfg2Thunk.efi
Loading PEIM at 0x000FFFD7220 PlatformInfo.efi
Board ID = BoardIdTornadoNode
Board Revision : 06
BIOS Version: HPE SPI Image 05.05.07. Release version. 14:14:20 Nov 01 2019.

Loading PEIM at 0x000FFFDA5A0 PeiVariable.efi
Loading PEIM at 0x000FFFDBE20 ReadOnlyVariableOnReadOnlyVariable2Thunk.efi
Loading PEIM at 0x000FFFCE1A0 PchInitPeim.efi
Loading PEIM at 0x000FFFDCF40 PlatformEarlyInit.efi
Loading PEIM at 0x000FFFCA820 CpuIoPei.efi
Loading PEIM at 0x000FFFD61C0 IioDmiInitPeim.efi
Loading PEIM at 0x000FFFE1120 CapsulePei.efi
Loading PEIM at 0x000FFFE4F20 PchSmbusArpDisabled.efi
Loading PEIM at 0x000FFFE5FA0 PchSpiPeim.efi
Loading PEIM at 0x000FFFE8740 TxtPei.efi
Loading PEIM at 0x000FFF00100 UncoreInit.efi
Tickling EOS Failsafe Boot Watchdog Timer.
MRC log data - Socket: 0x0, Channel : 0x1, DIMM: 0x0, Rank: 0x0
Corresponding DIMM on node board: DIMM 0.1.0
OemHook Assert discovered: codetype: 0x38 subcode: 0x3014 data: 0x10000
PEIM Assert, waiting 10 seconds..........Successfully wrote to Serial PROM for error log.
Loading PEIM at 0x000FFFD8F20 CpuPeim.efi
Loading PEIM at 0x000FFFCC0A0 PcatSingleSegmentPciCfg2Pei.efi
Loading PEIM at 0x000FFFCD9A0 PciCfgOnPciCfg2Thunk.efi
Loading PEIM at 0x000FFFD6F20 PlatformInfo.efi
Board ID = BoardIdTornadoNode
Board Revision : 06
BIOS Version: HP SPI Image 04.08.38. Release version. 16:01:11 Jul 17 2015.

Loading PEIM at 0x000FFFDA1A0 PeiVariable.efi
Loading PEIM at 0x000FFFDBA20 ReadOnlyVariableOnReadOnlyVariable2Thunk.efi
Loading PEIM at 0x000FFFCDF20 PchInitPeim.efi
Loading PEIM at 0x000FFFDCB40 PlatformEarlyInit.efi
Loading PEIM at 0x000FFFCA6A0 CpuIoPei.efi
Loading PEIM at 0x000FFFD5EC0 IioDmiInitPeim.efi
Loading PEIM at 0x000FFFE0D20 CapsulePei.efi
Loading PEIM at 0x000FFFE4AA0 PchSmbusArpDisabled.efi
Loading PEIM at 0x000FFFE5B20 PchSpiPeim.efi
Loading PEIM at 0x000FFFE8240 TxtPei.efi
Loading PEIM at 0x000FFF00100 UncoreInit.efi
Loading PEIM at 0x000FFFD8F20 CpuPeim.efi
Loading PEIM at 0x000FFFCC0A0 PcatSingleSegmentPciCfg2Pei.efi
Loading PEIM at 0x000FFFCD9A0 PciCfgOnPciCfg2Thunk.efi
Loading PEIM at 0x000FFFD6F20 PlatformInfo.efi
Board ID = BoardIdTornadoNode
Board Revision : 06
BIOS Version: HP SPI Image 04.08.38. Release version. 16:01:11 Jul 17 2015.

Loading PEIM at 0x000FFFDA1A0 PeiVariable.efi
Loading PEIM at 0x000FFFDBA20 ReadOnlyVariableOnReadOnlyVariable2Thunk.efi
Loading PEIM at 0x000FFFCDF20 PchInitPeim.efi
Loading PEIM at 0x000FFFDCB40 PlatformEarlyInit.efi
Loading PEIM at 0x000FFFCA6A0 CpuIoPei.efi
Loading PEIM at 0x000FFFD5EC0 IioDmiInitPeim.efi
Loading PEIM at 0x000FFFE0D20 CapsulePei.efi
Loading PEIM at 0x000FFFE4AA0 PchSmbusArpDisabled.efi
Loading PEIM at 0x000FFFE5B20 PchSpiPeim.efi
Loading PEIM at 0x000FFFE8240 TxtPei.efi
Loading PEIM at 0x000FFF00100 UncoreInit.efi
Tickling EOS Failsafe Boot Watchdog Timer.
MRC log data - Socket: 0x0, Channel : 0x2, DIMM: 0x0, Rank: 0x0
OemHook Assert discovered: codetype: 0x38 subcode: 0x3013 data: 0x20000
PEIM Assert, waiting 10 seconds..........Successfully wrote to Serial PROM for error log.
NEMOE taking over from FPGA.
NEMOE attention due to failsafe boot failed (FSBC_STAT: 0x1c): enter ^N^E^M^O^E to exit.
S1 dnodef1481007 NEMOE (HARDWARE FAILURE) ->
S1 dnodef1481007 NEMOE (HARDWARE FAILURE) ->
S1 dnodef1481007 NEMOE (HARDWARE FAILURE) ->
S1 dnodef1481007 NEMOE (HARDWARE FAILURE) ->
S1 dnodef1481007 NEMOE (HARDWARE FAILURE) ->

I tried reset command it from here, but the problem didn't resolve.

S1 dnodef1481007 NEMOE (HARDWARE FAILURE) -> ?
cpu - Control CPU
c - Change memory or device registers
debug - Change debug output flags
diag - Change diagnostic level
d - Display memory or device registers
echo - Print arguments on output
exit - Exit from console takeover
fault - Change diagnostic fault injections
filter - Filter task output flags
flash - Show or program firmware in flash
fpga - Control onboard FPGA
help - Show usage for all commands or a specific command
i2c - I2C transaction
inject_event - Inject a TPD event
led - Control node LEDs
link - Control links
log - Display or log events
loop - Repeatedly execute a command
mailbox - Manage BIOS/sysmgr mailboxes
mem - Perform miscellaneous operations on memory
panic - Call panic or print reason if already panicked
pex - Perform PEX8648 operations
pci - Access PCI devices
post - Perform Power On Self Tests
prom - Control EEPROM data
reset - Reset board component
sched - Enable / Disable scheduled task processing
sensors - Show Sensors
shownode - Show node state
time - Time operation
sysmgr - Manage BIOS/sysmgr communication
var - Display or change variables
ver - Display NEMOE firmware version
what - Redisplay console output
whack - Force Whack entry on local or remote node

And debug all log:

I2C 2.a6.16 (AT24C16B) read 01
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
EEPROM READ: dev=0x0 addr=0x316 sz=0x01
I2C 2.a6.16 (AT24C16B) read 01
I2C 2.a6.16 (AT24C16B) read 01
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
EEPROM READ: dev=0x0 addr=0x316 sz=0x01
I2C 2.a6.16 (AT24C16B) read 01
I2C 2.a6.16 (AT24C16B) read 01
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
EEPROM READ: dev=0x0 addr=0x316 sz=0x01
I2C 2.a6.16 (AT24C16B) read 01
I2C 2.a6.16 (AT24C16B) read 01
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
EEPROM READ: dev=0x0 addr=0x316 sz=0x01
I2C 2.a6.16 (AT24C16B) read 01
I2C 2.a6.16 (AT24C16B) read 01
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
Sys LED Off
HP LED Off
Sys LED Amber
HP LED Blue
Local sysmgr/BIOS is busy or not responding..
INT0:LINK link=0 space=4607 blk_count=16
Link 0 [0] RECV 8f 0c 9f 25 00 01 00 01 00 ba 31 00 55 33 21 62
hdr len=100
Link 0 [0] RECV 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 01 ff ff ff ff ff ff ff ff ff ff ff ff ff ff
Link 0 [0] RECV 03 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 03 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 08 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Link 0 [0] RECV 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
RECV Link 0 [0] len=0100 src=0 dst=1 0:0 tag=ba cmd=31 (0,62213355) data 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ff ff ff ff ff ff ff ff ff ff ff ff ff ff 03 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 00 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
EEPROM READ: dev=0x0 addr=0x316 sz=0x01
I2C 2.a6.16 (AT24C16B) read 01
I2C 2.a6.16 (AT24C16B) read 01
Sys LED Off
HP LED Off
Local sysmgr/BIOS is busy or not responding..
Sys LED Amber
HP LED Blue

Pls support resolve it.

Thank you!

[sbhat09]

Hello @DUNGPQ,

Please check if any of the similar discussions below help you to address the issue:

https://community.hpe.com/t5/HPE-3PAR-StoreServ-Storage/3Par-7200-Interface-Card-0-Failed-Interface-Card-Firmware/m-p/7142392#M6843

https://community.hpe.com/t5/HPE-3PAR-StoreServ-Storage/3Par-7200c-Node-Offline-Due-to-Failure-0xd-Node-IDE-Drive/m-p/7110714#M6270

https://community.hpe.com/t5/HPE-3PAR-StoreServ-Storage/Failed-to-see-controller-1-HP-3PAR-7400/m-p/7067108#M5351

Regards,
Srinivas Bhat

If you feel this was helpful please click the KUDOS! thumb below!
Note: All of my comments are my own and are not any official representation of HPE.

[DUNGPQ]

Thank you replied.

I read your suggestion, but these are not my case. After connect serial cables to Mfg port, I unplug, then plug controller again, but my controller cannot boot to whack, or go to step to run crtl + W.

[veeyarvi]

Hi DUNGPQ,

It looks like the node is unable to boot and has a hardware failure. It is not passing the hardware test it appears assuming it has a blue and amber LED active on the node.

PS: Try ctrl + w + h + a + c + k (hold down the ctrl key and type whack) to get in to the whack. If that also does not work, the node is unable to get in to this booting stage as well.