Failed to see controller 1 HP 3PAR 7400

[Torsten.]

Once you run the startnoderescue command, you should see at the serial line:

+-----------------------------------------------------------------------------+
HP SPI Image 3.0.10. Release version. 13:40:19 Feb 03 2014.
+-----------------------------------------------------------------------------+
| CPU 1 x 1.80 GHz Sandy Bridge hexa core HT dual
| Control Cache Size 15.100 GB (cpu mem type 11) CL9
|               Pair0 DIMM0 (J0155): 16384 MB CL6/11
| Data Cache Size 4 GB CL4/6
|               Set0 DC DIMM 0.0.0 (J0300): 2048 MB CL4/6
|               Set0 DC DIMM 0.1.0 (J0301): 2048 MB CL4/6
| Slot ID 0 [2 Node HP 3PAR 7220 Centerplane]
| FPGA Eos v0.f
| SATA0 Disk SanDisk DX110128A5xnNMRI 151311401318 128 GB
| PCI Slot 0 LSI-SAS 9205-8e
| PCI Slot 1 Emulex LPe12002 VM80175440 2-port
| Board 920-200040.B6 FXN 2015/24/Tue 03131653 fc:15:b4:8d:f9:6a
| Cluster serial        <SERIAL>               fc:15:b4:8d:f9:6b
| Board reset reason     EXTERNAL_RESET
| Current Time          2015-09-18 17:50:23 (UTC)
+-----------------------------------------------------------------------------+
Booting from net...
TFTP "install" from 192.168.100.81....................................................................................................complete

Setting FSB WDT Boot Complete State.
Devs: 263
Linux version 2.6.18-prep-3pardata (ddejong@moes) (gcc version 4.4.5 (Debian 4.4.5-8) ) #8 SMP Tue Jul 10 15:21:46 PDT 2012
Command line: BOOT_IMAGE=install console=ttyS0,57600 acpi_hest=off acpi=ht genapic=phys cr=2 ipaddr=169.254.154.129 nm=255.255.0.0 rp=192.168.100.81::rescue hn=SERIAL_0

...

Whack>

Whack>set perm cnt_no_os_boot=0

Whack>set perm cnt_no_cluster=0

Whack>set perm cnt_no_shutdown=0

Whack>set perm cnt_os_panic=0

Whack>set perm cnt_same_fatal=0

Whack>set perm cnt_log_error=0

Whack>set perm sys_serial=<SERIAL>

Whack>net addr 169.254.154.129
My address is 169.254.154.129

Whack>net netmask 0.0.0.16
Network mask 0.0.0.16

Whack>net gateway 192.168.100.81
Gateway address 192.168.100.81

Whack>net server 192.168.100.81
Server address 192.168.100.81

Whack>boot net install cr=2 ipaddr=169.254.154.129 nm=255.255.0.0

Can you confirm?

[djryan]

I ran it again and this is the output up to the sync.

HP SPI Image 3.1.05. Release version. 12:21:30 Aug 19 2014.

Bootstrap CPU0
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3
SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU speed: 1800.00 MHz
Bus speed: 100.00 MHz
Bus to CPU ratio: 1/18

Board reset reason: EXTERNAL_RESET
Previous reason : EXTERNAL_RESET

--> Starting: [0] CPU Interrupt Vector Initialization
--> Starting: [1] CPU SDRAM Memory Scan
Disabled FSBC boot watchdog.

Warning: PCI scan has not completed. It is not safe to use most Whack
commands at this point. Please resume initialization by typing "go" now.
Whack>
Whack>set perm cnt_no_os_boot=0
Whack>set perm cnt_no_cluster=0
Whack>set perm cnt_no_shutdown=0
Whack>set perm cnt_os_panic=0
Whack>set perm cnt_same_fatal=0
Whack>set perm cnt_log_error=0
Whack>set perm sys_serial=1635026
Whack>net addr 169.254.222.196
My address is 169.254.222.196
Whack>net netmask 0.0.0.16
Network mask 0.0.0.16
Whack>net gateway 147.19.151.107
Gateway address 147.19.151.107
Whack>net server 147.19.151.107
Server address 147.19.151.107
Whack>boot net install cr=2 ipaddr=169.254.222.196 nm=255.255.0.0 rp=147.19.151.107::rescue hn=1635026_1
--> Starting: [2] PCI Bus Initialization
--------- Starting PCI scan #2:

00.00.00 SandyBridge DMI2:
Vendor Intel, Device 0x3c00, Rev 0x07
Host/PCI bridge - Intel Device
00.01.00 SandyBridge PCIe Root Port 1a:
Vendor Intel, Device 0x3c02, Rev 0x07
PCI/PCI bridge, bus 01-07
01.00.00 PEX8648 #0:
Vendor PLX, Device 0x8648, Rev 0xbb
PCI/PCI bridge, bus 02-07
02.01.00 PEX8648 #0 DS1 (slot 2):
Vendor PLX, Device 0x8648, Rev 0xbb
PCI/PCI bridge, bus 03
02.04.00 PEX8648 #0 DS4:
Vendor PLX, Device 0x8648, Rev 0xbb
PCI/PCI bridge, bus 04
04.00.00 Harrier LPC2:
Vendor 3PAR, Device 0x0003, Rev 0x00
Large Memory Window 1 at 30:00000000
Disk cache controller
02.05.00 PEX8648 #0 DS5:
Vendor PLX, Device 0x8648, Rev 0xbb
PCI/PCI bridge, bus 05
05.00.00 Harrier LPC0:
Vendor 3PAR, Device 0x0003, Rev 0x00
Small Memory Window 1 at a0000000
Large Memory Window 1 at 10:00000000
Medium Memory Window 1 at 0c:00000000
Disk cache controller
02.08.00 PEX8648 #0 DS8:
Vendor PLX, Device 0x8648, Rev 0xbb
PCI/PCI bridge, bus 06
06.00.00 Slot 0:
Vendor LSI, Device 0x0087, Rev 0x01
SAS mass storage controller
02.09.00 PEX8648 #0 DS9:
Vendor PLX, Device 0x8648, Rev 0xbb
PCI/PCI bridge, bus 07
07.00.00 Slot 1:
Vendor Emulex, Device 0xf100, Rev 0x03
Fibre Channel controller
07.00.01 Slot 1:
Vendor Emulex, Device 0xf100, Rev 0x03
Fibre Channel controller
00.01.01 SandyBridge PCIe Root Port 1b:
Vendor Intel, Device 0x3c03, Rev 0x07
PCI/PCI bridge, bus 08
00.03.00 SandyBridge PCIe Root Port 3a:
Vendor Intel, Device 0x3c08, Rev 0x07
PCI/PCI bridge, bus 09
00.03.01 SandyBridge PCIe Root Port 3b:
Vendor Intel, Device 0x3c09, Rev 0x07
PCI/PCI bridge, bus 0a
00.03.02 SandyBridge PCIe Root Port 3c:
Vendor Intel, Device 0x3c0a, Rev 0x07
PCI/PCI bridge, bus 0b
00.03.03 SandyBridge PCIe Root Port 3d:
Vendor Intel, Device 0x3c0b, Rev 0x07
PCI/PCI bridge, bus 0c
00.04.00 SandyBridge DMA Engine CH0:
Vendor Intel, Device 0x3c20, Rev 0x07
Base system controller
00.04.01 SandyBridge DMA Engine CH1:
Vendor Intel, Device 0x3c21, Rev 0x07
Base system controller
00.04.02 SandyBridge DMA Engine CH2:
Vendor Intel, Device 0x3c22, Rev 0x07
Base system controller
00.04.03 SandyBridge DMA Engine CH3:
Vendor Intel, Device 0x3c23, Rev 0x07
Base system controller
00.04.04 SandyBridge DMA Engine CH4:
Vendor Intel, Device 0x3c24, Rev 0x07
Base system controller
00.04.05 SandyBridge DMA Engine CH5:
Vendor Intel, Device 0x3c25, Rev 0x07
Base system controller
00.04.06 SandyBridge DMA Engine CH6:
Vendor Intel, Device 0x3c26, Rev 0x07
Base system controller
00.04.07 SandyBridge DMA Engine CH7:
Vendor Intel, Device 0x3c27, Rev 0x07
Base system controller
00.05.00 SandyBridge Memory Map/VTd:
Vendor Intel, Device 0x3c28, Rev 0x07
Base system controller
00.05.02 SandyBridge RAS:
Vendor Intel, Device 0x3c2a, Rev 0x07
Base system controller
00.05.04 SandyBridge IOAPIC:
Vendor Intel, Device 0x3c2c, Rev 0x07
Base system controller
00.11.00 PCH Virtual Root Port:
Vendor Intel, Device 0x1d3e, Rev 0x06
PCI/PCI bridge, bus 0d
00.1c.00 PCH PCI Express Port 1:
Vendor Intel, Device 0x1d10, Rev 0xb6
PCI/PCI bridge, bus 0e
00.1c.06 PCH PCI Express Port 7:
Vendor Intel, Device 0x1d1c, Rev 0xb6
PCI/PCI bridge, bus 0f
0f.00.00 Intel 82574 #0:
Vendor Intel, Device 0x10d3, Rev 0x00
Ethernet network controller
00.1c.07 PCH PCI Express Port 8:
Vendor Intel, Device 0x244e, Rev 0xb6
PCI/PCI bridge, bus 10
10.00.00 Intel 82574 #1:
Vendor Intel, Device 0x10d3, Rev 0x00
Ethernet network controller
00.1e.00 PCH DMI to PCI Bridge:
Vendor Intel, Device 0x1d25, Rev 0xa6
PCI/PCI bridge, bus 11
00.1f.00 PCH LPC:
Vendor Intel, Device 0x1d41, Rev 0x06
PCI/ISA bridge
00.1f.02 PCH SATA:
Vendor Intel, Device 0x1d02, Rev 0x06
SATA mass storage controller
00.1f.03 PCH SMBus:
Vendor Intel, Device 0x1d22, Rev 0x06
System Management Bus controller

PCI address allocations:
16-bit I/O space: [ 1000 .. 503f]
32-bit prefetchable memory: [ (none) ]
32-bit non-prefetchable memory: [ 90000000 .. c07fffff]
64-bit prefetchable memory: [10:00000000 .. 3f:ffffffff]
36-bit prefetchable memory: [08:00000000 .. 0f:ffffffff]

PCI bus scan and allocation completed in 2 passes.

PCIe 1.0.0: Serial EEPROM contents verified.
--> Starting: [3] Serial Port Initialization
--> Starting: [4] Remote Console Initialization
--> Starting: [5] Random Number Generator Init
--> Skipping: [6] CPU SDRAM Memory Test
--> Starting: [7] Process NVRAM values
Prior reset 2019-10-23 21:15:13
Last reset 2019-10-24 13:26:53
This reset 2019-10-24 13:30:54
Time now 2019-10-24 13:31:34
--> Starting: [9] CPU APIC Initialization
CPU 0 [0] Microcode check...
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU speed: 1800.00 MHz
Bus speed: 100.00 MHz
Bus to CPU ratio: 1/18
CPU 0 [0] Done

CPU 3 [3] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 3 [3] shutting down.

CPU 5 [5] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 5 [5] shutting down.

CPU 6 [6] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 6 [6] shutting down.

CPU 8 [8] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 8 [8] shutting down.

CPU b [b] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU b [b] shutting down.

CPU 2 [2] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 2 [2] shutting down.

CPU a [a] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU a [a] shutting down.

CPU 9 [9] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 9 [9] shutting down.

CPU 1 [1] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 1 [1] shutting down.

CPU 7 [7] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 7 [7] shutting down.

CPU 4 [4] starting up.
Intel(R) Xeon(R) CPU E5-2428L 0 @ 1.80GHz
Family 6, Model 45, Stepping 7, Features bfebfbff:3

SKU Info: Max Frequency Ratio 18, Max Core Count 6
CPU 4 [4] shutting down.

..............................

ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
ata1.00: ATA-9: PU1000064GBATSSD, CS.51.00, max UDMA/133
ata1.00: 125045424 sectors, multi 1: LBA48 NCQ (depth 31/32)
ata1.00: configured for UDMA/133
Vendor: ATA Model: PU1000064GBATSSD Rev: CS.5
Type: Direct-Access ANSI SCSI revision: 05
SCSI device sda: 125045424 512-byte hdwr sectors (64023 MB)
sda: Write Protect is off
SCSI device sda: drive cache: write back
SCSI device sda: 125045424 512-byte hdwr sectors (64023 MB)
sda: Write Protect is off
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 >
sd 0:0:0:0: Attached scsi disk sda
sd 0:0:0:0: Attached scsi generic sg0 type 0
LKCD: Crash dump driver initialized.
LKCD: block device driver registered
LKCD: dump gzip safety buffer: ffff81027d60a000, 4096
oprofile: using NMI interrupt.
Netfilter messages via NETLINK v0.30.
ip_conntrack version 2.4 (8192 buckets, 65536 max) - 280 bytes per conntrack
ctnetlink v0.90: registering with nfnetlink.
ip_tables: (C) 2000-2006 Netfilter Core Team
arp_tables: (C) 2002 David S. Miller
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 10
IPv6 over IPv4 tunneling driver
ip6_tables: (C) 2000-2006 Netfilter Core Team
NET: Registered protocol family 17
Freeing unused kernel memory: 4352k freed
Write protecting the kernel read-only data: 610k
=============================================================================

Netimagic(tm) Software Installer v
(C) 1991-2012 3PAR Inc. and others. All rights reserved.

=============================================================================
>> Configuring eth0: 169.254.222.196/255.255.0.0 gw
ADDRCONF(NETDEV_UP): eth0: link is not ready
>> Retrieving installation profile from http://147.19.151.107/iderescue/profile-DEFAULT
e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx
ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
mknod: /dev/zero: File exists
Creating partition table.
Disk is EOS sized.
1+0 records in
1+0 records out
SCSI device sda: 125045424 512-byte hdwr sectors (64023 MB)
sda: Write Protect is off
SCSI device sda: drive cache: write back
sda: unknown partition table
SCSI device sda: 125045424 512-byte hdwr sectors (64023 MB)
sda: Write Protect is off
SCSI device sda: drive cache: write back
sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 >
Building filesystems.
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda2, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda3, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
kjournald starting. Commit interval 5 seconds
EXT3 FS on sda7, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
Syncing 147.19.151.107::rescue to /mnt.

Dave

[djryan]

Sorry I might have cut off some of the output

+-----------------------------------------------------------------------------+
HP SPI Image 3.1.05. Release version. 12:21:30 Aug 19 2014.
+-----------------------------------------------------------------------------+
| CPU 1 x 1.80 GHz Sandy Bridge hexa core HT dual
| Control Cache Size 7.100 GB (cpu mem type 11) CL9
| Pair0 DIMM0 (J0155): 8192 MB CL5/11
| Data Cache Size 8 GB CL4/6
| Set0 DC DIMM 0.0.0 (J0300): 4096 MB CL4/6
| Set0 DC DIMM 0.1.0 (J0301): 4096 MB CL4/6
| Slot ID 1 [4 Node HP 3PAR 7400 Centerplane]
| FPGA Eos v0.f
| SATA0 Disk SanDisk PU1000064GBATSSD 131462402083 64 GB
| PCI Slot 0 LSI-SAS 9205-8e
| PCI Slot 1 Emulex LPe12002 VM80175440 2-port
| Board 920-200040.B5 FXN 2014/35/Thu 00264540 a4:5d:36:2e:2d:6a
| Cluster serial 1635026 a4:5d:36:2e:2d:6b
| Board reset reason EXTERNAL_RESET
| Current Time 2019-10-24 13:52:48 (UTC-4)
+-----------------------------------------------------------------------------+
Booting from net...
TFTP "install" from 147.19.151.107....................................................................................................complete

Setting FSB WDT Boot Complete State.
Devs: 263
Linux version 2.6.18-prep-3pardata (ddejong@moes) (gcc version 4.4.5 (Debian 4.4.5-8) ) #8 SMP Tue Jul 10 15:21:46 PDT 2012
Command line: BOOT_IMAGE=install console=ttyS0,57600 acpi_hest=off acpi=ht genapic=phys cr=2 ipaddr=169.254.17.183 nm=255.255.0.0 rp=147.19.151.107::rescue hn=1635026_1

....

Warning: PCI scan has not completed. It is not safe to use most Whack
commands at this point. Please resume initialization by typing "go" now.
Whack>
Whack>set perm cnt_no_os_boot=0
Whack>set perm cnt_no_cluster=0
Whack>set perm cnt_no_shutdown=0
Whack>set perm cnt_os_panic=0
Whack>set perm cnt_same_fatal=0
Whack>set perm cnt_log_error=0
Whack>set perm sys_serial=1635026
Whack>net addr 169.254.17.183
My address is 169.254.17.183
Whack>net netmask 0.0.0.16
Network mask 0.0.0.16
Whack>net gateway 147.19.151.107
Gateway address 147.19.151.107
Whack>net server 147.19.151.107
Server address 147.19.151.107
Whack>boot net install cr=2 ipaddr=169.254.17.183 nm=255.255.0.0 rp=147.19.151.107::rescue hn=1635026_1

Dave

[Torsten.]

This looks good.

You should see a task also (showtask).

How about task details? (showtask <task# -d)

Can you logon now to node 1?

[djryan]

No it fails, and the green light is solid. It can't join the cluster, that is why I created this ticket.

2019-10-24 10:51:43 PDT Created task.
2019-10-24 10:51:43 PDT Updated Running node rescue for node 1 as 0:1454
2019-10-24 10:51:51 PDT Updated Using IP 169.254.17.183
2019-10-24 10:51:51 PDT Updated Informing system manager to not autoreset node 1.
2019-10-24 10:51:51 PDT Updated Resetting node 1.
2019-10-24 10:51:51 PDT Updated Attempting to contact node 1 via NEMOE.
2019-10-24 10:52:20 PDT Updated Setting boot parameters.
2019-10-24 10:52:40 PDT Updated Waiting for node 1 to boot the node rescue kernel.
2019-10-24 10:52:56 PDT Updated Kernel on node 1 has started. Waiting for node to retrieve install details.
2019-10-24 10:53:15 PDT Updated Node 1 has retrieved the install details. Waiting for file sync to begin.
2019-10-24 10:53:32 PDT Updated File sync has begun. Estimated time to complete this step is 5 minutes on a lightly loaded system.
2019-10-24 11:00:38 PDT Updated Remote node has completed file sync, and will reboot.
2019-10-24 11:00:38 PDT Updated Waiting for node to rejoin cluster.
2019-10-24 11:30:38 PDT Error The node has not rejoined the cluster after 30 minutes. The rescue was not successful.
2019-10-24 11:30:38 PDT Failed Could not complete task.

[djryan]

This is why the previous HP engineer said I need to get into WHACK which I haven't been able to do in order to do:

WHACK>prom edit

[Torsten.]

What exactly should you edit?

[Torsten.]

post deleted, this was the wrong thread, sorry.

[djryan]

I was told to edit the following as seen on page 1 of my ticket.

Remember 7400 and 7400C parameters are different ; 

Look at the example for a 7400 below ;

SMI Whack> prom edit

Board Spin:       03

Size * 256 bytes: 04

Board Class:      920

Board Base:       200040

Board Rev:        91

Assembly Vendor:  FXN

Assembly Year:    2012

Assembly Week:    30

Assembly Day:     03

Assembly Serial:  00103530

Alternate Class:  ffff

Alternate Base:   ffff

Alternate Dash:   ff

System Serial:    <7 digit serial>         < ----- System Serial is set with  the command 'set perm sys_serial=<7 digit serial>'

Node ID:          01                                 < --------- set correct node ID  - node 1 = 01   node 0 = 00 ; likewise. 

Midplane Type:    1b                            < ---------- This is the correct value for 7400

Node Type:        40                              < ---------- This is the correct value for 7400

[djryan]

I really have no idea what you are talking about. We had a controller go out and we ordered a new one from HP. We recieved it and it has never worked from day one. This is the reason for this ticket. I have no idea what your talking about. We have 2 fiber connections and 2 SAS connections and the ethernet connnections to it. I don't know what a VC module is.

If you don't have VC modules, there is no counterpart for the NICs, hence your problem.