MSA p2000 G3 - Lost configuration

[Dermot1]

Hello,

I seem to have lost all my configuration. The management utility shows no enclosures, no hosts, no (v)disks or volumes. The logs are empty where yesterday they were full of information. I have this icon in the upper-right corner. Link checks do not run and rescanning the disk does not show them

The system is 2 x p2000 enclosures each full with disks of various sizes. There were about 6 vdisks yesterday.

Problems began when a host that had a storage volume mapped remouted the storage as readonly. I tried various things from the host to mount the volume again but the volume was not writable. Other volumes from the enclosures were ok. In fact, as far as I can tell,. I have a host that is still mounting (r/w) a volume from the P2000. The hosts are linux and often the storage is mounted on a VM. I created a small volume on the same vdisk and tried to mount that on 2 different host and write a disklabel to it. It consistently timed out.

At some point yesterday I created a volume copy from a snapshot onto a different vdisk - in the hope that a new vdisk would be ok. I also tried to expand the problem vdisk because I was having trouble creating the copy on a different vdisk. The problem with creating a vdisk on a different volume turned out to be that the Controller owner was not the same. All of these events were in the log. Now I have nothing at all. Everything has gone from view. There are 24 disks but no matter how often I rescan, nothing returns.

At the rear of the units, everything seems ok.

I am very stuck now. I am out of ideas and could use some advice.

Thanks,

Dermot

[support_s]

System recommended content:

1. HPE P2000 G3 MSA Array System - Support Guides | Cause

2. HPE P2000 G3 MSA Array Systems - Support Guides | Environment

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[Dermot1]

Thanks for the reply however as the logs are empty, I cannot confirm if the supercapacitor has expiried.

My unit is listed in the environment, "HPE P2000 G3 MSA FC Dual Controller LFF Array System"

[Dermot1]

EDIT: Ignore this message, the file extract mentioned is not useful

I am not sure if this reveals something but the log file that I managed to download from the management UI, shows a jump in the dates. The date jump from "Sep 22 05:11:00" to "Jan 1 00:00:".

Below is a snippet from the messages file.

Jan 1 00:00:06 (none) user.err kernel: 0078 => 00000005
Jan 1 00:00:06 (none) user.err kernel: 0080 => 00000000
Jan 1 00:00:06 (none) user.err kernel: 0088 => 00100000
Jan 1 00:00:06 (none) user.err kernel: 0090 => 00000000
Jan 1 00:00:06 (none) user.err kernel: 0098 => 00000000
Jan 1 00:00:06 (none) user.err kernel: ----------------
Jan 1 00:00:06 (none) user.err kernel: CPLD SC Up (CPLD value = 0x7f)
Jan 1 00:00:06 (none) user.info kernel: Mouse_Process_StoM_IMSG: Force SC UP imsg=4 cpld=0x7f
Jan 1 00:00:06 (none) user.warn kernel: Send Up Message
Jan 1 00:00:06 (none) user.info kernel: MOUSE Driver Version A10 Installed at Major 150
Jan 1 00:00:06 (none) user.info kernel: Mouse_Process_MtoS_IMSG: SC UP imsg=1 cpld=0x7f
Jan 1 00:00:06 (none) user.info kernel: squashfs: version 4.0 (2009/01/31) Phillip Lougher
Jan 1 00:00:06 (none) user.info kernel: g_serial gadget: Gadget Serial v2.4
Jan 1 00:00:06 (none) user.info kernel: g_serial gadget: g_serial ready
Jan 1 00:00:06 (none) user.err kernel: STATE: Mouse Birth
Jan 1 00:00:06 (none) user.err kernel: CPLD Mouse Birth (CPLD value = 0x7f)
Jan 1 00:00:06 (none) user.warn kernel: Reenable Bottom Driver
Jan 1 00:00:06 (none) user.notice root: init: Crond started
Sep 22 05:11:00 (none) user.notice root: init: read system time from file
Sep 22 05:11:00 (none) user.notice root: init: initialize slip at 230400 576
Sep 22 05:11:00 (none) user.notice root: slip.srv: starting slip SPEED: 230400 MTU: 576
Sep 22 05:11:00 (none) user.notice root: hotplug net
Sep 22 05:11:02 (none) user.notice root: init: Initializing network interfaces lo, eth0
Sep 22 05:11:02 (none) user.notice root: networking.srv: Restarting network service on eth0:

[ArunKKR]

Hi,

Please launch an SSH session to both the controllers and check whether they provide the outputs:

show vdisks

show system

show unwritable-cache

show disks

show controllers

This issue requires log analysis and I would suggest logging an HPE support case to avoid data loss.

[Dermot1]

Below is the output from the commands.

# show vdisks
Name Size Free Own Pref RAID Disks Spr Chk Status Jobs Job% Serial Number Drive Spin Down Spin Down Delay
---------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully.

# show system
System Information
------------------
System Name: hp-p2000-01
System Contact: Dermot Paikkos
System Location: Science Photo Library
System Information: Motion Storage
Midplane Serial Number:
Vendor Name: HP StorageWorks
Product ID: P2000 G3 FC
Product Brand: MSA Storage
SCSI Vendor ID: HP
SCSI Product ID: P2000 G3 FC
Enclosure Count: 0
Health: OK
Health Reason:
Supported Locales: English (English)

Success: Command completed successfully.

#show unwritable-cache
Error: Unable to communicate with Storage Controller. Please retry the command.

# show disks
Location Serial Number Vendor Rev How Used Type Size Rate*(Gb/s) SP Health Health Reason
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
Info: * Rates may vary. This is normal behavior.

Success: Command completed successfully.

# show controllers

Controllers
-----------
Controller ID: A
Serial Number: Not Available
Hardware Version:
CPLD Version:
MAC Address: 00:00:00:00:00:00
WWNN: 0000000000000000
IP Address: No_data
IP Subnet Mask: No_data
IP Gateway: No_data
Disks: 0
Vdisks: 0
Cache Memory Size (MB): 0
Host Ports: 0
Disk Channels: 0
Disk Bus Type: Unknown
Status: Operational
Failed Over: No
Fail Over Reason: Not applicable
Position: Top
Phy Isolation: Enabled

Controller ID: B
Serial Number: Not Available
Hardware Version:
CPLD Version:
MAC Address: 00:00:00:00:00:00
WWNN: 0000000000000000
IP Address: No_data
IP Subnet Mask: No_data
IP Gateway: No_data
Disks: 0
Vdisks: 0
Cache Memory Size (MB): 0
Host Ports: 0
Disk Channels: 0
Disk Bus Type: Unknown
Status: Operational
Failed Over: No
Fail Over Reason: Not applicable
Position: Bottom
Phy Isolation: Enabled

Info: Could not read SAS info, so skipping enclosure 0.

Info: Could not read SAS info, so skipping enclosure 0.

Success: Command completed successfully.

I assure you, there are physical disk.

Front view of p2000

[ArunKKR · Accepted solution]

Hi, This issue seems to require a power cycle. I would suggest getting an HPE support case logged for further assistance.

[Dermot1]

That seemed to do have fixed it. I have to physically pull the power out of the PSU.

I can see disks and vdisks now.

# show vdisks
Name Size Free Own Pref RAID Disks Spr Chk Status Jobs Job% Serial Number Drive Spin Down Spin Down Delay
-----------------------------------------------------------------------------------------------------------------------------------
antares_vd04 15.9TB 8787.0GB B B RAID5 5 0 64k FTOL 00c0ff134fb10000b51a215b00000000 Disabled 0
pollux 5995.1GB 0B B B RAID1 2 0 N/A FTOL 00c0ff134fb100000e8cdd6100000000 Disabled 0
vd01 11.9TB 4165.3GB A A RAID5 3 0 64k FTOL 00c0ff134fb10000bf44155a00000000 Disabled 0
vd02 11.9TB 353.5GB B B RAID5 4 0 64k FTOL EXPD 0% 00c0ff134fb10000c06c155a00000000 Disabled 0
vd03 11.9TB 1499.3GB A A RAID5 3 0 64k FTOL 00c0ff134fb100001f98165a00000000 Disabled 0
vd06 11.9TB 7441.3GB B B RAID5 3 0 64k FTOL VRSC 0% 00c0ff134fb10000fb162b6300000000 Disabled 0
-----------------------------------------------------------------------------------------------------------------------------------
Success: Command completed successfully.

Thanks for you help.

Dermot

[Sunitha_Mod]

@Dermot1

Hello Dermot, 

Awesome! Glad to know the problem has been resolved.