Unable to Activate VG01 | Smart Array logical drive failed

[Aamir_Majeed]

Hi,

We have VG01 on HP-UX 11.31 server, it is created on MSA 30 JBOD connected via Smart Array 64000 controller.

We have 2 logical drives as mentioned below;

---- LOGICAL DRIVE SUMMARY ---------------------------------------------------

# RAID Size Status

0 1+0 286097 MB FAILED

1 1+0 286097 MB RECOVERING

　

---- SCSI DEVICE SUMMARY -----------------------------------------------------

Location Ch ID Type Capacity Status

external 1 0 DISK 300.0 GB OK

external 1 1 DISK 300.0 GB OK

external 1 2 DISK 300.0 GB OK

external 1 3 DISK 300.0 GB OK

Both of these logical drives are part of  Volume group VG01;

# strings /etc/lvmtab

/dev/vg00

/dev/disk/disk3_p2

/dev/dsk/c2t1d0s2

/dev/vg01

M>q7

/dev/dsk/c4t0d0

/dev/dsk/c4t0d1

disk 6 0/2/1/0/4/0.0.0 sdisk CLAIMED DEVICE HP LOGICAL VOLUME /dev/dsk/c4t0d0 /dev/rdsk/c4t0d0

disk 7 0/2/1/0/4/0.0.1 sdisk CLAIMED DEVICE HP LOGICAL VOLUME /dev/dsk/c4t0d1 /dev/rdsk/c4t0d1

Please note both of the logical HDDs are showing claimed status, I am susspecting /dev/dsk/c4t0d0 as one Logical Drive is shown Failed.

Is it possible to activate VG01 without this PV.

Thanks and Regards,

Aamir Majeed.

[Torsten.]

If the LVOLs are not mirrored at LVM level you need all the PVs present.

Find out the controllers /dev/ciss_ device and send a

# sautil /dev/ciss_

# saconfig /dev/ciss_


The status of FAILED doesn't looks good - any disks replaced???

[Aamir_Majeed]

Hi,

Yes, LVOLs are not mirrored at logical level, Please find the attached output; and also please tell me how to assign points on replies, as I am not able to see any option.

Thanks,

Aamir Majeed.

[Torsten.]

Please send


# saconfig /dev/ciss4

# ioscan -fn

# vgdisplay -v vg01

# strings /etc/lvmtab

Any disks already replaced?

To assign points click the white KUDOS! start in the left column.

[Torsten.]

logical drive 0 consists of disk 0 and 1.

both were "REMOVED IN HOT PLUG" - now FAILED

logical drive 1 consists of disk 2 and 3.

disk 2 was also REMOVED IN HOT PLUG and is now RECOVERING.



Looks like you pulled all 3 disks online and now you have a destroyed logical drive!

What have you done?

[Aamir_Majeed]

Actually, mentioned setup is installed at customer site. As per customer there were amber LEDs on 3 disks, before considering any thing he power cycled disk enclosure and rebooted server.

Now he called us, to look into issue.

Currently we are unable to activate VG01.

Disks are showing green LED now, and also we are able to see both drives as claimed status

and also diskinfo is showing proper information, but dd is showing errors on one logical drive please find attachment for further output.

Also note we are getting proper results when we

At the moment my question is how we can activate Vg01, and is there any way that we can recover those disks.

[Aamir_Majeed]

Sorry, missed log file.

[Torsten.]

I'm absolutely sure the disks 0,1 and 2 were pulled.

While disks 0 and 1 were pulled, the array did not have any disks left, hence it is failed.

Disk 2 was also pulled, currently the array synchronizes disk 3 to disk 2 (REBUILDING).


Is this a Itanium server?


Consider to reboot the server, watch out for smartarray messages on the console during boot.


You did not post the requested information yet.

[Aamir_Majeed]

Yes this is rx4640 Itanium server, and also status has been changed to OK of Raid group 1 which was previously rebuilding.

Is there any way to activate this VG01 with only one PV.

[Aamir_Majeed]

Please refer to my previous post, I have attached file named ems.log.

It contains required information except Vgdisplay as I cannot get this info as VG is not active.

[Torsten.]

You may try to activate the VG, maybe some of the LVOLS are still there, but likely 50% of the data (the other PV) is just gone - hence you probably have corrupted filesystems.

What does the controller say (on console) during boot?

[Aamir_Majeed]

After enabling Logical Drive during initail startup, I can activate Volume Group now, But some of LVOLs are not coming.

I am getting following errors;

===============================================================================================

Searching for devices
Scsi(Pun0,Lun0) HP 73.4GST373454LC      HPC2 (320 MBytes/sec)
Scsi(Pun1,Lun0) HP 73.4GST373454LC      HPC2 (320 MBytes/sec)
HP Smart Array 6400-128 Controller       (version 2.34)  2 Logical Drives
1792-Drive Array - Valid Data Found in Array Accelerator
     Data will automatically be written to drive array.
1794-Drive Array - Array Accelerator Battery Charge Low
     Array Accelerator Posted-Write Cache is temporarily disabled.
     Array Accelerator batteries have failed to charge and should be replaced
1779-Drive Array - Replacement drive(s) detected OR previously failed
     drive(s) now appear to be operational:
         SCSI Port 1:  SCSI ID(s) 0, 1  [ Enclosure bay(s): 1, 2 ]
     Logical drive(s) disabled due to possible data loss.
     Restore data from backup if replacement drive(s) have been installed.

Press "F1" to continue...

# mountall
mountall: /dev/vg01/windb has to be fsck'd
mountall: diagnostics from fsck
UX:vxfs fsck: ERROR: V-3-20023: file system on /dev/vg01/windb has structural damage
mountall: cannot fsck /dev/vg01/windb
mountall: diagnostics from fsck
file system check failure, aborting ...
nt 0 has bad checksum
read of primary OLT failed
UX:vxfs fsck: ERROR: V-3-20727: OLT extent 1 has bad magic
read of OLT copy failed
UX:vxfs fsck: ERROR: V-3-20718: no valid OLT, cannot continue
mountall: /dev/vg01/windb failed in fsck
mountall: /dev/vg01/windb cannot be mounted
mountall: /dev/vg01/u02 has to be fsck'd
mountall: diagnostics from fsck
UX:vxfs fsck: ERROR: V-3-20023: file system on /dev/vg01/u02 has structural damage
mountall: cannot fsck /dev/vg01/u02
mountall: diagnostics from fsck
file system check failure, aborting ...
nt 0 has bad checksum
read of primary OLT failed
UX:vxfs fsck: ERROR: V-3-20727: OLT extent 1 has bad magic
read of OLT copy failed
UX:vxfs fsck: ERROR: V-3-20718: no valid OLT, cannot continue
mountall: /dev/vg01/u02 failed in fsck
mountall: /dev/vg01/u02 cannot be mounted
mountall: /dev/vg01/u01 has to be fsck'd
mountall: diagnostics from fsck
UX:vxfs fsck: ERROR: V-3-20023: file system on /dev/vg01/u01 has structural damage
mountall: cannot fsck /dev/vg01/u01
mountall: diagnostics from fsck
file system check failure, aborting ...
nt 0 has bad checksum
read of primary OLT failed
UX:vxfs fsck: ERROR: V-3-20726: OLT extent 1 has bad checksum
read of OLT copy failed
UX:vxfs fsck: ERROR: V-3-20718: no valid OLT, cannot continue
mountall: /dev/vg01/u01 failed in fsck
mountall: /dev/vg01/u01 cannot be mounted

===============================================================================================

Is there any way to recover this damage ?

Thanks,

AM

[Torsten.]

What is the status in

# sautil /dev/ciss4

# saconfig /dev/ciss4

now?

[Aamir_Majeed]

Hi,

One logical drive status is OK and other is still failed, is there any solution available to recover Logical Volumes;

UX:vxfs fsck: ERROR: V-3-20023: file system on /dev/vg01/windb has structural damage
mountall: cannot fsck /dev/vg01/windb
mountall: diagnostics from fsck
file system check failure, aborting ...
nt 0 has bad checksum
read of primary OLT failed
UX:vxfs fsck: ERROR: V-3-20727: OLT extent 1 has bad magic
read of OLT copy failed
UX:vxfs fsck: ERROR: V-3-20718: no valid OLT, cannot continue

Thanks, AM.