System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

Volume group on MSA-30 degraded

atari83
Advisor

Volume group on MSA-30 degraded

Dears,

we have two node cluster (HA - rx2660) and both servers are connected to two MSA-30 enclusores via SCSI cables. basically each server has two iethernet interface (each iethernet interface provides two SCSI interface) and each iethernet interface is connected to a MSA-30 enclsure.

First of all before i go any furthur, i'm 100% sure there is no clustering issue.

Today we informed about a failure on service, so we checked the cluster status and found out that the cluster package is "halted". before checking anything else we decided to switch over to other node and see what will happen. well ... it didnt worked out either and we ended up with same result. the cluster-package got halted.

We start to check basic logs all around the server, and it was our finding:

1- The reason that the cluster-package couldnt get started is that a volume group (vgsmp) in the storage could not be activated, so then the package also couldnt start. 

2- resmon and syslogs, shows more info that what could be the reason:

syslog:

Sep 23 16:04:08 smp1 vmunix: SCSI Ultra320 0/2/1/0/4/0 instance 1: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 00 7e 4e 00 00 02 00 00 
Sep 23 16:10:12 smp1 vmunix: SCSI Ultra320 0/3/1/0/4/0 instance 3: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 01 12 b0 00 00 02 00 00 
Sep 23 16:14:06 smp1 vmunix: SCSI Ultra320 0/2/1/0/4/0 instance 1: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 00 28 08 00 00 02 00 00 
Sep 23 16:19:40 smp1 vmunix: SCSI Ultra320 0/3/1/0/4/0 instance 3: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 01 0f 1a 00 00 02 00 00 
Sep 23 16:20:12 smp1 vmunix: SCSI Ultra320 0/3/1/0/4/0 instance 3: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 01 12 b0 00 00 02 00 00 
Sep 23 16:22:38 smp1 vmunix: SCSI Ultra320 0/2/1/0/4/0 instance 1: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 00 1c 6c 00 00 02 00 00 
Sep 23 16:35:08 smp1 vmunix: SCSI Ultra320 0/2/1/0/4/0 instance 1: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 00 7e 4e 00 00 02 00 00 
Sep 23 16:38:36 smp1 vmunix: SCSI Ultra320 0/2/1/0/4/0 instance 1: IO Type : SCSI IO has timed-out. Target ID: 3, LUN ID: 0. Read Command - CDB: 28 00 00 ef 28 00 00 02 00 00 

 

resmon log:

*
Physical Device Path...: 0/2/1/0/4/0.3.0
Device Class...........: Disk
Inquiry Vendor ID......: COMPAQ
Inquiry Product ID.....: BF07284961
Firmware Version.......: HPB5
Serial Number..........: J2VU88AC

*
Physical Device Path...: 0/3/1/0/4/0.3.0
Device Class...........: Disk
Inquiry Vendor ID......: COMPAQ
Inquiry Product ID.....: BF07284961
Firmware Version.......: HPB5
Serial Number..........: J2VU8M2C

 

Now that we made sure at least some broken disks could cause the issue, first we start to collect some info:

lvmtab:

/dev/vgsmp
/dev/dsk/c3t1d0
/dev/dsk/c3t2d0
/dev/dsk/c3t3d0
/dev/dsk/c1t1d0
/dev/dsk/c1t2d0
/dev/dsk/c1t3d0

ioscan with -fnCdisk showed all disks are CLAIMED (which is odd i think)

so, we tried replace a disk for that volume and lets see what will happen (we tried to replace disk on0/3/1/0/4/0.3.0 which is mapped to /dev/dsk/c3t3d0)

we replaced the disk and we did new ioscan and diskinfo :

# ioscan -fnCdisk
Class I H/W Path Driver S/W State H/W Type Description
====================================================================================
disk 3 0/0/2/1.0.16.0.0 sdisk CLAIMED DEVICE TEAC DVD-ROM DW-224EV
/dev/dsk/c0t0d0 /dev/rdsk/c0t0d0
disk 1 0/1/1/0.0.0.0.0 sdisk CLAIMED DEVICE HP DG146BB976
/dev/dsk/c7t0d0 /dev/dsk/c7t0d0s1 /dev/dsk/c7t0d0s2 /dev/dsk/c7t0d0s3 /dev/rdsk/c7t0d0 /dev/rdsk/c7t0d0s1 /dev/rdsk/c7t0d0s2 /dev/rdsk/c7t0d0s3
disk 2 0/1/1/0.0.0.1.0 sdisk CLAIMED DEVICE HP DG146BB976
/dev/dsk/c7t1d0 /dev/dsk/c7t1d0s1 /dev/dsk/c7t1d0s2 /dev/dsk/c7t1d0s3 /dev/rdsk/c7t1d0 /dev/rdsk/c7t1d0s1 /dev/rdsk/c7t1d0s2 /dev/rdsk/c7t1d0s3
disk 4 0/2/1/0/4/0.0.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c1t0d0 /dev/rdsk/c1t0d0
disk 7 0/2/1/0/4/0.1.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c1t1d0 /dev/rdsk/c1t1d0
disk 10 0/2/1/0/4/0.2.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0
disk 13 0/2/1/0/4/0.3.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c1t3d0 /dev/rdsk/c1t3d0
disk 16 0/2/1/0/4/0.4.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c1t4d0 /dev/rdsk/c1t4d0
disk 19 0/2/1/0/4/0.5.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c1t5d0 /dev/rdsk/c1t5d0
disk 23 0/2/1/0/4/0.8.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c1t8d0 /dev/rdsk/c1t8d0
disk 5 0/2/1/0/4/1.0.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c2t0d0 /dev/rdsk/c2t0d0
disk 9 0/2/1/0/4/1.1.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c2t1d0 /dev/rdsk/c2t1d0
disk 11 0/2/1/0/4/1.2.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c2t2d0 /dev/rdsk/c2t2d0
disk 14 0/2/1/0/4/1.3.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c2t3d0 /dev/rdsk/c2t3d0
disk 18 0/2/1/0/4/1.4.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c2t4d0 /dev/rdsk/c2t4d0
disk 21 0/2/1/0/4/1.5.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c2t5d0 /dev/rdsk/c2t5d0
disk 24 0/2/1/0/4/1.8.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c2t8d0 /dev/rdsk/c2t8d0
disk 6 0/3/1/0/4/0.0.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c3t0d0 /dev/rdsk/c3t0d0
disk 8 0/3/1/0/4/0.1.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c3t1d0 /dev/rdsk/c3t1d0
disk 12 0/3/1/0/4/0.2.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c3t2d0 /dev/rdsk/c3t2d0
disk 15 0/3/1/0/4/0.3.0 sdisk CLAIMED DEVICE COMPAQ BF0728B26A
/dev/dsk/c3t3d0 /dev/rdsk/c3t3d0
disk 17 0/3/1/0/4/0.4.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c3t4d0 /dev/rdsk/c3t4d0
disk 20 0/3/1/0/4/0.5.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c3t5d0 /dev/rdsk/c3t5d0
disk 22 0/3/1/0/4/0.8.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c3t8d0 /dev/rdsk/c3t8d0
disk 25 0/3/1/0/4/1.0.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c4t0d0 /dev/rdsk/c4t0d0
disk 26 0/3/1/0/4/1.1.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c4t1d0 /dev/rdsk/c4t1d0
disk 27 0/3/1/0/4/1.2.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c4t2d0 /dev/rdsk/c4t2d0
disk 28 0/3/1/0/4/1.3.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c4t3d0 /dev/rdsk/c4t3d0
disk 29 0/3/1/0/4/1.4.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c4t4d0 /dev/rdsk/c4t4d0
disk 30 0/3/1/0/4/1.5.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c4t5d0 /dev/rdsk/c4t5d0
disk 31 0/3/1/0/4/1.8.0 sdisk CLAIMED DEVICE COMPAQ BF07284961
/dev/dsk/c4t8d0 /dev/rdsk/c4t8d0

 

# diskinfo /dev/rdsk/c3t3d0
SCSI describe of /dev/rdsk/c3t3d0:
vendor: COMPAQ 
product id: BF0728B26A 
type: direct access
size: 71132000 Kbytes
bytes per sector: 512

 

From here it seems the disk is normally detected by OS, so we try to re-active the VG again:

# vgchange -a e vgsmp
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c3t3d0":
Cross-device link
Activated volume group in Exclusive Mode.
vgchange: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
Volume group "vgsmp" has been successfully changed.



# vgdisplay -v vgsmp
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to 
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
--- Volume groups ---
VG Name /dev/vgsmp
VG Write Access read/write 
VG Status available, exclusive 
Max LV 255 
Cur LV 1 
Open LV 1 
Max PV 16 
Cur PV 6 
Act PV 5 
Max PE per PV 2170 
VGDA 10 
PE Size (Mbytes) 32 
Total PE 10850 
Alloc PE 2667 
Free PE 8183 
Total PVG 2 
Total Spare PVs 0 
Total Spare PVs in use 0

vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to 
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
--- Logical volumes ---
LV Name /dev/vgsmp/lvsmp
LV Status available/stale 
LV Size (Mbytes) 51200 
Current LE 1600 
Allocated PE 3200 
Used PV 5


--- Physical volumes ---
PV Name /dev/dsk/c3t1d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to 
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Status available 
Total PE 2170 
Free PE 1636 
Autoswitch On 
Proactive Polling On

PV Name /dev/dsk/c3t2d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to 
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Status available 
Total PE 2170 
Free PE 1637 
Autoswitch On 
Proactive Polling On

PV Name /dev/dsk/c1t1d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to 
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Status available 
Total PE 2170 
Free PE 1636 
Autoswitch On 
Proactive Polling On

PV Name /dev/dsk/c1t2d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to 
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Status available 
Total PE 2170 
Free PE 1637 
Autoswitch On 
Proactive Polling On

PV Name /dev/dsk/c1t3d0
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to 
this volume group
vgdisplay: Warning: couldn't query all of the physical volumes.
PV Status available 
Total PE 2170 
Free PE 1637 
Autoswitch On 
Proactive Polling On


--- Physical volume groups ---
PVG Name pvg0 
PV Name /dev/dsk/c3t1d0 
PV Name /dev/dsk/c3t2d0 
PV Name /dev/dsk/c3t3d0

PVG Name pvg1 
PV Name /dev/dsk/c1t1d0 
PV Name /dev/dsk/c1t2d0 
PV Name /dev/dsk/c1t3d0

 

I dont know what to do at this point, the path is already CLAIMED by OS, but seems VG cannot see it. We knew running Cluster package would not work either but we just ran it to get more logs:

########### Node "smp1": Starting package at Sun Sep 24 19:48:47 Irn 2017 ###########
Sep 24 19:48:47 - Node "smp1": Activating volume group vgsmp with vgchange -a e command.
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c3t3d0":
Cross-device link
Activated volume group in Exclusive Mode.
vgchange: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
Volume group "vgsmp" has been successfully changed.
Sep 24 19:48:47 - Node "smp1": Activating volume group vgdata with vgchange -a e command.
Activated volume group in Exclusive Mode.
Volume group "vgdata" has been successfully changed.
/tellin/smp: fuser: could not find file system mounted at /tellin/smp.
/tellin/smp:

Sep 24 19:48:48 - Node "smp1":
Checking filesystems:
/dev/vgsmp/lvsmp
vxfs fsck: file system had I/O error(s) on meta-data.
vxfs fsck: file system had I/O error(s) on user data.
log replay in progress
log replay failed to clean file system
file system is not clean, full fsck required
pass0 - checking structural files
pass1 - checking inode sanity and blocks
vxfs fsck: fsck read failure bno = 2050328, off = 0, len = 8192
file system check failure, aborting ...
ERROR: Function check_and_mount
ERROR: Failed to fsck one of the logical volumes.
Sep 24 19:49:46 - Node "smp1": Deactivating volume group vgsmp
vgchange: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Warning: couldn't query physical volume "/dev/dsk/c3t3d0":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
Deactivated volume group in Exclusive Mode.
Volume group "vgsmp" has been successfully changed.
Sep 24 19:49:46 - Node "smp1": Deactivating volume group vgdata
Deactivated volume group in Exclusive Mode.

 

Can anybody help me on this ? thanks.

 

# uname -a
HP-UX smp1 B.11.23 U ia64 0701204120 unlimited-user license

 

 

23 REPLIES
atari83
Advisor

Re: Volume group on MSA-30 degraded

somewhere i read that in this scenario i have to use vgreduce / lvredcue (and later extend commands) for replacing the faulty disks.

I tried to:

 vgreduce vgsmp /dev/dsk/c3t3d0

but it responds with error which says some physical extends are still in disk ..

in this case i tried:

lvreduce -m 0 -k /dev/vgsmp/lvsmp /dev/dsk/c3t3d0

but i almost received same error

is VG lost ?

 

Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

Don't do any reduce now! You may be in trouble after this.

Hold on ...


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

Thank you Torsten,

Please let me know if you need any log or information.

Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

The MSA30 has 2 cables to the first server and 2 cables to the second?

So it is the "MI" module (multi initiator).

Did you already replace something?

Please paste a full "ioscan -fn".

 

Are all 3 LED at the MSA30 I/O module green (steady/flashing)?

 

Your config look like this (but only 2 servers - let's say A+B and C+D is a single server)?

MSA30-MI-01.jpg


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

So it is the "MI" module (multi initiator).

Correct !

 

Did you already replace something?

We only tried to replace disks for :

0/3/1/0/4/0.3.0 -> c3t3d0
0/2/1/0/4/0.3.0 -> c1t3d0

Which any of our solutions didnt work, so we put back the old faulty disks into disk enclosures. so basically everything is like before.

atari83
Advisor

Re: Volume group on MSA-30 degraded

# ioscan -fn
Class        I  H/W Path        Driver          S/W State   H/W Type     Description
=====================================================================================
root         0                  root            CLAIMED     BUS_NEXUS    
ioa          0  0               sba             CLAIMED     BUS_NEXUS    System Bus Adapter (4030)
ba           0  0/0             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)
tty          0  0/0/1/0         rmp3f01         CLAIMED     INTERFACE    PCI class(255,0) (103c1303)
tty          1  0/0/1/1         rmp3f01         CLAIMED     INTERFACE    PCI SimpleComm (103c1302)
tty          2  0/0/1/2         asio0           CLAIMED     INTERFACE    PCI Serial (103c1048)
                               /dev/diag/mux2  /dev/mux2       /dev/tty2p0   
OO           0  0/0/2/0         UsbOhci         CLAIMED     INTERFACE    USB OHCI Interface
OO           1  0/0/2/0.0       UsbMiniBus      CLAIMED     INTERFACE    USB Composite Device
OO           2  0/0/2/0.0.0     UsbBootKeyboard CLAIMED     DEVICE       Virtual Keyboard
unknown     -1  0/0/2/0.0.1                     UNCLAIMED   UNKNOWN      OO Device Driver
OO           4  0/0/2/0.2       UsbMiniBus      CLAIMED     INTERFACE    USB Composite Device
OO           5  0/0/2/0.2.0     UsbBootKeyboard CLAIMED     DEVICE       HID Keyboard
OO           7  0/0/2/0.2.1     UsbBootMouse    CLAIMED     DEVICE       USB Boot Protocol Mouse
OO           3  0/0/2/1         UsbOhci         CLAIMED     INTERFACE    USB OHCI Interface
OO           8  0/0/2/1.0       UsbMiniBus      CLAIMED     INTERFACE    USB Composite Device
OO           9  0/0/2/1.0.0     UsbBulkOnlyMS   CLAIMED     DEVICE       USB Bulk Only
ext_bus      0  0/0/2/1.0.16    UsbScsiAdaptor  CLAIMED     INTERFACE    USB SCSI Stack Adaptor
target       0  0/0/2/1.0.16.0  tgt             CLAIMED     DEVICE       
disk         3  0/0/2/1.0.16.0.0  sdisk           CLAIMED     DEVICE       TEAC    DVD-ROM DW-224EV
                               /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0
target       1  0/0/2/1.0.16.7  tgt             CLAIMED     DEVICE       
ctl          7  0/0/2/1.0.16.7.0  sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c0t7d0
OO           6  0/0/2/2         UsbEhci         CLAIMED     INTERFACE    USB EHCI Interface
graphics     0  0/0/3/0         gvid_core       CLAIMED     INTERFACE    PCI Display (1002515e)
ba           1  0/1             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)
escsi_ctlr   0  0/1/1/0         sasd            CLAIMED     INTERFACE    HP  PCI/PCI-X SAS MPT Adapter
                               /dev/sasd0
ext_bus      7  0/1/1/0.0.0     sasd_vbus       CLAIMED     INTERFACE    SAS Device Interface
target      38  0/1/1/0.0.0.0   tgt             CLAIMED     DEVICE       
disk         1  0/1/1/0.0.0.0.0   sdisk           CLAIMED     DEVICE       HP      DG146BB976
                               /dev/dsk/c7t0d0     /dev/dsk/c7t0d0s1   /dev/dsk/c7t0d0s2   /dev/dsk/c7t0d0s3   /dev/rdsk/c7t0d0    /dev/rdsk/c7t0d0s1  /dev/rdsk/c7t0d0s2  /dev/rdsk/c7t0d0s3
target      39  0/1/1/0.0.0.1   tgt             CLAIMED     DEVICE       
disk         2  0/1/1/0.0.0.1.0   sdisk           CLAIMED     DEVICE       HP      DG146BB976
                               /dev/dsk/c7t1d0     /dev/dsk/c7t1d0s1   /dev/dsk/c7t1d0s2   /dev/dsk/c7t1d0s3   /dev/rdsk/c7t1d0    /dev/rdsk/c7t1d0s1  /dev/rdsk/c7t1d0s2  /dev/rdsk/c7t1d0s3
lan          0  0/1/2/0         igelan          CLAIMED     INTERFACE    HP PCI-X 1000Base-T Dual-port Built-in
lan          1  0/1/2/1         igelan          CLAIMED     INTERFACE    HP PCI-X 1000Base-T Dual-port Built-in
ba           2  0/2             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (12ee)
ba           3  0/2/1/0         PCItoPCI        CLAIMED     BUS_NEXUS    PCItoPCI Bridge
ext_bus      1  0/2/1/0/4/0     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt1
target       2  0/2/1/0/4/0.0   tgt             CLAIMED     DEVICE       
disk         4  0/2/1/0/4/0.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t0d0   /dev/rdsk/c1t0d0
target       3  0/2/1/0/4/0.1   tgt             CLAIMED     DEVICE       
disk         7  0/2/1/0/4/0.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t1d0   /dev/rdsk/c1t1d0
target       4  0/2/1/0/4/0.2   tgt             CLAIMED     DEVICE       
disk        10  0/2/1/0/4/0.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t2d0   /dev/rdsk/c1t2d0
target       5  0/2/1/0/4/0.3   tgt             CLAIMED     DEVICE       
disk        13  0/2/1/0/4/0.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t3d0   /dev/rdsk/c1t3d0
target       6  0/2/1/0/4/0.4   tgt             CLAIMED     DEVICE       
disk        16  0/2/1/0/4/0.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t4d0   /dev/rdsk/c1t4d0
target       7  0/2/1/0/4/0.5   tgt             CLAIMED     DEVICE       
disk        19  0/2/1/0/4/0.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t5d0   /dev/rdsk/c1t5d0
target       8  0/2/1/0/4/0.7   tgt             CLAIMED     DEVICE       
ctl          1  0/2/1/0/4/0.7.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c1t7d0
target       9  0/2/1/0/4/0.8   tgt             CLAIMED     DEVICE       
disk        23  0/2/1/0/4/0.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t8d0   /dev/rdsk/c1t8d0
ext_bus      2  0/2/1/0/4/1     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt2
target      10  0/2/1/0/4/1.0   tgt             CLAIMED     DEVICE       
disk         5  0/2/1/0/4/1.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t0d0   /dev/rdsk/c2t0d0
target      11  0/2/1/0/4/1.1   tgt             CLAIMED     DEVICE       
disk         9  0/2/1/0/4/1.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t1d0   /dev/rdsk/c2t1d0
target      12  0/2/1/0/4/1.2   tgt             CLAIMED     DEVICE       
disk        11  0/2/1/0/4/1.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t2d0   /dev/rdsk/c2t2d0
target      13  0/2/1/0/4/1.3   tgt             CLAIMED     DEVICE       
disk        14  0/2/1/0/4/1.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t3d0   /dev/rdsk/c2t3d0
target      14  0/2/1/0/4/1.4   tgt             CLAIMED     DEVICE       
disk        18  0/2/1/0/4/1.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t4d0   /dev/rdsk/c2t4d0
target      15  0/2/1/0/4/1.5   tgt             CLAIMED     DEVICE       
disk        21  0/2/1/0/4/1.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t5d0   /dev/rdsk/c2t5d0
target      16  0/2/1/0/4/1.7   tgt             CLAIMED     DEVICE       
ctl          2  0/2/1/0/4/1.7.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c2t7d0
target      17  0/2/1/0/4/1.8   tgt             CLAIMED     DEVICE       
disk        24  0/2/1/0/4/1.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t8d0   /dev/rdsk/c2t8d0
lan          2  0/2/1/0/6/0     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
lan          3  0/2/1/0/6/1     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
ba           4  0/3             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (12ee)
ba           5  0/3/1/0         PCItoPCI        CLAIMED     BUS_NEXUS    PCItoPCI Bridge
ext_bus      3  0/3/1/0/4/0     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt3
target      18  0/3/1/0/4/0.0   tgt             CLAIMED     DEVICE       
disk         6  0/3/1/0/4/0.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t0d0   /dev/rdsk/c3t0d0
target      19  0/3/1/0/4/0.1   tgt             CLAIMED     DEVICE       
disk         8  0/3/1/0/4/0.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t1d0   /dev/rdsk/c3t1d0
target      20  0/3/1/0/4/0.2   tgt             CLAIMED     DEVICE       
disk        12  0/3/1/0/4/0.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t2d0   /dev/rdsk/c3t2d0
target      21  0/3/1/0/4/0.3   tgt             CLAIMED     DEVICE       
disk        15  0/3/1/0/4/0.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t3d0   /dev/rdsk/c3t3d0
target      22  0/3/1/0/4/0.4   tgt             CLAIMED     DEVICE       
disk        17  0/3/1/0/4/0.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t4d0   /dev/rdsk/c3t4d0
target      23  0/3/1/0/4/0.5   tgt             CLAIMED     DEVICE       
disk        20  0/3/1/0/4/0.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t5d0   /dev/rdsk/c3t5d0
target      24  0/3/1/0/4/0.7   tgt             CLAIMED     DEVICE       
ctl          3  0/3/1/0/4/0.7.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c3t7d0
target      25  0/3/1/0/4/0.8   tgt             CLAIMED     DEVICE       
disk        22  0/3/1/0/4/0.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t8d0   /dev/rdsk/c3t8d0
target      34  0/3/1/0/4/0.15  tgt             CLAIMED     DEVICE       
ctl          8  0/3/1/0/4/0.15.0  sctl            CLAIMED     DEVICE       COMPAQ  PROLIANT 7L7E*DB
                               /dev/rscsi/c3t15d0
ext_bus      4  0/3/1/0/4/1     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt4
target      26  0/3/1/0/4/1.0   tgt             CLAIMED     DEVICE       
disk        25  0/3/1/0/4/1.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t0d0   /dev/rdsk/c4t0d0
target      27  0/3/1/0/4/1.1   tgt             CLAIMED     DEVICE       
disk        26  0/3/1/0/4/1.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t1d0   /dev/rdsk/c4t1d0
target      28  0/3/1/0/4/1.2   tgt             CLAIMED     DEVICE       
disk        27  0/3/1/0/4/1.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t2d0   /dev/rdsk/c4t2d0
target      29  0/3/1/0/4/1.3   tgt             CLAIMED     DEVICE       
disk        28  0/3/1/0/4/1.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t3d0   /dev/rdsk/c4t3d0
target      30  0/3/1/0/4/1.4   tgt             CLAIMED     DEVICE       
disk        29  0/3/1/0/4/1.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t4d0   /dev/rdsk/c4t4d0
target      31  0/3/1/0/4/1.5   tgt             CLAIMED     DEVICE       
disk        30  0/3/1/0/4/1.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t5d0   /dev/rdsk/c4t5d0
target      32  0/3/1/0/4/1.7   tgt             CLAIMED     DEVICE       
ctl          4  0/3/1/0/4/1.7.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c4t7d0
target      33  0/3/1/0/4/1.8   tgt             CLAIMED     DEVICE       
disk        31  0/3/1/0/4/1.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t8d0   /dev/rdsk/c4t8d0
lan          4  0/3/1/0/6/0     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
lan          5  0/3/1/0/6/1     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
ba           6  0/4             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (12ee)
ext_bus      5  0/4/1/0         mpt             CLAIMED     INTERFACE    SCSI Ultra320 A6961-60111
                               /dev/mpt5
target      35  0/4/1/0.7       tgt             CLAIMED     DEVICE       
ctl          5  0/4/1/0.7.0     sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c5t7d0
ext_bus      6  0/4/1/1         mpt             CLAIMED     INTERFACE    SCSI Ultra320 A6961-60111
                               /dev/mpt6
target      36  0/4/1/1.4       tgt             CLAIMED     DEVICE       
tape         1  0/4/1/1.4.0     stape           CLAIMED     DEVICE       HP      C7438A
                               /dev/rmt/1m            /dev/rmt/1mn           /dev/rmt/c6t4d0BEST    /dev/rmt/c6t4d0BESTn   /dev/rmt/c6t4d0DDS     /dev/rmt/c6t4d0DDSn  
                               /dev/rmt/1mb           /dev/rmt/1mnb          /dev/rmt/c6t4d0BESTb   /dev/rmt/c6t4d0BESTnb  /dev/rmt/c6t4d0DDSb    /dev/rmt/c6t4d0DDSnb 
target      37  0/4/1/1.7       tgt             CLAIMED     DEVICE       
ctl          6  0/4/1/1.7.0     sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c6t7d0
processor    0  120             processor       CLAIMED     PROCESSOR    Processor
processor    1  121             processor       CLAIMED     PROCESSOR    Processor
ba           7  250             pdh             CLAIMED     BUS_NEXUS    Core I/O Adapter
ipmi         0  250/0           ipmi            CLAIMED     INTERFACE    IPMI Controller
                               /dev/ipmi
tty          3  250/1           asio0           CLAIMED     INTERFACE    Built-in RS232C
                               /dev/diag/mux3  /dev/mux3       /dev/tty3p0   
acpi_node    0  250/2           acpi_node       CLAIMED     INTERFACE    Acpi Hardware
Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

These both disks are back into the original positions?

Did you pull them the same time?

Could you paste the first lines of a lvdisplay -v of a LVOL from that VG  (until the extend section begins)?

*
Physical Device Path...: 0/2/1/0/4/0.3.0
Device Class...........: Disk
Inquiry Vendor ID......: COMPAQ
Inquiry Product ID.....: BF07284961
Firmware Version.......: HPB5
Serial Number..........: J2VU88AC

*
Physical Device Path...: 0/3/1/0/4/0.3.0
Device Class...........: Disk
Inquiry Vendor ID......: COMPAQ
Inquiry Product ID.....: BF07284961
Firmware Version.......: HPB5
Serial Number..........: J2VU8M2C

 


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

Are all 3 LED at the MSA30 I/O module green (steady/flashing)?

The status is like this:

Upper Enclusore: (from left to right)

First LED (has no label): Steady Green

Second LED (A): Flashing Green

Third LED (B): Flashing Green

 

Lower Enclusore: (from left to right)

First LED (has no label): Flashing Green

Second LED (A): Flashing Green

Third LED (B): Flashing Green

 

atari83
Advisor

Re: Volume group on MSA-30 degraded

These both disks are back into the original positions?

yes, 100% sure.

 

Did you pull them the same time?

I think the time gap was something like 10mins

 

Could you paste the first lines of a lvdisplay -v of a LVOL from that VG  (until the extend section begins)?

 

# lvdisplay -v /dev/vgsmp/lvsmp | more
--- Logical volumes ---
LV Name                     /dev/vgsmp/lvsmp
VG Name                     /dev/vgsmp
LV Permission               read/write   
LV Status                   available/stale           
Mirror copies               1            
Consistency Recovery        MWC                 
Schedule                    parallel     
LV Size (Mbytes)            51200           
Current LE                  1600      
Allocated PE                3200        
Stripes                     0       
Stripe Size (Kbytes)        0                   
Bad block                   off          
Allocation                  PVG-strict/distributed
IO Timeout (Seconds)        default             

   --- Distribution of logical volume ---
   PV Name                 LE on PV  PE on PV  
   /dev/dsk/c3t1d0         534       534       
   /dev/dsk/c3t2d0         533       533       
   /dev/dsk/c3t3d0         533       533       
   /dev/dsk/c1t1d0         534       534       
   /dev/dsk/c1t2d0         533       533       
   /dev/dsk/c1t3d0         533       533       

 

 

Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

Please add let's say 5 more lines to the lvdisplay, just to be sure of the status.

 

The LED meaning is that's OK.

Status (power) of the module itself - Status Bus A - Status Bus B

 

 

And - please - an "ioscan -fn" of the second server.


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

Please add let's say 5 more lines to the lvdisplay, just to be sure of the status.

sure

   --- Logical extents ---
   LE    PV1                     PE1   Status 1 PV2                     PE2   Status 2 
   00000 /dev/dsk/c3t1d0         00000 current  /dev/dsk/c1t1d0         00000 current  
   00001 /dev/dsk/c3t2d0         00000 current  /dev/dsk/c1t2d0         00000 current  
   00002 /dev/dsk/c3t3d0         00000 current  /dev/dsk/c1t3d0         00000 current  
   00003 /dev/dsk/c3t1d0         00001 current  /dev/dsk/c1t1d0         00001 current  
   00004 /dev/dsk/c3t2d0         00001 current  /dev/dsk/c1t2d0         00001 current  
   00005 /dev/dsk/c3t3d0         00001 current  /dev/dsk/c1t3d0         00001 current  
   00006 /dev/dsk/c3t1d0         00002 current  /dev/dsk/c1t1d0         00002 current  
   00007 /dev/dsk/c3t2d0         00002 current  /dev/dsk/c1t2d0         00002 current  
   00008 /dev/dsk/c3t3d0         00002 current  /dev/dsk/c1t3d0         00002 current  
   00009 /dev/dsk/c3t1d0         00003 current  /dev/dsk/c1t1d0         00003 current  
   00010 /dev/dsk/c3t2d0         00003 current  /dev/dsk/c1t2d0         00003 current  
atari83
Advisor

Re: Volume group on MSA-30 degraded

And - please - an "ioscan -fn" of the second server.

 

# ioscan -fn
Class        I  H/W Path        Driver          S/W State   H/W Type     Description
=====================================================================================
root         0                  root            CLAIMED     BUS_NEXUS    
ioa          0  0               sba             CLAIMED     BUS_NEXUS    System Bus Adapter (4030)
ba           0  0/0             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)
tty          0  0/0/1/0         rmp3f01         CLAIMED     INTERFACE    PCI class(255,0) (103c1303)
tty          1  0/0/1/1         rmp3f01         CLAIMED     INTERFACE    PCI SimpleComm (103c1302)
tty          2  0/0/1/2         asio0           CLAIMED     INTERFACE    PCI Serial (103c1048)
                               /dev/diag/mux2  /dev/mux2       /dev/tty2p0   
OO           0  0/0/2/0         UsbOhci         CLAIMED     INTERFACE    USB OHCI Interface
OO           1  0/0/2/0.0       UsbMiniBus      CLAIMED     INTERFACE    USB Composite Device
OO           2  0/0/2/0.0.0     UsbBootKeyboard CLAIMED     DEVICE       Virtual Keyboard
unknown     -1  0/0/2/0.0.1                     UNCLAIMED   UNKNOWN      OO Device Driver
OO           4  0/0/2/0.2       UsbMiniBus      CLAIMED     INTERFACE    USB Composite Device
OO           5  0/0/2/0.2.0     UsbBootKeyboard CLAIMED     DEVICE       HID Keyboard
OO           7  0/0/2/0.2.1     UsbBootMouse    CLAIMED     DEVICE       USB Boot Protocol Mouse
OO           3  0/0/2/1         UsbOhci         CLAIMED     INTERFACE    USB OHCI Interface
OO           8  0/0/2/1.0       UsbMiniBus      CLAIMED     INTERFACE    USB Composite Device
OO           9  0/0/2/1.0.0     UsbBulkOnlyMS   CLAIMED     DEVICE       USB Bulk Only
ext_bus      0  0/0/2/1.0.16    UsbScsiAdaptor  CLAIMED     INTERFACE    USB SCSI Stack Adaptor
target       0  0/0/2/1.0.16.0  tgt             CLAIMED     DEVICE       
disk         3  0/0/2/1.0.16.0.0  sdisk           CLAIMED     DEVICE       TEAC    DVD-ROM DW-224EV
                               /dev/dsk/c0t0d0   /dev/rdsk/c0t0d0
target       1  0/0/2/1.0.16.7  tgt             CLAIMED     DEVICE       
ctl          7  0/0/2/1.0.16.7.0  sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c0t7d0
OO           6  0/0/2/2         UsbEhci         CLAIMED     INTERFACE    USB EHCI Interface
graphics     0  0/0/3/0         gvid_core       CLAIMED     INTERFACE    PCI Display (1002515e)
ba           1  0/1             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)
escsi_ctlr   0  0/1/1/0         sasd            CLAIMED     INTERFACE    HP  PCI/PCI-X SAS MPT Adapter
                               /dev/sasd0
ext_bus      7  0/1/1/0.0.0     sasd_vbus       CLAIMED     INTERFACE    SAS Device Interface
target      36  0/1/1/0.0.0.0   tgt             CLAIMED     DEVICE       
disk         1  0/1/1/0.0.0.0.0   sdisk           CLAIMED     DEVICE       HP      DG0146BALVN
                               /dev/dsk/c7t0d0     /dev/dsk/c7t0d0s1   /dev/dsk/c7t0d0s2   /dev/dsk/c7t0d0s3   /dev/rdsk/c7t0d0    /dev/rdsk/c7t0d0s1  /dev/rdsk/c7t0d0s2  /dev/rdsk/c7t0d0s3
target      37  0/1/1/0.0.0.1   tgt             CLAIMED     DEVICE       
disk         2  0/1/1/0.0.0.1.0   sdisk           CLAIMED     DEVICE       HP      DG146BB976
                               /dev/dsk/c7t1d0     /dev/dsk/c7t1d0s1   /dev/dsk/c7t1d0s2   /dev/dsk/c7t1d0s3   /dev/rdsk/c7t1d0    /dev/rdsk/c7t1d0s1  /dev/rdsk/c7t1d0s2  /dev/rdsk/c7t1d0s3
lan          0  0/1/2/0         igelan          CLAIMED     INTERFACE    HP PCI-X 1000Base-T Dual-port Built-in
lan          1  0/1/2/1         igelan          CLAIMED     INTERFACE    HP PCI-X 1000Base-T Dual-port Built-in
ba           2  0/2             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (12ee)
ba           3  0/2/1/0         PCItoPCI        CLAIMED     BUS_NEXUS    PCItoPCI Bridge
ext_bus      1  0/2/1/0/4/0     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt1
target       2  0/2/1/0/4/0.0   tgt             CLAIMED     DEVICE       
disk        25  0/2/1/0/4/0.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t0d0   /dev/rdsk/c1t0d0
target       3  0/2/1/0/4/0.1   tgt             CLAIMED     DEVICE       
disk        26  0/2/1/0/4/0.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t1d0   /dev/rdsk/c1t1d0
target       4  0/2/1/0/4/0.2   tgt             CLAIMED     DEVICE       
disk        27  0/2/1/0/4/0.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t2d0   /dev/rdsk/c1t2d0
target       5  0/2/1/0/4/0.3   tgt             CLAIMED     DEVICE       
disk        28  0/2/1/0/4/0.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t3d0   /dev/rdsk/c1t3d0
target       6  0/2/1/0/4/0.4   tgt             CLAIMED     DEVICE       
disk        29  0/2/1/0/4/0.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t4d0   /dev/rdsk/c1t4d0
target       7  0/2/1/0/4/0.5   tgt             CLAIMED     DEVICE       
disk        30  0/2/1/0/4/0.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t5d0   /dev/rdsk/c1t5d0
target       8  0/2/1/0/4/0.6   tgt             CLAIMED     DEVICE       
ctl          8  0/2/1/0/4/0.6.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c1t6d0
target       9  0/2/1/0/4/0.8   tgt             CLAIMED     DEVICE       
disk        31  0/2/1/0/4/0.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c1t8d0   /dev/rdsk/c1t8d0
ext_bus      2  0/2/1/0/4/1     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt2
target      10  0/2/1/0/4/1.0   tgt             CLAIMED     DEVICE       
disk         4  0/2/1/0/4/1.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t0d0   /dev/rdsk/c2t0d0
target      11  0/2/1/0/4/1.1   tgt             CLAIMED     DEVICE       
disk         9  0/2/1/0/4/1.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t1d0   /dev/rdsk/c2t1d0
target      12  0/2/1/0/4/1.2   tgt             CLAIMED     DEVICE       
disk        11  0/2/1/0/4/1.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t2d0   /dev/rdsk/c2t2d0
target      13  0/2/1/0/4/1.3   tgt             CLAIMED     DEVICE       
disk        14  0/2/1/0/4/1.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t3d0   /dev/rdsk/c2t3d0
target      14  0/2/1/0/4/1.4   tgt             CLAIMED     DEVICE       
disk        17  0/2/1/0/4/1.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t4d0   /dev/rdsk/c2t4d0
target      15  0/2/1/0/4/1.5   tgt             CLAIMED     DEVICE       
disk        20  0/2/1/0/4/1.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t5d0   /dev/rdsk/c2t5d0
target      16  0/2/1/0/4/1.6   tgt             CLAIMED     DEVICE       
ctl          9  0/2/1/0/4/1.6.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c2t6d0
target      17  0/2/1/0/4/1.8   tgt             CLAIMED     DEVICE       
disk        23  0/2/1/0/4/1.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c2t8d0   /dev/rdsk/c2t8d0
lan          2  0/2/1/0/6/0     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
lan          3  0/2/1/0/6/1     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
ba           4  0/3             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (12ee)
ba           5  0/3/1/0         PCItoPCI        CLAIMED     BUS_NEXUS    PCItoPCI Bridge
ext_bus      3  0/3/1/0/4/0     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt3
target      18  0/3/1/0/4/0.0   tgt             CLAIMED     DEVICE       
disk         5  0/3/1/0/4/0.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t0d0   /dev/rdsk/c3t0d0
target      19  0/3/1/0/4/0.1   tgt             CLAIMED     DEVICE       
disk         7  0/3/1/0/4/0.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t1d0   /dev/rdsk/c3t1d0
target      20  0/3/1/0/4/0.2   tgt             CLAIMED     DEVICE       
disk        12  0/3/1/0/4/0.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t2d0   /dev/rdsk/c3t2d0
target      38  0/3/1/0/4/0.3   tgt             CLAIMED     DEVICE       
disk        15  0/3/1/0/4/0.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t3d0   /dev/rdsk/c3t3d0
target      21  0/3/1/0/4/0.4   tgt             CLAIMED     DEVICE       
disk        18  0/3/1/0/4/0.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t4d0   /dev/rdsk/c3t4d0
target      22  0/3/1/0/4/0.5   tgt             CLAIMED     DEVICE       
disk        21  0/3/1/0/4/0.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t5d0   /dev/rdsk/c3t5d0
target      23  0/3/1/0/4/0.6   tgt             CLAIMED     DEVICE       
ctl         10  0/3/1/0/4/0.6.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c3t6d0
target      24  0/3/1/0/4/0.8   tgt             CLAIMED     DEVICE       
disk        24  0/3/1/0/4/0.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c3t8d0   /dev/rdsk/c3t8d0
target      33  0/3/1/0/4/0.15  tgt             CLAIMED     DEVICE       
ctl         12  0/3/1/0/4/0.15.0  sctl            CLAIMED     DEVICE       COMPAQ  PROLIANT 7L7E*DB
ext_bus      4  0/3/1/0/4/1     mpt             CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X U320 SCSI 2-port U320 SCSI/2-port 1000B-T Combo Adapter
                               /dev/mpt4
target      25  0/3/1/0/4/1.0   tgt             CLAIMED     DEVICE       
disk         6  0/3/1/0/4/1.0.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t0d0   /dev/rdsk/c4t0d0
target      26  0/3/1/0/4/1.1   tgt             CLAIMED     DEVICE       
disk         8  0/3/1/0/4/1.1.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t1d0   /dev/rdsk/c4t1d0
target      27  0/3/1/0/4/1.2   tgt             CLAIMED     DEVICE       
disk        10  0/3/1/0/4/1.2.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t2d0   /dev/rdsk/c4t2d0
target      28  0/3/1/0/4/1.3   tgt             CLAIMED     DEVICE       
disk        13  0/3/1/0/4/1.3.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t3d0   /dev/rdsk/c4t3d0
target      29  0/3/1/0/4/1.4   tgt             CLAIMED     DEVICE       
disk        16  0/3/1/0/4/1.4.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t4d0   /dev/rdsk/c4t4d0
target      30  0/3/1/0/4/1.5   tgt             CLAIMED     DEVICE       
disk        19  0/3/1/0/4/1.5.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t5d0   /dev/rdsk/c4t5d0
target      31  0/3/1/0/4/1.6   tgt             CLAIMED     DEVICE       
ctl         11  0/3/1/0/4/1.6.0   sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c4t6d0
target      32  0/3/1/0/4/1.8   tgt             CLAIMED     DEVICE       
disk        22  0/3/1/0/4/1.8.0   sdisk           CLAIMED     DEVICE       COMPAQ  BF07284961
                               /dev/dsk/c4t8d0   /dev/rdsk/c4t8d0
lan          4  0/3/1/0/6/0     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
lan          5  0/3/1/0/6/1     iether          CLAIMED     INTERFACE    HP AB290-60001 PCI/PCI-X 1000Base-T 2-port U320 SCSI/2-port 1000B-T Combo Adapter
ba           6  0/4             lba             CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (12ee)
ext_bus      5  0/4/1/0         mpt             CLAIMED     INTERFACE    SCSI Ultra320 A6961-60111
                               /dev/mpt5
target      34  0/4/1/0.7       tgt             CLAIMED     DEVICE       
ctl          5  0/4/1/0.7.0     sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c5t7d0
ext_bus      6  0/4/1/1         mpt             CLAIMED     INTERFACE    SCSI Ultra320 A6961-60111
                               /dev/mpt6
target      35  0/4/1/1.7       tgt             CLAIMED     DEVICE       
ctl          6  0/4/1/1.7.0     sctl            CLAIMED     DEVICE       Initiator
                               /dev/rscsi/c6t7d0
processor    0  120             processor       CLAIMED     PROCESSOR    Processor
processor    1  121             processor       CLAIMED     PROCESSOR    Processor
ba           7  250             pdh             CLAIMED     BUS_NEXUS    Core I/O Adapter
ipmi         0  250/0           ipmi            CLAIMED     INTERFACE    IPMI Controller
                               /dev/ipmi
tty          3  250/1           asio0           CLAIMED     INTERFACE    Built-in RS232C
                               /dev/diag/mux3  /dev/mux3       /dev/tty3p0   
acpi_node    0  250/2           acpi_node       CLAIMED     INTERFACE    Acpi Hardware

 

Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

Let's see where the "stale" extends are

 

# lvdisplay -v /dev/vgsmp/lvsmp | grep stale

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

Let's see where the "stale" extends are

 

# lvdisplay -v /dev/vgsmp/lvsmp | grep stale
LV Status                   available/stale           
   00026 /dev/dsk/c3t3d0         00008 stale    /dev/dsk/c1t3d0         00008 current  
   00044 /dev/dsk/c3t3d0         00014 stale    /dev/dsk/c1t3d0         00014 current  
   00062 /dev/dsk/c3t3d0         00020 stale    /dev/dsk/c1t3d0         00020 current  
   00065 /dev/dsk/c3t3d0         00021 stale    /dev/dsk/c1t3d0         00021 current  
   00068 /dev/dsk/c3t3d0         00022 stale    /dev/dsk/c1t3d0         00022 current  
   00086 /dev/dsk/c3t3d0         00028 stale    /dev/dsk/c1t3d0         00028 current  
   00089 /dev/dsk/c3t3d0         00029 stale    /dev/dsk/c1t3d0         00029 current  
   00098 /dev/dsk/c3t3d0         00032 stale    /dev/dsk/c1t3d0         00032 current  
   00104 /dev/dsk/c3t3d0         00034 stale    /dev/dsk/c1t3d0         00034 current  
   00113 /dev/dsk/c3t3d0         00037 stale    /dev/dsk/c1t3d0         00037 current  
   00116 /dev/dsk/c3t3d0         00038 stale    /dev/dsk/c1t3d0         00038 current  
   00122 /dev/dsk/c3t3d0         00040 stale    /dev/dsk/c1t3d0         00040 current  
   00125 /dev/dsk/c3t3d0         00041 stale    /dev/dsk/c1t3d0         00041 current  
   00128 /dev/dsk/c3t3d0         00042 stale    /dev/dsk/c1t3d0         00042 current  
   00164 /dev/dsk/c3t3d0         00054 stale    /dev/dsk/c1t3d0         00054 current  
   00275 /dev/dsk/c3t3d0         00091 stale    /dev/dsk/c1t3d0         00091 current  
   00281 /dev/dsk/c3t3d0         00093 stale    /dev/dsk/c1t3d0         00093 current  
   00284 /dev/dsk/c3t3d0         00094 stale    /dev/dsk/c1t3d0         00094 current  
   00287 /dev/dsk/c3t3d0         00095 stale    /dev/dsk/c1t3d0         00095 current  
   00290 /dev/dsk/c3t3d0         00096 stale    /dev/dsk/c1t3d0         00096 current  
   00299 /dev/dsk/c3t3d0         00099 stale    /dev/dsk/c1t3d0         00099 current  
   00356 /dev/dsk/c3t3d0         00118 stale    /dev/dsk/c1t3d0         00118 current  
   00380 /dev/dsk/c3t3d0         00126 stale    /dev/dsk/c1t3d0         00126 current  
   00422 /dev/dsk/c3t3d0         00140 stale    /dev/dsk/c1t3d0         00140 current  
   00434 /dev/dsk/c3t3d0         00144 stale    /dev/dsk/c1t3d0         00144 current  
   00446 /dev/dsk/c3t3d0         00148 stale    /dev/dsk/c1t3d0         00148 current  
   00449 /dev/dsk/c3t3d0         00149 stale    /dev/dsk/c1t3d0         00149 current  
   00452 /dev/dsk/c3t3d0         00150 stale    /dev/dsk/c1t3d0         00150 current  
   00455 /dev/dsk/c3t3d0         00151 stale    /dev/dsk/c1t3d0         00151 current  
   00464 /dev/dsk/c3t3d0         00154 stale    /dev/dsk/c1t3d0         00154 current  
   00467 /dev/dsk/c3t3d0         00155 stale    /dev/dsk/c1t3d0         00155 current  
   00479 /dev/dsk/c3t3d0         00159 stale    /dev/dsk/c1t3d0         00159 current  
   00548 /dev/dsk/c3t3d0         00182 stale    /dev/dsk/c1t3d0         00182 current  
   00551 /dev/dsk/c3t3d0         00183 stale    /dev/dsk/c1t3d0         00183 current  
   00584 /dev/dsk/c3t3d0         00194 stale    /dev/dsk/c1t3d0         00194 current  
   00599 /dev/dsk/c3t3d0         00199 stale    /dev/dsk/c1t3d0         00199 current  
   00623 /dev/dsk/c3t3d0         00207 stale    /dev/dsk/c1t3d0         00207 current  
   00626 /dev/dsk/c3t3d0         00208 stale    /dev/dsk/c1t3d0         00208 current  
   00650 /dev/dsk/c3t3d0         00216 stale    /dev/dsk/c1t3d0         00216 current  
   00668 /dev/dsk/c3t3d0         00222 stale    /dev/dsk/c1t3d0         00222 current  
   00683 /dev/dsk/c3t3d0         00227 current  /dev/dsk/c1t3d0         00227 stale    
   00686 /dev/dsk/c3t3d0         00228 stale    /dev/dsk/c1t3d0         00228 current  
   00719 /dev/dsk/c3t3d0         00239 stale    /dev/dsk/c1t3d0         00239 current  
   00728 /dev/dsk/c3t3d0         00242 stale    /dev/dsk/c1t3d0         00242 current  
   00812 /dev/dsk/c3t3d0         00270 stale    /dev/dsk/c1t3d0         00270 current  
   00815 /dev/dsk/c3t3d0         00271 current  /dev/dsk/c1t3d0         00271 stale    
   00821 /dev/dsk/c3t3d0         00273 current  /dev/dsk/c1t3d0         00273 stale    
   00824 /dev/dsk/c3t3d0         00274 current  /dev/dsk/c1t3d0         00274 stale    
Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

For the replacement of

/dev/dsk/c3t3d0

you did, you just pulled the disk, inserted the new disk and did a "vgchange -a e ..."?

No "vgcfgrestore ..." to the new disk?


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

This is a bad problem:

00668 /dev/dsk/c3t3d0         00222 stale    /dev/dsk/c1t3d0         00222 current  
   00683 /dev/dsk/c3t3d0         00227 current  /dev/dsk/c1t3d0         00227 stale    
   00686 /dev/dsk/c3t3d0         00228 stale    /dev/dsk/c1t3d0         00228 current  

 

 

You have current and stale extends on both disks ... how about backup/restore - is it possible?


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

you did, you just pulled the disk, inserted the new disk and did a "vgchange -a e ..."?

Exactly !

No "vgcfgrestore ..." to the new disk?

Yes, didnt run vgcfgrestore command on it

atari83
Advisor

Re: Volume group on MSA-30 degraded

You have current and stale extends on both disks ... how about backup/restore - is it possible?

Yes, i noticed that ..

Well, the problem is the local engineer here didnt keep the backup on server local disk not or on tape. so there is no data backup to restore ..

Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

You could do a "vgsync" and check, if you get less stales.

 

The situation is this:

 

You have 3 disks with distributed extends, this means

extend1 -> disk1

extend2 -> disk2

extend3 -> disk3

extend4 -> disk1

...

and this is mirrored to another set of disks.

But now you have stale extends on the original and the mirrored disk, so you cannot change one without data loss.

So your only choices are:

backup data, repair the VG, restore data

*or*

insert a new set of 3 disks and create a second mirror (and hope all data can be read).


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

You could do a "vgsync" and check, if you get less stales.

vgsync: Couldn't re-synchronize stale partitions of the logical volume:
I/O error
vgsync: Couldn't resynchronize logical volume "/dev/vgsmp/lvsmp".
vgsync: Couldn't resynchronize volume group "vgsmp".

But about your solutions, sorry to ask but i couldnt get the idea:

backup data, repair the VG, restore data

There is no backup from that VG/LV , and also when i do "vgchange -a e" still cannot mount or access to filesystem because of some I/O errors. tried to use fsck but no chance. so this solution should be ignored right ?

insert a new set of 3 disks and create a second mirror (and hope all data can be read).

Could you explain more about this one ? I've no idea how to implement this :)

 

 

Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

If you cannot read the data and you don't have a backup, so there is probably no solution without data loss.

In other words - data is already lost.

Obviously you destroyed it finally by pulling those 2 disks.


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
atari83
Advisor

Re: Volume group on MSA-30 degraded

Thank you Torsten for following up the case with me. to be honest it wasnt me who pulled the disks out, i just ignored to explain that part to shorten the explanation. anyhow, it is what i expected. thanks.

Torsten.
Acclaimed Contributor

Re: Volume group on MSA-30 degraded

well, the (old) MSA30 is known from the proliant world, normally behind a smart array which does an automatic rebuild after a disk swap, but not in this case. here it is under LVM control. remind your peers, first ask, if not sure what do to. sorry.

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!