san disks inaccessible after system restart

[QASCO]

Hello experts,

i rebooted the machine & was not able to access 3 volume groups despite the structure of volume groups are there in /etc/lvmtab also device files do exist in /dev. below some output

mount -a
vxfs mount: Cannot open /dev/vg06/dvp: No such device or address
vxfs mount: Cannot open /dev/vg05/vis: No such device or address
vxfs mount: Cannot open /dev/vg04/lvol1: No such device or address
mount: /dev/vg00/lvol8 is already mounted on /var
mount: /dev/vg00/lvol7 is already mounted on /usr
mount: /dev/vg00/lvol6 is already mounted on /tmp
mount: /dev/vg00/lvol5 is already mounted on /opt
mount: /dev/vg00/lvol4 is already mounted on /home
mount: /dev/vg00/lvol1 is already mounted on /stand

==========================================
strings /etc/lvmtab
/dev/vg00
/dev/dsk/c1t2d0
/dev/dsk/c2t2d0
/dev/vg04
/dev/dsk/c9t0d5
/dev/vg05
/dev/dsk/c9t0d6
/dev/vg06
/dev/dsk/c9t0d7


Any idea how to get those volume group back.

Regards

[Muthukumar_5]

May be /etc/lvmtab file is corrupted. You have to restore as,

# vgrestore vgname

~regards

[Muthukumar_5]

sorry. Before that,

# ioscan -fnC disk

what is saying for /dev/dsk/c9t0d7, /dev/dsk/c9t0d6 and /dev/dsk/c9t0d5?

~regards

[QASCO]

it says the following :-

ioscan -funC disk
Class I H/W Path Driver S/W State H/W Type Description
==========================================================================
disk 0 0/0/1/1.2.0 sdisk CLAIMED DEVICE HP 36.4GST336753LC
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0
disk 1 0/0/2/0.2.0 sdisk CLAIMED DEVICE HP 36.4GST336753LC
/dev/dsk/c2t2d0 /dev/rdsk/c2t2d0
disk 2 0/0/2/1.2.0 sdisk CLAIMED DEVICE HP DVD-ROM 305
/dev/dsk/c3t2d0 /dev/rdsk/c3t2d0
disk 3 255/255/0.0.0 sdisk CLAIMED DEVICE HSV110 (C)COMPAQ LUN 0x600508B4000104AF0000E0
0000360000
/dev/dsk/c9t0d0 /dev/rdsk/c9t0d0
disk 11 255/255/0.0.1 sdisk CLAIMED DEVICE HSV110 (C)COMPAQ LUN 0x600508B4000104AF000100
0000540000
/dev/dsk/c9t0d1 /dev/rdsk/c9t0d1
disk 4 255/255/0.0.2 sdisk CLAIMED DEVICE HSV110 (C)COMPAQ LUN 0x600508B4000104AF0000E0
00008A0000
/dev/dsk/c9t0d2 /dev/rdsk/c9t0d2
disk 5 255/255/0.0.3 sdisk CLAIMED DEVICE HSV110 (C)COMPAQ LUN 0x600508B4000104AF0000E0
0000BB0000
/dev/dsk/c9t0d3 /dev/rdsk/c9t0d3
disk 6 255/255/0.0.4 sdisk CLAIMED DEVICE HSV110 (C)COMPAQ LUN 0x600508B4000104AF000100
0000390000
/dev/dsk/c9t0d4 /dev/rdsk/c9t0d4
disk 7 255/255/0.0.5 sdisk CLAIMED DEVICE HSV110 (C)COMPAQ LUN 0x600508B4000104AF000100
0000410000
/dev/dsk/c9t0d5 /dev/rdsk/c9t0d5
#
regards

[Arunvijai_4]

What abt # diskinfo says ? Post it as well

-Arun

[QASCO]

diskinfo output:

diskinfo: can't open c9t0d7: Invalid argument
diskinfo: can't open c9t0d6: Invalid argument
SCSI describe of c9t0d5:
vendor: COMPAQ
product id: HSV110 (C)COMPAQ
type: direct access
size: 125829120 Kbytes
bytes per sector: 512

regards,

[Muthukumar_5]

If all device files /dev/vg06/dvp, /dev/vg05/vis and /dev/vg04/lvol1 are there then,

# vgdisplay -v /dev/vg06
# vgdisplay -v /dev/vg05
# vgdisplay -v /dev/vg04

Is it having those lvol's. Are these vg's having more lvol's. Are they active.?

# mount -F vxfs /dev/vg06/dvp

what is saying if above are passed.

~regards

[Kent Ostby]

The "Cannot open" and "No such device or address" means that the VGs are probably not active.

Try manually activating the vgs:

vgchange -a y /dev/vg04
vgchange -a y /dev/vg05
vgchange -a y /dev/vg06
mount -a

[QASCO]

those vg's were active before system restart & every volume group has one lv.

vgdisplay -v vg06
vgdisplay: Volume group not activated.
vgdisplay: Cannot display volume group "vg06".
# vgdisplay vg05
vgdisplay: Volume group not activated.
vgdisplay: Cannot display volume group "vg05".
# vgdisplay vg04
vgdisplay: Volume group not activated.
vgdisplay: Cannot display volume group "vg04".
# vgchange -a y vg06
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c9t0d7":
Invalid argument
vgchange: Warning: couldn't query physical volume "/dev/dsk/c9t0d7":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "vg06":
Quorum not present, or some physical volume(s) are missing.

# vgchange -a y vg05
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c9t0d6":
Invalid argument
vgchange: Warning: couldn't query physical volume "/dev/dsk/c9t0d6":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "vg05":
Quorum not present, or some physical volume(s) are missing.

# vgchange -a y vg04
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c9t0d5":
Cross-device link
vgchange: Warning: couldn't query physical volume "/dev/dsk/c9t0d5":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
vgchange: Couldn't activate volume group "vg04":
Quorum not present, or some physical volume(s) are missing.

Regards

[mirco_1]

hi,

see
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=422226

DM.

[Muthukumar_5]

May be few disks are lost?

# vgchange -a y -q n -p vg04
# vgchange -a y -q n -p vg05
# vgchange -a y -q n -p vg06

what is it saying?

~regards

[QASCO]

vgchange -a y -q n -p vg05
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c9t0d6":
Invalid argument
vgchange: Could not attach to volume group "vg05" all of
its physical volumes.
vgchange: Warning: couldn't query physical volume "/dev/dsk/c9t0d6":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.
# vgchange -a y -q n -p vg06
vgchange: Warning: Couldn't attach to the volume group physical volume "/dev/dsk/c9t0d7":
Invalid argument
vgchange: Could not attach to volume group "vg06" all of
its physical volumes.
vgchange: Warning: couldn't query physical volume "/dev/dsk/c9t0d7":
The specified path does not correspond to physical volume attached to
this volume group
vgchange: Warning: couldn't query all of the physical volumes.


Regards

[NathanK]

Just throwing out an idea here...Not sure what is causing your problems. You may want to make sure that your host system is logged into the SAN properly first. If not have your SAN admin disable and re-enable the port for them to log in.

[Alzhy]

You are missing 2 of your 3 EVA disks: c9t0d5 and c9t0d6.

What does spmgr display tell you?

Confirm with your EVA person what LUNs your server should be seeing and ask why if you are indeed missing em.

[Sยภเl Kย๓คг]

Hi,

it's very clear that some of the luns are not visible for the system. Consult your support personnel for further help.

It is recommended that you can have small script that run cron in a week interval to take conf info abt the system. This will help you to identify the problems on any failures easily..

Enclosed a script like purpose which we are using here.
Regards,
Sunil

[Devender Khatana]

Hi,

The error here is because the disks are missing or their device files has changed. Out of three disks configured in vg04, vg05 and vg06 two disks are not available at all and the third one which is available is not probably the one which was their earlier.

The problem seems to be due to device file changing only. SAN administrators shall be able to say clearly on this.How many LUNs are allocated ? How many LUNs are visible ?

If the device files changes are confirmed, you might be required to reimport the VGs again to have your file systems back.

Alongwith response from SAN team you can see any old config files for disk device files prior to reboot for other LUNs. I mean to say now you have /dev/dsk/c9t0d1-d5 available but have configured only three /dev/dsk/c9t0d5 -7. So check prior to reboot you had 7 LUNs allocated or only 5 which are available now as well but with different names.

HTH,
Devender