syslog error

[kunjuttan]

Hi guys,
I have a problem in my Server.In syslog it is showing the following error.Can anybody help me why it is showing??
Oct 28 13:51:27 bilqa vmunix: LVM: WARNING: VG 128 0x003000: LV 1: Some I/O requests to this LV are waiting
Oct 28 13:51:27 bilqa vmunix: indefinitely for an unavailable PV. These requests will be queued until
Oct 28 13:51:27 bilqa vmunix: the PV becomes available (or a timeout is specified for the LV).
Oct 28 13:51:28 bilqa vmunix: LVM: NOTICE: VG 128 0x003000: LV 1: All I/O requests to this LV that were
Oct 28 13:51:28 bilqa vmunix: waiting indefinitely for an unavailable PV have now completed.
Oct 28 13:51:58 bilqa vmunix: LVM: WARNING: VG 128 0x003000: LV 1: Some I/O requests to this LV are waiting
Oct 28 13:51:58 bilqa vmunix: indefinitely for an unavailable PV. These requests will be queued until
Oct 28 13:51:58 bilqa vmunix: the PV becomes available (or a timeout is specified for the LV).
Oct 28 13:51:59 bilqa vmunix: LVM: NOTICE: VG 128 0x003000: LV 1: All I/O requests to this LV that were
Oct 28 13:51:59 bilqa vmunix: waiting indefinitely for an unavailable PV have now completed.

OS Version is 11.31

[Kapil Jha · Accepted solution]

do ioscan -fnC disk and check if all are in claimed state.
in vgdislay -v vg_name
check if all PV are there and intact.
#df -k

should give you output and should not hung.

check pvdisplay pv_name
and let us know TIme out value.

BR,
Kapil+

[madhuchakkaravarthy]

hi

check with ll /dev/*/group to find the particular vg and check the corresponding lv.

and can u post the dmesg- output.

regards

MC

[kunjuttan]

LVM: NOTICE: VG 128 0x001000: LV 5: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 3: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 3: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x001000: LV 2: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x001000: LV 2: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 3: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 3: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 6: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 6: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 4: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 4: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 5: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 5: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 8: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 8: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 5: Some I/O requests to this LV are waiting
indefinitely for an unavailable PV. These requests will be queued until
the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 5: All I/O requests to this LV that were
waiting indefinitely for an unavailable PV have now completed.
LVM: WARNING: VG 128 0x000000: LV 5: Some I/O requests to this LV are waiting

[Kapil Jha]

did you check the commands i asked earlier,

I suppose those are SAN disk in volume and most probably those LUNs are nto visible.

First basic thing is to run
ioscan -fnC disk

BR,
Kapil+

[kunjuttan]

Yah.. I already did ioscan and it is showing all claimed.df -k is giving output.io timeout is showing like default..

[Kapil Jha]

Are you using powerpath?
are these san devices?

from error it seems
--------------------
LVM: WARNING: VG 128 0x000000: LV 8: Some I/O requests to this LV are waiting indefinitely for an unavailable PV. These requests will be queued until the PV becomes available (or a timeout is specified for the LV).
LVM: NOTICE: VG 128 0x000000: LV 8: All I/O requests to this LV that were waiting indefinitely for an unavailable PV have now completed.
--------------------
PV is getting unavailable and then getting back.

so requested are completing.
Need to check SAN,FC,multipath software.
Any other error in dmesg apart from this.

BR,
Kapil+

[kunjuttan]

Hi,
May I know what is powerpath..here all are san devices.but how can i able to know which disk is having problem because all are showing claimed and vgdisplay also showing all vgs available..

[Kapil Jha]

powerpath is a multipathing software from EMC.

Do you have Storage team you can seek there help.

i think you need to see syslog.log file properly there should be some error messages for the LUNs availability.

Need more error message to dig more.

BR,
Kapil+

[P Arumugavel]

hi,

Is there any LUN was deleted in SAN. Because deleting LUNs in an HP StorageWorks Virtual Array (VA) may cause host I/O timeouts to occur.

Browse the below link:
http://bizsupport1.austin.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&taskId=110&prodSeriesId=3542195&prodTypeId=12169&prodSeriesId=3542195&objectID=c01820153

[madhuchakkaravarthy]

hi

storage name.

check device power.
check disk cable it may be loose.

check for any alerts in storage.

have u done any changes in host machine or in storage.

regards

MC

[eric lipede_1]

Hi
ok - the message seems to refer to a VG thats unavailable ...so....

1. What discs belong to VG 128 0x003000 ?
2. what lv's belong to that VG ?
3. what does strings have to say about this?
4. when did the messaging start and what was
done to the system before?
5. a PV is unavailable ...where is this from..a SAN, JBOD etc ?

[kunjuttan]

How can I check this particular VG 128 0x003000 corresponds to which pv and lv??

[cnb]

See page 9 to identify the disks in question:

http://bizsupport2.austin.hp.com/bc/docs/support/SupportManual/c01911837/c01911837.pdf

Rgds,

[P Arumugavel]

hi,

To map this error 0x003000 to a specific disk, search the lower 6 digit of this from below command,

# ll /dev/*dsk | grep 003000

Regds \ Vel

[P Arumugavel]

or put as below

# ll /dev/*disk | grep 003000

Rgds \ vel

[kunjuttan]

done