need help urgent......

[djoshi]

Hi All,

We have HP-9000 having HP-UNIX 10.01 OS. With this system a Disk array is attached.

Today I found a error on message logs which says below.

Oct 17 00:25
Disk at 10/16/12.6.0 is not responding. Check device, power, and cables.
LVM: PV 0 has been returned to vg[2].
Disk at 10/16/4.5.1 is not responding. Check device, power, and cables.
Disk at 10/16/4.5.1 is not responding. Check device, power, and cables.
LVM: PV 0 has been returned to vg[3].

No fault status showing on the disk array.
I can query to all the volume group using vgdisplay -v /dev/xxxx and getting no errors.
Secondly I can see the disk and getting the output with no erros with diskinfo -v /dev/rdsk/xxxx command.

I found the above disk path is associated with vg03 and vg02 and I can query to both vg03 and vg02.

In this server informix database is ruuning and all the disk is online.

Please advise me about this message and how and what to troubleshoot? I am attaching the output of vgdisplay and diskinfo command.

Any help is highly appreciated.

[djoshi]

Also, I am attaching the output of diskinfo and the mention on error logs is showing a claimed state.
On syslog error

Disk at 10/16/12.6.0 is not responding. Check device, power, and cables.
LVM: PV 0 has been returned to vg[2].
Disk at 10/16/4.5.1 is not responding. Check device, power, and cables.
Disk at 10/16/4.5.1 is not responding. Check device, power, and cables.
LVM: PV 0 has been returned to vg[3].
~
Output of diskinfo


Class I H/W Path Driver S/W State H/W Type Description
==================================================================
disk 14 10/0.2.0 sdisk CLAIMED DEVICE SEAGATE ST15150W
/dev/dsk/c0t2d0 /dev/rdsk/c0t2d0
disk 0 10/0.5.0 sdisk CLAIMED DEVICE SEAGATE ST34371W
/dev/dsk/c0t5d0 /dev/rdsk/c0t5d0
disk 1 10/0.6.0 sdisk CLAIMED DEVICE SEAGATE ST34371W
/dev/dsk/c0t6d0 /dev/rdsk/c0t6d0
disk 12 10/0.12.0 sdisk CLAIMED DEVICE SEAGATE ST15150W
/dev/dsk/c0t12d0 /dev/rdsk/c0t12d0
disk 2 10/4/4.6.0 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c2t6d0 /dev/floppy/c2t6d0 /dev/rdsk/c2t6d0 /dev/rfloppy/c2t6d0
disk 3 10/4/4.6.1 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c2t6d1 /dev/floppy/c2t6d1 /dev/rdsk/c2t6d1 /dev/rfloppy/c2t6d1
disk 4 10/4/12.5.0 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c6t5d0 /dev/floppy/c6t5d0 /dev/rdsk/c6t5d0 /dev/rfloppy/c6t5d0
disk 5 10/4/12.5.1 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c6t5d1 /dev/floppy/c6t5d1 /dev/rdsk/c6t5d1 /dev/rfloppy/c6t5d1
disk 6 10/16/4.5.0 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c7t5d0 /dev/floppy/c7t5d0 /dev/rdsk/c7t5d0 /dev/rfloppy/c7t5d0
disk 7 10/16/4.5.1 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c7t5d1 /dev/floppy/c7t5d1 /dev/rdsk/c7t5d1 /dev/rfloppy/c7t5d1
disk 8 10/16/12.6.0 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c8t6d0 /dev/floppy/c8t6d0 /dev/rdsk/c8t6d0 /dev/rfloppy/c8t6d0
disk 9 10/16/12.6.1 disc3 CLAIMED DEVICE DGC C2300WDR5
/dev/dsk/c8t6d1 /dev/floppy/c8t6d1 /dev/rdsk/c8t6d1 /dev/rfloppy/c8t6d1


please advise

[Aneesh Mohan · Accepted solution]

...
The below message says there was an I/O interruption and after the disk paths backto Online.

Disk at 10/16/12.6.0 is not responding. Check device, power, and cables.
LVM: PV 0 has been returned to vg[2].
Disk at 10/16/4.5.1 is not responding. Check device, power, and cables.
Disk at 10/16/4.5.1 is not responding. Check device, power, and cables.
LVM: PV 0 has been returned to vg[3].

Aneesh

[djoshi]

Hello Aneesh,

Thanks for your reply

Is there was any issues with our disk? What's the status of our disk now?

Can you advise the cause of this error?
I am afraid with this error, please advise me.
Is there anything we can do on this?

Thanks,

[Aneesh Mohan]

For more details ...

a) You can get it from GSP logs (If u hav gsp card intalled).

b) please check in /var/opt/resmon/log/event.log

Aneesh

[TTr]

How many of these errors do you get and how often? Do they occur at the same time of the day or day of the week?

These errors are caused either by flaky hardware or by high server i/o or missing patches.

This looks like a model20 disk array. LUNs 0 and 1 momentarily disappeared. Since it happened on two different LUNs and two separate physical paths, I suspect high i/o. You can try increasing the PV timeout value. This is an old OS version so finding patches is not an option.

[shjn]

Hi Joshi

Please let me know whether the same error is repeating in your syslog ? Also check any ems alert genarated on /var/opt/resmon/log/event.log . If error is not repeated you can ignore the same.

[djoshi]

Thx for all to reply and advise.

I didn't find any /var/opt/resmon/logs/event.log on 10.01?

I can see this log file on other version of OS, i.e on HPUNIX 11.

Can anyone advise me on this? why the file is not there on HPUNIX 10.01?

[chris huys_4]

Hi,

Check the current /var/stm/logs/os/"disklog", to check if anything "disk related" has been logged, "through cstm".

I.e. follow the cstm procedure at the end of this thread.

https://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1450580

Greetz,
Chris

[Doug O'Leary]

Hey;

As others have stated, it looks like you lost access to your disk for a few seconds.

While it appears that everything is back, I would take this as the system giving you fair warning that it's not going to last forever.

If this follows the standard paradigm, then this is a mission critical application on an unsupported OS on what is probably unsupported hardware.

If I were you, I'd be pushing to get this upgraded *very* strongly as there's a significant chance the old box won't be there for much longer.

Doug

[djoshi]

Thx to all advising me.

We are working to find the root cause of IO error and I will share the result with you.

Thx again.