HP array with EMC

[SCOTT GETTY]

On a 9000 K460 with fiber connectivity to a
EMC symmetrix I am getting the following error:

LVM: vg[1]: pvnum=3 (dev_t=0x1f03e500) is POWERFAILED
LVM: PV 3 has been returned to vg[1].

The volume in question is one of 6 drives on the EMC utilized in the vgmass2 group. I have applied the latest patches for the fiber controller. The filesystems are mounted and respond fine. What causes this problem?

[Stefan Farrelly]

Looks like one of your disks has powered down (first sign its dying) but it repowered up and is now aok. Call EMC to check for errors on their EMC frame - if there is a faulty disk they will know about it and get it replaced.

[James R. Ferguson · Accepted solution]

Scott:

The timeout value on your drives is too low. See document #A5523444, which in part starts:

This errors have been associated with timeout problems. Use the following command to change the timeout value of the problem disk drive:

# pvchange -t 180 /dev/dsk/disk_device

...JRF...

[Kofi ARTHIABAH]

Yes Scott... I had a similar problem (on a StorageTek 9145) and it turned out to be a failing disk (in your case physical volume # 3 ) you might want to take a look at using vgdisplay -v vg1 to identify the exact disk that is failing.

good luck

[SCOTT GETTY]

I have had EMC check the connections and drives and all is fine. This error occurs intermittently. The system is connected through a Vixel hub with another K460. Could this be a fiber loop timeout problem? I have already changed the Vixel hub and GBICs. Before the system was complaining about all of the drives and now it is just this one.
Thanks.

[Eileen Millen]

It sounds like there is a problem with that disk. If your EMC is enabled to dial home, it should have reported this problem. I think that you should call EMC.

[James R. Ferguson]

Scott:

Another good one is document #KBRC00000668.

If the error is accompanied by a message about pv[#] returned to vg[#], then the error can usually be attributed to a low timeout value on the disk driver. By default, this timeout is 30 seconds.

If the error is NOT accompanied by a message about pv[#] returned to vg[#], then the error can usually be attributed to a hardware problem on the disk.

...JRF...

[SCOTT GETTY]

Jim,
Even when I am properly logged in I can't find the two documents that you noted. I did try updating the timeout value for the volume and I am still getting the same message in the dmesg output even at -t 240.
Thanks.

[Kofi ARTHIABAH]

Scott:

dmesg is not very good for this sort of thing because it is not time-stamped. you might want to look at /var/adm/syslog/syslog.log

[James R. Ferguson]

Scott:

Here are the documents (attached). I'll keep looking for more info, too. Kofi makes a good point - look in /var/adm/syslog/syslog.log rather than the truncated dmesg buffer.

...JRF...

[James R. Ferguson]

Scott:

Guess I only can get one attachment at a time, so here's the other!

...JRF...

[SCOTT GETTY]

Great tips from all, thanks for the help. The pvchange -t on all drives on the EMC seems to have done the trick.
Kudos and farewell