Powerfailed VA7410 SAN LVM Volume disk

[Nick Wickens]

This ones confusing me a bit so any light shed may be helpfull. I have a VA7410 connected up via two HP FC swtiches to two 11i RP7410's. Most of the disk volumes are used in striped volumes for Raw Informix database use. However our program directory VG (VG13) is standard LVM. Because of the twin FC switch config a total of 4 paths are available to the single "disk" volume in this VG.

Since last Friday when an engineer was onsite to stcik some memory in one of the RP's I am getting a quick powerfailed blip just on this volume - always at the same time (01:35)

< Oct 12 00:35:25 live vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x000000004a409800), from raw device 0x1f080200 (with priority: 0, and current flags: 0x40) to raw device 0x1f0a0200 (with priority: 1, and current flags: 0x0).
< Oct 12 00:35:25 live vmunix: LVM: vg[13]: pvnum=0 (dev_t=0x1f0a0200) is POWERFAILED
< Oct 12 00:35:30 live vmunix: LVM: Performed a switch for Lun ID = 0 (pv = 0x000000004a409800), from raw device 0x1f0a0200 (with priority: 1, and current flags: 0x0) to raw device 0x1f080200 (with priority: 0, and current flags: 0x0).

Its not caused any problems but I can't think of any reason why this is occuring at the same time as this particular volume is in constant use during the day with no effect. I have checked for Cron processes.

Any views/clues ?

[Prashant Zanwar_4]

The probable cause can be the disk in question can a very busy one. So this will switch to the alternate path of the disk from primary.
Another probable reason is the timeout value of the PV if it is set too low..which you can change using pvchange -t

You have very good threads in ITRC on these messages, just try searching, you will find out the answer. I will try providing link to you.

Thanks
Prashant

[Prashant Zanwar_4]

Date: 2/29/00
Document description: vg[#]: pvnum=# (dev_t=##) is powerfailed; connection timed out
Document id: KBRC00000668

[Prashant Zanwar_4]

Sorry for pasting it so badly..But hope the extract helped...

Thanks
Prashant

[Steven E. Protter]

I would visually inspect cabling, make sure the drives in the array are firmly inserted.

IF possible, boot the system and see if it continues to happen. One actual powerfail can be replayed in the logs until boot.

When a hot-swap disk is replaced there is an lbolt in the log and dmesg output until the next boot.

Increasing the timeout is to me like turning up the radio to not hear the car engine knock. The default of 30 seconds should be enough for all but the busiest of disks.

After all causal situations related from Friday are eliminated, fire up mstm or cstm or xstm and run exercize on the disk. See if it spots any errors.

SEP

[Shawn M Harris_1]

You should run a pvdisplay on the PV (/dev/dsk/cxtxdx) to view a few settings.

One is the timeout value mentioned above. If this is default it is 30 seconds.

Also if Autoswitch is On then you may want to consider turning it off pending further analysis of what processes may be IO intensive at the time of the pvchange. From my understanding if Autoswitch is On LVM will determin the "best" path to disk. I don't know the algorithm in which LVM does this. This may be a good thing though. If your disk/controller/hba is busy at that time of day then you may want LVM to Autoswitch to a "better" path.

[Nick Wickens]

In the end I have settled on the problem being a bad FC cable (must have gone bad when it was unplugged by engineer).

If only they would put these FC cables in nice thick rubber coatings and invent more substantial connectors.!!