LVM: PV 0 has been returned to vg[4]

[John Lombardi]

What is the meaning of this message:

LVM: PV 0 has been returned to vg[4]

[Thomas Schler_1]

This message may appear shortly after the
POWERFAILED message. Maybe, that electric
power was interrupted for a short period
(or another short lasting defect appeared).
If the disk (physival volume, PV) becomes
functional again and in time, it comes back
to volume group 4 (vg[4]).

[Alan Riggs]

This is generally indicative of a disk that has had a temporary failure/loss of power. If you look above it in syslog, you should see an entry something like "PV0 has powerfailed . . ." When the disk failure (temporary loss of scsi connectivity) happened, the syslogd recognized it and reported that the physical volume was no longer available in the VG. The message you posted indicates the return to availability of the physical volume.

[Anthony deRito]

Actually vg[4] is referencing the 4th valid entry in your /etc/fstab file. LVM has determined that the physical volume is now available and is redefining the device file as the primary device file for the LUN. If you have a disk array with dual controllers and have LVM configured for alternate links, you may want to run the vgdisplay -v command on that volume group and check if the primary and alternate paths are correct.

If they are not, you can switch the two on-the-fly with vgreduce and vgextend. Of course you need to have confidence that your alternate links are working to do this.

Tony

[Ramesh Donti]

Hi,
Pls find below the HP document which explains about ur problem.

vg[#]: pvnum=# (dev_t=##) is powerfailed; connection timed out DocId: KBRC00000668 Updated: 2/29/00 6:53:39 AM

PROBLEM
Errors in /var/adm/syslog/syslog.log or on console:


vg[#]: pvnum=# (dev_t=##) is powerfailed
recv: Connection timed out
RESOLUTION
First, determine the physical volume in question:

The disk device can be determined by the using the dev_t value.

For example:

dev_t value of 0x1c045000 is associated with c4t5d0


04 = instance number
5 = SCSI address number
0 = LUN number



The errors could be the result of one or more of the following:


If the error is accompanied by a message about pv[#] returned to vg[#], then the error can usually be attributed to a low timeout value on the disk driver. By default, this timeout is 30 seconds.

Increase the timeout up to the maximum of 180 seconds:
pvchange -t 180 /dev/dsk/disk_device

Increasing the timeout will not affect I/O performance on the disk.


Make sure that the latest SCSI/LVM patch (and its dependencies) are installed. For s800 10.20, this patch is:
PHKL_16751 :LVM:JFS:PCI:SCSI:SIG_IGN:SIGCLD:LITS:

As with all patches, please use the Patch Database at http://itrc.hp.com to determine the latest version.


Check for an I/O bottleneck on the disk.
sar -d

A high amount of traffic on a disk can cause severe performance problems and can cause requests to timeout.


If the error is NOT accompanied by a message about pv[#] returned to vg[#], then the error can usually be attributed to a hardware problem on the disk. DO NOT install patches on the system until the hardware has been diagnosed.

Change the timeout value on the disk and watch for further errors. Contact HP Hardware Support immediately if the errors persist.

If the powerfail messages are accompanied by lbolt errors
For example:

SCSI: Request Timeout -- lbolt: #######, dev: ######

check the SCSI controller connections/terminators. Make sure all connections are tight. If the errors persist, contact HP Hardware support immediately.


Check for an I/O bottleneck on the disk.
sar -d

A high amount of traffic on a disk can cause severe performance problems and can mimic a hardware issue.

[Patrick Wessel]

Thomas,
A power fail messages generated by the LVM means that the LVM was not able to successfully complete a command within a defined period of time. This does not necessarily means that the disk has a defect.
Please keep in mind that the system does not monitor the power lines to the disks!

[Stefan Farrelly]

The normal cause of this message is a disk spinning down then up again, which is the first sign it is on its way out - ie. its going to die sometime in the short-medium term future. Youre within your rights to have HP replace it, so as a preventative measure log a hardware call and get it replaced.

If you need more info run xstm and go to TOOLS -> UTILITY -> RUN and select LOGTOOL and then go to view raw log and look for any recent errors. Their should be some entries for the disk in question which you can then attempt to anaylze, or better still send it to the HP ITRC and have them analyze it.

[CHRIS ANORUO]

The message is generated by the Logical Volume Manager (LVM) after a disk powerfails, then recovers, but a following I/O to the disk returns another error from the driver. The following is the sequence of events:
1. I/O was sent to the disk.
2. The disk didn't respond in a reasonable amount of time.
3. LVM considered the disk missing due to powerfail, and mirroring allowed the system continue to run without hanging (extents marked stale, etc).
4. LVM polls the disk, and one of the polling I/Os returned in error rather than either success (disk online) or powerfail (no response), generating the message they saw.
5. LVM continues to poll the disk, hoping the next poll is more successful. It is.
6. The disk returns to the volume group, and mirrors are resynced, etc. There's probably a message in syslog showing the disk timing-out and returning. This could have been caused by an actual power or connection loss, or by over-working the disk or bus so the drive couldn't respond in time.

[John Palmer]

We have been experiencing similar problems recently during periods of very heavy I/O with Fibre Channel FC1010D disks connected to two K580's in an MC/Serviceguard environment.

The problem was eventually traced to the Fibre Channel controller card in the K580 that wasn't experiencing the problems!