LVM---alternate links added to vol group in incorrent order

[Vernita Hall]

Need some LVM expertise. (HP-UX 11i version 1)

We have a volume group defined for a large data base; storage is SAN-attached array. Four disks were later added to the volume group. Let's call them c0t0d0, c2t0d0, c4t0d0, and c6t0d0. Their alternate disks are, respectively, c1t0d0, c3t0d0, c5t0d0, and c7t0d0. Logical volumes were created with a stripe count of 4.

However when the alternates were added to the volume group via vgextend, three of them were NOT added in an order that corresponds to their primary disks. The order of the alternates in /etc/lvmtab is: c5t0d0, c1t0d0, c3t0d0, and c7t0d0. vgdisplay -v of the volume group does show the correct pairing of primary to alternate; striping on the disks themselves shows the correct order of the primary disks. Just /etc/lvmtab shows the discrepancy.

We had an occurrence when the primary disk paths in the entire volume group attempted failover to the alternates (a switch failed). The results were unexpected, and not exactly successful.

What are your thoughts on the possible impact of the mismatched PV link definitions? Would you think it important to fix this error?

Thanks for your input.

[Torsten.]

How many alternates do you have per disk?

Sometimes it is a good decision to configure like this (for kind of load balancing):

HBA1 -> disk1 primary
HBA2 -> disk1 alt

HBA2 -> disk2 primary
HBA1 -> disk2 alt

vgdisplay will show you the current order.
To change this order, you may vgremove a link and add it again with vgextend at the end.

[James R. Ferguson]

Hi:

LVM alternate links provide high-availablity to a physical device when properly configured (among different controllers).

The first path defined is the primary link and any others become secondary (tertiary, quaternary, etc.)

To promote any link to be the primary link, you can 'vgreduce' the non-primary link and then 'vgextend' it back into the volume group.

If this proves too tedious for a large number of links, simply 'vgexport' the volume group using the '-f outfile' switch/argument; edit the order of the physical links within the 'outfile' to specify the device(s) you want to be the primary link(s) first; and then 'vgimport' using the '-f' option of 'vgimport'. See the manpages for more details.

It is the retained order of physical links in the '/etc/lvmtab' that provides the notion of primary versus secondary pvlinks. Recreating a 'lvmtab' with 'vgscan' is a common way to mis-arrange carefully crafted alternate links :-)

Regards!

...JRF...

[Vernita Hall]

Thanks for your feedback, guys.

There's just one alternate link per primary.

Also, the volume group is part of a ServiceGuard cluster.

I'm getting the impression that you would think it advisable to fix the mismatched alternate link definitions. Do you agree?

[James R. Ferguson]

Hi:

I'm getting the impression that you would think it advisable to fix the mismatched alternate link definitions. Do you agree?

Most certainly, yes. Part of making a successful high-availability complex is to eliminate all single points of failure (SPOF).

While LVM alternate links do not dynamically load-balance in 11.11 in the sense of issuing I/O between multiple paths, you eliminate a SPOF and you do allow different controllers to assume the I/O load together.

Regards!

...JRF...

[Nguyen Anh Tien]

Alternate PV links provides HA only. You can run vgreduce then vgextend to make it in your order
If you want to change the path online, you can use
#vgchange -t

[Torsten.]

@Nguyen Anh Tien:

Do you mean "pvchange -s ..." by chance?

"vgchange -t ..." is 11.31 only and has a different purpose, see
http://docs.hp.com/en/B2355-60130/vgchange.1M.html

Note: this change is only valid until the next boot.

[Nguyen Anh Tien]

Sorry, # pvchange -s is corrected

[Ninad_1]

Hi,

Your primary question/observation seems to be - if the alternate links are defined in a order different than the primary links while vgextend would have been used - will it affect the actual behaviuor of failover to alternate links in case the primary link becomes unavailable.
In my view this (the sequence in which you extended the patchs to provide alternate links) should not affect as long as alternate link is defined. I am not exactly sure why it has happened in your case, it would be more better if we can understand what was the situation that caused the primary links to become unavailable ? - was it fault on HBA port / switch port / storage port ???
In order to really resolve your problem - it would be unwise according to me - to proceed to resolve the problem without actually understanding the complete scenario in which the switching did not happen.
Please provide some background of the problem.

Regards,
Ninad

[Vernita Hall]

Ninad---
You hit my primary concern right on the nose: Might the incorrect order of the alternate link definitions impede successful LVM failover to alternate disks if the primary path is lost?

To answer your question:
Our scenario was 1) a ServiceGuard cluster of two nodes, each node with four fibre channels, 2) all 8 fibre channels are connected to two SAN switches, 3) one of the switches crashed and went out of service, 4) we lost two paths to all disks on both servers, 5) Some element(s) of the critical ServiceGuarded application(s) unexpectedly went non-responsive.

While investigating possible causes, starting with LVM, I saw two LVM anomalies, one of which I described here. Wanted to invite some of your thoughts on possible impact of the alternate links mismatch anomaly. Seeing it made me uncomfortable.