Alternate pathing

[Jeremy Loukinas_1]

How do I setup alternate pathing when creating volume groups.

Jeremy

[Marco Santerre]

you just

#vgextend vgname /dev/dsk/altdisk

and when you do a
#vgdisplay vgname

your disks will come up as

PV Name /dev/dsk/disk1
PV Name /dev/dsk/disk2 Alternate Link

[John Poff]

Hi,

It's pretty easy, if you have two paths to your disks. Let's say you have c6t0d0 through c6t0d7 pointing to 8 disks, and you have PV links through another controller pointing to the same 8 disks as c8t0d0 through c8t0d7. You can do something like this:

vgcreate vg_name /dev/dsk/c6d0d0 /dev/dsk/c8t0d1 /dev/dsk/c6t0d2

vgextend vg_name /dev/dsk/c8t0d0 /dev/dsk/c6t0d1 /dev/dsk/c8t0d2


Then, a vgdisplay -v vg_name should give you your PVs and your alternate links.


JP

[Sridhar Bhaskarla]

HI Jeremy,

Say c1t0d0,c2t0d0,c1t0d1 and c2t0d1 are links for two devices, follow the default vgcreate process.

vgcreate (options) vg01 /dev/dsk/c1t0d0
vgextend vg01 /dev/dsk/c2t0d0
/dev/dsk/c1t0d1 /dev/dsk/c2t0d1

This will put you into passive-passive configuration. Here only c1 path will be accessed for IO and the other path is idle until a failure occurs on c1 path.

Now to load balance, do

vgreduce vg01 /dev/dsk/c1t0d1
vgextend vg01 /dev/dsk/c1t0d1

This will change the order of PV so that all the file systems on c1t0d1 will be accessed through c2 path. This may not loadbalance 100%, but atleast you will be usign both the paths.

If you do lvm striping, then you can get around 90% load balance.


-Sri

[James R. Ferguson]

Hi Jeremy:

The rule is simple; the first path to a physical device specified during 'vgcreate' is the "primary" and sucessive ones (up to 8 paths, total) are "alternate"s.

If a volume group is 'vgcreate'd from one physical device, an alternate link can be added later by 'vgextend'ing the volume group with a device file that represents the same physical path as the existing one.

To swap an alternate and a primary link, 'vgreduce' the primary link. The 'alternate' is then promoted to the primary; Follow this by 'vgextend'ing the original primary link and it becomes the alternate link.

In reality, the first path to a particular device is the primary link. An duplicate path (in the form of another device file) is an alterante. Primary links occur first in '/etc/lvmtab', followed by the alternate link(s).

Remember that 'pvlinks' [alternates] are *not* for load-balancing. LVM uses alternate links only for failover. You want alternate and primary links on separate disk controllers or fibre channels for high-availability.

Regards!

...JRF...

[Jeremy Loukinas_1]

How do I figure out which disks are on my alternate path?

[Sridhar Bhaskarla]

Surprized..

JRF.. Are you giving a message that the Alternate link should not be used for loadbalancing?. We use it all the time here and with lvm striping it works great. With the auto switch over capability, you can have high availability during failures even with this setup. And when the link comes back, it will start *load balancing* automatically.

When the system does gigs of IO, I hate to see one path sitting idle.

-Sri

[John Poff]

Jeremy,

You'll need to look at an ioscan to see which disks you have defined. Also, you'll need to understand what kind of external disks you are using and how they are connected. Are you using a disk array?

Sri,

I thought the same thing when I first read James' post, but then I understood what he was saying. I think he means that when you see a PV Link that LVM isn't using the PV Link at all unless the main path fails. You can still do the pseudo-load balancing that we all do by alternating the paths.

JP

[James R. Ferguson]

Hi Sri and John:

Sri, no, John Poff interpreted what I was trying to say correctly! I was going to discuss the 'autoswitch' feature too, but got called away so I left the post as you see it.

Thank you both for clarifying this. Sorry I created confusion, but perhaps in doing so the important points will be driven home.

With my warmest regards to you both...

...JRF...

[Murray Yutzy]

All,
Regarding use of Alternate Paths, I'm not sure it was made clear the dependency on the RAID system you are using. In one case we attach through a SAN to an LSI. LSI controllers will experience some level of delay in switching a LUN from Ctrlr A to Ctrlr B. So, if primary is on A, secondary on B, you will experience delays as the LUN switches between A and B. This is unlike the HP XP controller that does allow LUNs to be equally accessed from both ctrlr paths.

[Sridhar Bhaskarla]

Hi Murray,

Good point. However with LVM's alternate pathing, you can only access one LUN through one PATH. So, the delay of LUN switching is not applicable.

However, this may affect depending on how the backend is setup. On XPs and EMCs etc., the storage administrators (if not you) might have already configured the LUNs to loadbalance on their side. So there is a chance that the loadbalance on the system side may counterfeit their setup. So, it is important that we their suggestions before we do anything on our side.

For a good setup, analysis alone is not sufficient. Good testing is paramount. I would not implement anything on production unless it is well-tested and proven on my staging environment.

-Sri

[Michael Tully]

Just to add something about EMC gear. If you have the EMC "powerpath" product installed, you don't need to do anything, PVLINKS, alternate pathing etc. It has it's own way of dealing with both, providing the secondary link has been set up correctly in the first place.

FWIW
Michael