LVM mirror on two VA7400 luns

[Daniel Fourie]

Hi

I have got a scenario witch needs some clarification. We have got two va7400â s each extended with two ds2405 disk systems that is connected to 4 Brocade fiber channel switches.
What I would like to do is host based mirroring on the shared volumes (Note: all these hosts run in a MC/Service guard cluster). I would like to know what is the best option for mirroring these volumes and what the advantages and disadvantages are, I would also like to know in the case of failure on either side of the mirror (primary or alternate) what I am to expect, will the systems still be able to perform normally and what needs to be done if the corrupted mirror is back online.

Thanks in advance.

Regar

[Bernhard Mueller · Accepted solution]

Daniel,

you can easily setup this, and actually this is then a typical "Campus Cluster" configuration. (Two separate storage arrays, two SANs to reach each array)

What you need to do is setup similar LUNs on your second VA and make them visible to the MC/SG nodes HBAs through Secure Manager.
(If the fabric zoning is curretnly setup so that your nodes see only one VA, you will still have to change the zoning config in both SANs).

Then it is pure LVM stuff:
pvcreate
vgextend
lvextend -m 1
etc.

Important note: you cannot do this is your current lvols are LVM striped! In that case you would have to re-create non-striped LUNs and copy the data (i.e. downtime).

If you want to be able to run the packages with one VA (in case the other one fails). Your cluster package config needs to set VGCHANGE to "vgchange -a e -q n" to activate VGs without quorum.

In case one VA is not available for some time an comes back online you should then run vgchange -a e and vgsync for all vgs that are active on a certain SG node and check with lvdisplay that all extents current again.

Regards,
Bernhard

[Bernhard Mueller]

Daniel,

a few things to add...

you should set up physical volume groups for each LVM mirrored VG and make the mirror pvg-strict (lvextend -m 1 -s g) so that you make sure you mirror a LUN on one VA onto a LUN on the other VA.

you need to reimport the shared VGs on the alternate nodes in the cluster.

when you vgextend on the primary node and vgimport on the alternate node(s) make sure you setup some static load balancing (Alternating primary and alternate links to the LUNs). If you vgimport -s you will have all your primary links to the PVs through the same path! So either add some vgreduce vgextend commands after vgimport or run vgimport with a list of device files in the desired sequence. Do not forget to add corresponding lines in /etc/lvmpvg on the alternate nodes so that you have similar PVGs.

Regards,
Bernhard

P.S. What you have to expect in case of a failure is very much depending on which kind of failure you have.

[Daniel Fourie]

If just all replies where like this.

This is excellent information you have given me (what I was looking for).

I have just got one more question that is if you can help. I want to chain two brocade fiber channels together to form one, is there any funny port behaviors I have to set or can they be chained directly together (direct fabric attached).

Thanks again.

[Sridhar Bhaskarla]

Hi Daniel,

It is not possible to chain the fiber channels together on the host like APA with the network cards.

If your idea is to use both the fiber channels (load balancing), then you will need to use AutoPath by HP or Powerpath (latest version is supposed to support HP disk subsystems) by EMC.

HOwever, if you specify the disks on the other fiber channel for the mirrors (using PVGs as already mentioned), then you can see the load going across both the channels. Good thing about mirroring is that it is possible for the system to use any of the disks to read the data. So, you may see good improvement if your application does a lot of reads when compared to writes.

-Sri

[Daniel Fourie]

I doint whant to chain the HBA's I what to daisy-chain the fiber switches

[Bernhard Mueller]

Daniel,

if you have 4 Brocade switches I would assume that one pair of switches has an interconnect (called ISL) and the other pair as well.

Then two switches form one fabric, i.e. you have two separate SANs. You should keep them separate so that *one* total SAN failure could be survived, since the other SAN is still working.

So I assume what you *might* want to do is have more than one link between the 2 switches that constitute a single fabric. So that 2 FC cables form one 4Gbit/s inter switch link. This is called trunking, which works seamlessly on the Brocades. However it is a licensed feature....

telnet to the switches and use
fabricshow
switchshow
licenseshow

Regards,
Bernhard

[Ted Buis]

MirrorDisk/UX for mirroring or the Full VxVM product is an option. If you are already used to Veritas then VxVM is possibly for you. However MirrorDisk/UX is likely what most people are using on HP-UX.