topic Re: /dev/cdisk/disk? in Operating System - HP-UX

/dev/cdisk/disk?

Doug O'Leary — Fri, 18 Feb 2011 23:21:57 GMT

Hey;

I'll be helping a cohort build a new cluster on a couple of HPUX 11.31 systems on monday - at the moment, since it's very late on a Friday, that's about all I know of the systems.

He was asking about some differences in the ioscans between the two systems in that one of them is showing /dev/cdisk/disk## along with most of the legacy devices..

What's a cdisk? I know what the legacy devices are and I know what the persistent ones are, but I've not seen a cdisk before...

Any hints greatly appreciated.

Doug O'Leary

Re: /dev/cdisk/disk?

nijokj — Sat, 19 Feb 2011 04:59:37 GMT

Cluster-wide Device Special Files (cDSFs)

Because DSF names may be duplicated between one host and other, it is possible fordifferent storage devices to have the same name on different nodes in a cluster, and for the same piece of storage to be addressed by different names. Cluster-wide device files (cDSFs), available as of the September 2010 HP-UX Fusion Release, ensure that each storage device used by the cluster has a unique device file name.
Please refer the following article for complete information,
http://www.hpuxtips.es/?q=node/263

Re: /dev/cdisk/disk?

P Arumugavel — Sat, 19 Feb 2011 06:33:19 GMT

hi,

It is cluster device special files (cDSFs). This feature is effective only in a cluster environment and requires that you install HP Serviceguard A.11.20 and patch it to support cluster-wide device special files.

Cluster device special files (cDSFs) require the installation of the following patches on HP-UX 11iv3 September 2010 OE:
1. PHSS_41225 11.31 Serviceguard A.11.20.00 patch
2. PHCO_41235 11.31 iocdsfd(1M) and io_cdsf_config(1M) patch

Rgds...

Re: /dev/cdisk/disk?

Manix — Sat, 19 Feb 2011 06:35:23 GMT

It`s a new feature i saw in MCSG 11.20

Cluster-wide device special files (cDSFs) are persistent device special files applied
across a set of nodes. That is, they ensure that the same piece of storage has the same
devicefile name on all of the nodes that share it; no matter how many paths there are
to the device, the same cluster DSF is used to address it. If the device is moved, the
same cDSF still addresses it.

Create the cDSFs.

NOTE: cDSFs apply only to shared storage;
â ¢ If the cluster does not exist yet, specify the name of each prospective node,
for example:
cmsetdsfgroup -n node1 -n node2 -n node3 -n node4
â ¢ If the cluster does exist, you can simply run:
cmsetdsfgroup -c

Do read release notes for MCSG 11.20 edition
for details & implementations

Thanks
Manix

Re: /dev/cdisk/disk?

Ismail Azad — Sat, 19 Feb 2011 08:09:40 GMT

Hi doug,

As mentioned by others it's a cluster wide device file which means that the DSF will be consistent accross the nodes of the cluster as on a local system the device file is assigned by the kernel of "that node" and when a vgimport on the other node is done, the same disk would have a different DSF as it is assigned by the kernel on the other node.

To remove this feature of "asymmetry" of the classic "persistent" DSF, a DSF that "persists" accross reboots but normally did not "persist" accross the nodes of the cluster came the concept of the cDSF in the releases mentioned by the other forumers. The commands cmsetdsfgroup and io_cdsf_config are related to the cDSFs.

Regards
Ismail Azad

Re: /dev/cdisk/disk?

Doug O'Leary — Sat, 19 Feb 2011 17:20:41 GMT

Hey;

Thanks for the info. Looks like I got a bit more reading to do.

Out of curiosity, why is there a need for cluster dsfs? You vgexport -s, vgimport -s, why does it matter if /dev/disk/disk32 on one node is /dev/disk/disk28 on another?

Seems this is more of a solution looking for a problem... so I must be missing something.

Thanks again for the replies.

Doug O'Leary

Re: /dev/cdisk/disk?

Manix — Sat, 19 Feb 2011 17:38:17 GMT

Because DSF names may be duplicated between one host and other, it is possible for
different storage devices to have the same name on different nodes in a cluster, and
for the same piece of storage to be addressed by different names. Cluster-wide device files (cDSFs), available as of the September 2010 HP-UX Fusion Release, ensure thateach storage device used by the cluster has a unique device file name.

HP recommends that you use cDSFs for the storage devices in the cluster because this
makes it simpler to deploy and maintain a cluster, and removes a potential source of
configuration errors.

Hope this helps.

Thanks
Manix

Re: /dev/cdisk/disk?

Ismail Azad — Sat, 19 Feb 2011 18:08:10 GMT

Hi Doug,

> You vgexport -s, vgimport -s, why does it matter if /dev/disk/disk32 on one node is /dev/disk/disk28 on another?

There are times when I have configured a cluster on a "test server" and run into problems like... let's take the example you mentioned a /dev/disk/disk32 and /dev/disk/disk28 being the same disk on different nodes of the cluster. Let's say I am creating vg01 as a "shared volume group" and have chosen /dev/disk/disk32 as my P.V and the /dev/dsk/c2t3d0 happens to be the mirror of the boot disk on the other node!. Didn't get it right... That /dev/dsk/c2t3d0 very well could be your /dev/disk/disk32 . Now you've run into a problem. Ofcourse there are "best practises" to follow but as Manix rightly mentioned about cDSFs the key point being..

> and removes a potential source of
configuration errors.

Regards
Ismail Azad

Re: /dev/cdisk/disk?

Doug O'Leary — Sat, 19 Feb 2011 18:34:46 GMT

Hey;

I'm still not really seeing the problem; however, that doesn't mean there's not one there.

If I create /dev/vg01 with /dev/disk/disk32, export it with "vgexport -s ... " and "vgimport -s ..." it's going to grab that one disk regardless of the name on the second host because it's looking for the matching vgid in the descriptor area. I don't really care if the disk on the second node is /dev/disk/disk32, disk23, or even a legacy device.

Consistency's a good thing, don't get me wrong; I'm just not seeing the problem this is supposed to address.

I can see this coming in handy when you're trying to make sure a specific lun is either used or not used. I have scripts that parse out xpinfo (hds), inq (emc) and evainfo to identify which specifc lun matches which legacy and persistent device. Maybe that's why I'm not seeing the problem; I've already worked my way around it...

And, btw, the link that nijokj provided is excellent. When I had my first conversation with my cohort, he was saying that his directions told him to run csshsetup, cmsetdfsgroup, etc, which confused me greatly. At least now I know why he's running those commands.

Thanks again for the replies.

Doug O'Leary

Re: /dev/cdisk/disk?

Manix — Sat, 19 Feb 2011 18:47:14 GMT

Hello Dough,

What you said is correct that 'vgimport -s 'will always bring the right diks into a VG pool besause it don`t even reading device files. It`s reading VGID metadata on the diks.

But just think about persistent device files
on 11.31 where the device files names can be
kept same even after the HW replacements & change in lun paths, in that regard if we have the same devices with the exact similar names across the nodes then maintenance would be easy , example to check if the same lun is visible from all the nodes at the same time you may need to run complex commands to get PVID /VGID ,what if the device files tells you that , make sense on 11.31 .

Other wise vgimport -s & xd command with PVID/VGID is good enough to do things.

Don`t you think it makes the job easy ??

Thanks
Manix

Re: /dev/cdisk/disk?

Ismail Azad — Sat, 19 Feb 2011 18:56:28 GMT

Hi,

> I have scripts that parse out xpinfo (hds), inq (emc) and evainfo to identify which specifc lun matches which legacy and persistent device. Maybe that's why I'm not seeing the problem; I've already worked my way around it...

YES... that's why u don't see the difficulty for anyone who would face it. :).

That was "your solution" and I am sure many people have "worked their way around this".. The same purpose that was served by your scripts is now a "feature" of the latest serviceguard.

Regards
Ismail Azad

Re: /dev/cdisk/disk?

Manix — Sat, 19 Feb 2011 19:32:01 GMT

Thanks correct !! Ismail.

btw Dough looks you have a bunch of good scripts , do share with us if possible -)

Thanks
Manix