topic Re: Missing cluster_dsf files in Operating System - HP-UX

Missing cluster_dsf files

Berd — Tue, 17 Apr 2012 14:32:25 GMT

Hello Peers,

I am looking at a 2 node serviceguard (11.20) cluster running on 11.31 and I am attempting to convert a volume group to use cluster device files. The command I use is vgcdsf <volume group name>. However, when I run this I get the below error message

Failed to convert persistent DSF(s) to cluster DSF(s) in VG /dev/vgdwhdba.
Corresponding Cluster DSF(s) do not exist.

When I then ioscan the system I find that there are no cdisk entries for the majority of luns, there are a one or two however. Below is an example of output.

disk     11 64000/0xfa00/0x12   esdisk CLAIMED     DEVICE       limited HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4005000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4005000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4005000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4005000000000000
                      /dev/cdisk/disk11   /dev/disk/disk11    /dev/rcdisk/disk11 /dev/rdisk/disk11
disk     36 64000/0xfa00/0x13   esdisk CLAIMED     DEVICE       limited HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4010000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4010000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4010000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4010000000000000
                      /dev/cdisk/disk12   /dev/disk/disk36    /dev/rcdisk/disk12 /dev/rdisk/disk36
disk     37 64000/0xfa00/0x14   esdisk CLAIMED     DEVICE       limited HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4011000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4011000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4011000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4011000000000000
                      /dev/disk/disk37   /dev/rdisk/disk37
disk     38 64000/0xfa00/0x15   esdisk CLAIMED     DEVICE       limited HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4012000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4012000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4012000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4012000000000000

How do I go about re/creating the cdisk device file entries where they are missing ? I have tried insf -e but this did not help ?

Any input gratefully received.

Cheers,

Berd

Re: Missing cluster_dsf files

Patrick Wallek — Tue, 17 Apr 2012 15:33:38 GMT

Have a look here:

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02273756/c02273756.pdf

and here:

http://www.hpuxtips.es/?q=node/263

Re: Missing cluster_dsf files

Berd — Tue, 17 Apr 2012 15:48:50 GMT

Hi Patrick,

I have looked at the man page for vgcdsf, it is when I run this that I get the error 'Corresponding Cluster DSF(s) do not exist' ..

Do I have to create these manually, I thought they should appear automatically when the storage is presented in 11.31 ? I tried running a insf -e command but they still did not appear. The /etc/cmcluster/cdsf directory structure is present, so I should not need to run the cmsetdsfgroup command ? Although I have tried without luck :-(

Thanks,

Berd

Re: Missing cluster_dsf files

Patrick Wallek — Tue, 17 Apr 2012 19:09:05 GMT

It appears that you need to run the command 'cmsetdsfgroup' to add the node to the appropriate cluster DSF group.

This is mentioned in the "Managing Serviceguard Eighteenth Edition" manual starting on page 207.

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02437444/c02437444.pdf

Re: Missing cluster_dsf files

Berd — Tue, 17 Apr 2012 20:09:29 GMT

Hi Patrick,

That is what I thought, however this did not created the device files ?

Maybe I remove the server and then re-add it ? The funny thing is that there are 4-5 cdisk device files present. I would have expected them all to be there, or there to be none ?

Can you see any harm in removing the node and re-adding it. Will I have an issue with the 4-5 cdisks already referenced or will it just revert back to 'agile' references ?

Berd

Re: Missing cluster_dsf files

likid0 — Thu, 19 Apr 2012 12:24:20 GMT

its the cmsetdsfgroup who creates the deviced:

I don't have any cdsf files:

root@vmcluste:/sbin/init.d> ioscan -fnNC disk
Class     I H/W Path Driver S/W State   H/W Type     Description
===================================================================
disk      1 64000/0xfa00/0x0 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk1      /dev/disk/disk1_p2   /dev/rdisk/disk1     /dev/rdisk/disk1_p2
                      /dev/disk/disk1_p1   /dev/disk/disk1_p3   /dev/rdisk/disk1_p1 /dev/rdisk/disk1_p3
disk      3 64000/0xfa00/0x4 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk3   /dev/rdisk/disk3
disk      5 64000/0xfa00/0x5 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk5   /dev/rdisk/disk5
disk      7 64000/0xfa00/0x6 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk7   /dev/rdisk/disk7
disk      9 64000/0xfa00/0x7 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk9   /dev/rdisk/disk9
disk     19 64000/0xfa00/0xc esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk19   /dev/rdisk/disk19
disk     20 64000/0xfa00/0xd esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk20   /dev/rdisk/disk20

I do a cmsetdsfgroup and they get created

root@vmcluste:/sbin/init.d> cmsetdsfgroup -c
Cluster device file group configuration set to 'vmcluster2 vmcluster3'.
root@vmcluste:/sbin/init.d> ioscan -fnNC disk
Class     I H/W Path Driver S/W State   H/W Type     Description
===================================================================
disk      1 64000/0xfa00/0x0 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk1      /dev/disk/disk1_p2   /dev/rdisk/disk1     /dev/rdisk/disk1_p2
                      /dev/disk/disk1_p1   /dev/disk/disk1_p3   /dev/rdisk/disk1_p1 /dev/rdisk/disk1_p3
disk      3 64000/0xfa00/0x4 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk1   /dev/disk/disk3    /dev/rcdisk/disk1 /dev/rdisk/disk3
disk      5 64000/0xfa00/0x5 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk2   /dev/disk/disk5    /dev/rcdisk/disk2 /dev/rdisk/disk5
disk      7 64000/0xfa00/0x6 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk3   /dev/disk/disk7    /dev/rcdisk/disk3 /dev/rdisk/disk7
disk      9 64000/0xfa00/0x7 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk4   /dev/disk/disk9    /dev/rcdisk/disk4 /dev/rdisk/disk9
disk     19 64000/0xfa00/0xc esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk5   /dev/disk/disk19   /dev/rcdisk/disk5 /dev/rdisk/disk19
disk     20 64000/0xfa00/0xd esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk6   /dev/disk/disk20   /dev/rcdisk/disk6 /dev/rdisk/disk20

If I delete the config , they don't get deleted:

root@vmcluste:/sbin/init.d> cmsetdsfgroup -f -r
Node vmcluster2 is part of current cluster membership.
Node vmcluster3 is part of current cluster membership.

WARNING: removing nodes from the cluster device files group which are part of the local cluster.
Cluster device files on these nodes will not be kept in sync with the rest of the cluster.
Cluster device file creation disabled.
root@vmcluste:/sbin/init.d> ioscan -fnNC disk
Class     I H/W Path Driver S/W State   H/W Type     Description
===================================================================
disk      1 64000/0xfa00/0x0 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk1      /dev/disk/disk1_p2   /dev/rdisk/disk1     /dev/rdisk/disk1_p2
                      /dev/disk/disk1_p1   /dev/disk/disk1_p3   /dev/rdisk/disk1_p1 /dev/rdisk/disk1_p3
disk      3 64000/0xfa00/0x4 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk1   /dev/disk/disk3    /dev/rcdisk/disk1 /dev/rdisk/disk3
disk      5 64000/0xfa00/0x5 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk2   /dev/disk/disk5    /dev/rcdisk/disk2 /dev/rdisk/disk5
disk      7 64000/0xfa00/0x6 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk3   /dev/disk/disk7    /dev/rcdisk/disk3 /dev/rdisk/disk7
disk      9 64000/0xfa00/0x7 esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk4   /dev/disk/disk9    /dev/rcdisk/disk4 /dev/rdisk/disk9
disk     19 64000/0xfa00/0xc esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk5   /dev/disk/disk19   /dev/rcdisk/disk5 /dev/rdisk/disk19
disk     20 64000/0xfa00/0xd esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk6   /dev/disk/disk20   /dev/rcdisk/disk6 /dev/rdisk/disk20

root@vmcluste:/dev/cdisk> diskinfo /dev/rcdisk/disk3
SCSI describe of /dev/rcdisk/disk3:
             vendor: HP
         product id: Virtual LvDisk
               type: direct access
               size: 1024000 Kbytes
   bytes per sector: 512

In the end its just and alias with the same minor, so the cluster disk names don't colide

root@vmcluste:/dev/cdisk> ls -l /dev/rcdisk/disk3
crw-r----- 1 bin sys 23 0x000006 Apr 19 15:53 /dev/rcdisk/disk3
root@vmcluste:/dev/cdisk> ls -l /dev/rdisk/disk7
crw-r----- 1 bin sys 23 0x000006 Feb 3 2011 /dev/rdisk/disk7
root@vmcluste:/dev/cdisk>

If you need anything else tested, just tell me I will give it a go