Missing cluster_dsf files

[Berd]

Hello Peers,

I am looking at a 2 node serviceguard (11.20) cluster running on 11.31 and I am attempting to convert a volume group to use cluster device files. The command I use is vgcdsf <volume group name>. However, when I run this I get the below error message

Failed to convert persistent DSF(s) to cluster DSF(s) in VG /dev/vgdwhdba.
Corresponding Cluster DSF(s) do not exist.

When I then ioscan the system I find that there are no cdisk entries for the majority of luns, there are a one or two however. Below is an example of output.

disk     11  64000/0xfa00/0x12   esdisk  CLAIMED     DEVICE       limited  HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4005000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4005000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4005000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4005000000000000
                      /dev/cdisk/disk11   /dev/disk/disk11    /dev/rcdisk/disk11  /dev/rdisk/disk11
disk     36  64000/0xfa00/0x13   esdisk  CLAIMED     DEVICE       limited  HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4010000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4010000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4010000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4010000000000000
                      /dev/cdisk/disk12   /dev/disk/disk36    /dev/rcdisk/disk12  /dev/rdisk/disk36
disk     37  64000/0xfa00/0x14   esdisk  CLAIMED     DEVICE       limited  HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4011000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4011000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4011000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4011000000000000
                      /dev/disk/disk37   /dev/rdisk/disk37
disk     38  64000/0xfa00/0x15   esdisk  CLAIMED     DEVICE       limited  HP      OPEN-V
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30850.0x4012000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30840.0x4012000000000000
             34/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30860.0x4012000000000000
             32/0/0/0/0/0/0/4/0/0/0.0x50060e8004f30870.0x4012000000000000

How do I go about re/creating the cdisk device file entries where they are missing ? I have tried insf -e but this did not help ?

Any input gratefully received.

Cheers,

Berd

[Patrick Wallek]

Have a look here: 

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02273756/c02273756.pdf

and here:

http://www.hpuxtips.es/?q=node/263

[Berd]

Hi Patrick,

I have looked at the man page for vgcdsf, it is when I run this that I get the error 'Corresponding Cluster DSF(s) do not exist' ..

Do I have to create these manually, I thought they should appear automatically when the storage is presented in 11.31 ? I tried running a insf -e command but they still did not appear. The /etc/cmcluster/cdsf directory structure is present, so I should not need to run the cmsetdsfgroup command ? Although I have tried without luck :-(

Thanks,

Berd

[Patrick Wallek]

It appears that you need to run the command 'cmsetdsfgroup' to add the node to the appropriate cluster DSF group.

This is mentioned in the "Managing Serviceguard Eighteenth Edition" manual starting on page 207.

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02437444/c02437444.pdf

[Berd]

Hi Patrick,

That is what I thought, however this did not created the device files ?

Maybe I remove the server and then re-add it ? The funny thing is that there are 4-5 cdisk device files present.  I would have expected them all to be there, or there to be none ?

Can you see any harm in removing the node and re-adding it. Will I have an issue with the 4-5 cdisks already referenced or will it just revert back to 'agile' references ?

Berd

[likid0]

its the cmsetdsfgroup who creates the deviced:

I don't have any cdsf files:

root@vmcluste:/sbin/init.d> ioscan -fnNC disk
Class     I  H/W Path  Driver S/W State   H/W Type     Description
===================================================================
disk      1  64000/0xfa00/0x0  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk1      /dev/disk/disk1_p2   /dev/rdisk/disk1     /dev/rdisk/disk1_p2
                      /dev/disk/disk1_p1   /dev/disk/disk1_p3   /dev/rdisk/disk1_p1  /dev/rdisk/disk1_p3
disk      3  64000/0xfa00/0x4  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk3   /dev/rdisk/disk3
disk      5  64000/0xfa00/0x5  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk5   /dev/rdisk/disk5
disk      7  64000/0xfa00/0x6  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk7   /dev/rdisk/disk7
disk      9  64000/0xfa00/0x7  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk9   /dev/rdisk/disk9
disk     19  64000/0xfa00/0xc  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk19   /dev/rdisk/disk19
disk     20  64000/0xfa00/0xd  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk20   /dev/rdisk/disk20

I do a cmsetdsfgroup and they get created

root@vmcluste:/sbin/init.d> cmsetdsfgroup -c    
Cluster device file group configuration set to 'vmcluster2 vmcluster3'.
root@vmcluste:/sbin/init.d> ioscan -fnNC disk
Class     I  H/W Path  Driver S/W State   H/W Type     Description
===================================================================
disk      1  64000/0xfa00/0x0  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk1      /dev/disk/disk1_p2   /dev/rdisk/disk1     /dev/rdisk/disk1_p2
                      /dev/disk/disk1_p1   /dev/disk/disk1_p3   /dev/rdisk/disk1_p1  /dev/rdisk/disk1_p3
disk      3  64000/0xfa00/0x4  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk1   /dev/disk/disk3    /dev/rcdisk/disk1  /dev/rdisk/disk3
disk      5  64000/0xfa00/0x5  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk2   /dev/disk/disk5    /dev/rcdisk/disk2  /dev/rdisk/disk5
disk      7  64000/0xfa00/0x6  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk3   /dev/disk/disk7    /dev/rcdisk/disk3  /dev/rdisk/disk7
disk      9  64000/0xfa00/0x7  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk4   /dev/disk/disk9    /dev/rcdisk/disk4  /dev/rdisk/disk9
disk     19  64000/0xfa00/0xc  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk5   /dev/disk/disk19   /dev/rcdisk/disk5  /dev/rdisk/disk19
disk     20  64000/0xfa00/0xd  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk6   /dev/disk/disk20   /dev/rcdisk/disk6  /dev/rdisk/disk20

If I delete the config , they don't get deleted:

root@vmcluste:/sbin/init.d> cmsetdsfgroup -f -r
Node vmcluster2 is part of current cluster membership.
Node vmcluster3 is part of current cluster membership.

WARNING: removing nodes from the cluster device files group which are part of the local cluster.
Cluster device files on these nodes will not be kept in sync with the rest of the cluster.
Cluster device file creation disabled.
root@vmcluste:/sbin/init.d> ioscan -fnNC disk  
Class     I  H/W Path  Driver S/W State   H/W Type     Description
===================================================================
disk      1  64000/0xfa00/0x0  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/disk/disk1      /dev/disk/disk1_p2   /dev/rdisk/disk1     /dev/rdisk/disk1_p2
                      /dev/disk/disk1_p1   /dev/disk/disk1_p3   /dev/rdisk/disk1_p1  /dev/rdisk/disk1_p3
disk      3  64000/0xfa00/0x4  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk1   /dev/disk/disk3    /dev/rcdisk/disk1  /dev/rdisk/disk3
disk      5  64000/0xfa00/0x5  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk2   /dev/disk/disk5    /dev/rcdisk/disk2  /dev/rdisk/disk5
disk      7  64000/0xfa00/0x6  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk3   /dev/disk/disk7    /dev/rcdisk/disk3  /dev/rdisk/disk7
disk      9  64000/0xfa00/0x7  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk4   /dev/disk/disk9    /dev/rcdisk/disk4  /dev/rdisk/disk9
disk     19  64000/0xfa00/0xc  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk5   /dev/disk/disk19   /dev/rcdisk/disk5  /dev/rdisk/disk19
disk     20  64000/0xfa00/0xd  esdisk   CLAIMED     DEVICE       HP      Virtual LvDisk
                      /dev/cdisk/disk6   /dev/disk/disk20   /dev/rcdisk/disk6  /dev/rdisk/disk20

 root@vmcluste:/dev/cdisk> diskinfo /dev/rcdisk/disk3
SCSI describe of /dev/rcdisk/disk3:
             vendor: HP      
         product id: Virtual LvDisk  
               type: direct access
               size: 1024000 Kbytes
   bytes per sector: 512

In the end its just and alias with the same minor, so the cluster disk names don't colide

root@vmcluste:/dev/cdisk> ls -l /dev/rcdisk/disk3
crw-r-----   1 bin        sys         23 0x000006 Apr 19 15:53 /dev/rcdisk/disk3
root@vmcluste:/dev/cdisk> ls -l /dev/rdisk/disk7
crw-r-----   1 bin        sys         23 0x000006 Feb  3  2011 /dev/rdisk/disk7
root@vmcluste:/dev/cdisk>

If you need anything else tested, just tell me I will give it a go