Disk VGID problem

[Alex Lavrov.]

Hello,

I have a strange problem with VGID.

I'll strat from the begining. I asked storage team to assign couple of luns to my server. After they did it, I created a VG and checked VGID on the disks. I used "echo | adb /dev/dsk/cXtXdX". It showed the right VGID, the one I see in "vgexport -s -p".

The disks are connected with two channels, but I saw only one. So I run "symcfg discover", after this "ioscan -f" and then "insf -Cdisk" and I saw that new devices were found. So, decided to check the VGID on the new devices, it should be the same, because physically it's the same device. It showed the right VGID. BUT, when I checked the VGID on the first device files, then ones I created the VG, I saw 0.

For some reason, "symcfg disco" erased the VGID of those devices from kernel? Is it some kind of bug or am I missing something?

I know that my explanation is a bit unclear, so here's a picture.

[Michael Denney]

Does vgdisplay show the alternate links of the new and old devices?

[Alex Lavrov.]

Yes. There is no problem in VG functionality.

[sathish kannan]

Hi Alex,
Do you use any Multipath utilities like EMC powerpath? Do you have any BCVs attached for this device?

Regards
Sathish

[Alex Lavrov.]

Yep, there is powerpath, no BCV.

[Bill Costigan]

Perhaps, accessing the second path just returns zero because the I/O didn't work.

Did you try a diskinfo on both paths. Typically I see the disk size reported as zero if the path isn't working quite right.

Use powermt display dev=all to see if powerpath can access all the device paths.

[Devender Khatana]

Hi Alex,

I agee with Bill. It could be due to an error in accessing disk through one path. Can you check the available path's by "powermt display dev=all" and "diskinfo -v" on the device file showing VGID=0?

HTH,
Devender

[Alex Lavrov.]

Everything seems to be fine. No dead channels, no IO errors. There is a case opened in HP about this, but no solution found yet ...

[Bill Costigan]

What do you see with diskinfo -v

[Devesh Pant_1]

Alex,
you can try doing the vgexport -p -v -s -m
and create the map file check the vgid in the map file created. since you are using the -p option the volumegroup will not be exported in reality.

This will ascertain the vgid for the volumeroup.

thanks
DP

[D Block 2]

I'm guessing here, but run the INQ and find the matching devices for the same SERNUM-id. [you can get the INQ command from another machine or somewhere, nice to have.]

The SER NUM field, I think it's the 2nd to last column on the output).

# inq > inq.out
# #comment: so find you 1st device's SER_NUM
$ grep inq.out

So, each device /dev/rdsk/c#t#d## that has the same SER_NUM are pointing to the same area in the symmetrix.

you should find maybe more then one device here, they point to the same disk!

the VGID should be the same but only after you do the pvcreate, and then the lvextend on the alternate-link devices. (I think).

Did you do the lvextend follow'ed by the dump of the VGID using adb ?

Did you do the following..
so, run the pvcreate ONLY on one of them, and then run lvextend on the other alternate-link devices.

example:

the vgdisplay -v should show the Alternate Links, for example, there are 3 alternate links here below for c28t11d6.
(note: only one pvcreate for c28t11d6, but (three command runs of) vgextend with the 3 Alternate Link devices):

PV Name /dev/dsk/c28t11d6
PV Name /dev/dsk/c29t11d6 Alternate Link
PV Name /dev/dsk/c30t7d4 Alternate Link
PV Name /dev/dsk/c31t7d4 Alternate Link

[vinod_25]

Hi alex

try this on a test server first...

1. dd if=/dev/rdsk/primary_disk_id of=/tmp/root.disk bs=8k skip=1 \
count=1
2. dd if=/dev/rdsk/mirror_disk_id of=/tmp/root.mirror bs=8k skip=1 \
count=1

The above two commands with dump the header information which contains the VGID information.

3. od -Ax -tx /tmp/root.disk
4. od -Ax -tx /tmp/root.mirror

This will dump to contents of the data files in hexadecimal format output.

The output file will look some similar to this:

0000000 4c564d52 45433031 77d93e3c 3b713c28
0000010 77d93e3c 3b713c29 1fc74b 0
0000020 170 860 a6 a
0000030 860 918 8 910
0000040 9c8 1fb000 b60 16
0000050 1000 beb 1fbb60 fd
0000060 0 10 80 2
0000070 80 88 6 82
0000080 8a 0 0 0
0000090 0 0 0 0
*
0000400 44454645 43543031 0 0
0000410 0 0 0 0
*
0002000

In the example above, the VGID will be comprised of lines '0000000' or '0000010'. The last two fields of line '0000000' or the 1st two fields of line '0000010'. In the above example, the VGID for the primary disk should be:

77d93e3c3b713c28

or

77d93e3c3b713c29

Repeat above steps if more than one mirror exists for the existing vg that is currently having the issues. Match the outputs of the 'od' command output and ensure that they all match according the the fields (Primary & Mirrored copies, they should).

5. xd /etc/lvmtab

Output will be similar to output from the od command. Again look at the last two fields from line 400 and 1st two fields from line 410. These should also match the output from
the od commands:

0000400 0000 0000 0000 0000 0000 0000 77d9 3e3c
0000410 3b71 3c29 0000 0000 0000 0000 0002 0000

The end result should be:

77d93e3c3b713c29 -The true VGID entry

Match this to the output from the od commands you ran against the primary and mirrored copy disk(s). If they do not match, then attempt the following to try to correct the situation:

a. cp /etc/lvmtab /etc/lvmtab.orig
b. rm /etc/lvmtab
c. vgscan
d. rm root.disk root1.disk
e. repeat steps 1-5 again.

Good Luck

Vinod

[Florian Heigl (new acc)]

definitely You should map device serial numbers and vgid's here.

My rough guess is that the discover made the disks show up on the other paths and You see something else at the first path, You had used to create the vg.

If You have access to them, get the HP CSS tools and play around with lvmux11.