LVM and VxVM

disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

 
Lawrence Fisher
Visitor

disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

We are having an issue in our VG with 164 disks (agile).

There are a bunch of disks that have the same 17 char PVID.
I found out couple years ago this can happen, as HP internally uses more than just the 17 chars to uniquely identify a disk..  They use these 17 chars, plus a timestamp and cpuid to create a 128 char PVkey.

Where is the timestamp and CPUID stored?

We get the 17 char PVID from something like this:

xd -An -j8208 -N17  -tc

What else can I read to generate a unique key per disk?

Thank you.

10 REPLIES 10

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

You are looking at slightly the wrong offset. The LVMREC is made up of the PVID and the VGID The PVID is made up of the CPU ID and the time the PV was created, while the VGID is made up of the CPU ID and the time the VG was created.

You should be starting at offset 8200 and reading 16 bytes. Then you will get the PVID followed by the VGID with the CPU ID first in each case and the timestamp second. Here's an example taken from a system I have which was recently rebuilt - I used a different boot disk from the last time I built it and the old boot disk is still in place, so I can look at both:

 

 

# xd -j8200 -N16 -tu /dev/rdisk/disk3_p2
0000000       2757928893      1315221704      2757928893      1315222026
0000010
# xd -j8200 -N16 -tu /dev/rdisk/disk4_p2
0000000       2757928893      1591464639      2757928893      1591464640
0000010
# uname -i
2757928893
# perl -le 'print scalar localtime $ARGV[0]' 1315221704
Mon Sep  5 12:21:44 2011
# perl -le 'print scalar localtime $ARGV[0]' 1591464639
Sat Jun  6 18:30:39 2020

 

 

As you can see disk4_p2 is my current boot disk and disk3_p2 is my old one. I can see that all the pvcreate and vgcreate commands were run on this system and when they were run.

What more are you looking for?


I am an HPE Employee
Accept or Kudo

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

Incidentally there's nothing that says a LVMREC has to be unique - hence why when you create a snapshot on a disk array of a given LUN and present it back to the same host, if you want to do anything with it you have to run vgchgid before bringing into a new VG.


I am an HPE Employee
Accept or Kudo
Lawrence Fisher
Visitor

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

Hi Duncan, 

I'm a little confused you said 8200 and 16 bytes - that is for LVMREC01    - we are LVMREC02.

We have been using 8208 and 17 bytes for a long time, see this post as example:
https://community.hpe.com/t5/lvm-and-vxvm/how-to-find-pvid-which-is-part-of-vg-version-2-0-or-2-1/m-p/5386439#.Xwh9gihKhPY

 

We had some email from HP few years back, they said they use the 8208 and 17 chars PVID as part of the PVkey, plus a timestamp and CPUid.  Here is that email below.

For that time, we simple recreated the problem PV, it got a different 17 char PVID and we moved on.

Now different case we have 16 pairs of disks with same 17 char PVID and all are in use.  We need to update our key to use those 17 chars at 8208 plus the timestamp+cpuid as per the email below.  But I don't know where they are stored.

Thank you very much.

 

 

 Per HPE, the PVID is only part of the equation. They suggest we should be using a PVID+Timestamp+CPUID (“PVKey”) to reference the disks. Comments?

e.g.

PV[0]

0x2d67102848 lvmp_pv_v2_t {

0x2d67102848 lvmp_uid_v2_t {

0x2d67102848 char uid[128]; A0000000000000080Tue Sep 5 15:27:23 2017e97e08e0-0f42-11e3-87c1

0x2d671028c8 } pv_id;

PV[23]

0x2d67103c68 lvmp_pv_v2_t {

0x2d67103c68 lvmp_uid_v2_t {

0x2d67103c68 char uid[128]; A0000000000000080Mon Mar 21 20:03:39 2016c47072fb-0f42-11e3-a38b

0x2d67103ce8 } pv_id;

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

Ah OK, LVM v2.x does complicate things somewhat. 

If you have support you might ask the response centre if they will let you have a copy of the kernel support tool dskinfo - it's a tool used by the back-end support organisation to diagnose disk problems on HP-UX, but it also does a great job of pulling the necessary data out of the on-disk data structures and displaying for you.

I can see the code that is used, and it looks abit more complicated for v2 PVs - if I get the time I will study it and try and figure out some standard OS commands which might give you what you need, but I'm afraid I'm not allowed to just share the tool myself.


I am an HPE Employee
Accept or Kudo
Lawrence Fisher
Visitor

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

Duncan,

Thank you.  We can go through official HP support, but since I had luck here before figured this might actually be quicker.

I was hoping to just find out where the timestamp is stored, get the offset and number of bytes, and append it to our 17 char PVID - then we have a unique identifier key for the disk.

Thank you.

 

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

Lawrence,

Found time for a quick play to see what differences I could see - I'll let you interpret this as best you can. I'll show some output from dskinfo - I can't share the tool, but can at least show you some of the information it gathers.

Start with a blank disk:

 

# dskinfo | grep disk3:LVM
# xd -j8200 -N20 -tx /dev/rdisk/disk3
0000000                0               0               0               0
*
0000014
# xd -j8200 -N20 -tu /dev/rdisk/disk3
0000000                0               0               0               0
*
0000014

 

Now create a LVM v1 record on disk using pvcreate:

 

# pvcreate /dev/rdisk/disk3
Physical volume "/dev/rdisk/disk3" has been successfully created.
# dskinfo | grep disk3:LVM
disk3:LVM: HEADER struct lv_lvmrec 0x2000
disk3:LVM: lvm_id   : "LVMREC01"
disk3:LVM: pv_id    : 0xa462a7bd  0x5f09b489 2020/07/11 13:46:01
disk3:LVM: vg_id    : 0x00000000  0x00000000 1970/01/01 00:00:00
disk3:LVM: pv_num   : 0x00000000
# xd -j8200 -N20 -tx /dev/rdisk/disk3
0000000         a462a7bd        5f09b489               0               0
0000010          445dcc8
0000014
# xd -j8200 -N20 -tu /dev/rdisk/disk3
0000000       2757928893      1594471561               0               0
0000010         71687368
0000014
# uname -i
2757928893
# perl -le 'print scalar localtime $ARGV[0]' 1594471561
Sat Jul 11 13:46:01 2020

 

Now create a LVM v2.x record on disk using vgcreate:

 

# vgcreate -V 2.2  -S 4t -s 8 /dev/vg01 /dev/disk/disk3
Volume group "/dev/vg01" has been successfully created.
Volume Group configuration for /dev/vg01 has been saved in /etc/lvmconf/vg01.conf
# dskinfo | grep disk3:LVM
disk3:LVM: HEADER struct lv_lvmrec 0x2000
disk3:LVM: lvm_id   : "LVMREC02"
disk3:LVM: pv_id    : 0x00020000  0x00000000 1970/01/01 00:00:00
disk3:LVM: vg_id    : 0x41303030  0x30303030 1995/08/15 06:27:12
disk3:LVM: pv_num   : 0x30303030
# xd -j8200 -N20 -tx /dev/rdisk/disk3
0000000            20000               0        41303030        30303030
0000010         30303030
0000014
# xd -j8200 -N20 -tu /dev/rdisk/disk3
0000000           131072               0      1093677104       808464432
0000010        808464432
0000014

 

My take-away from this - for v2.x LVM records - the data is at the same location, but it is no longer based on the CPU ID and creation time. Not sure if this is useful for you or not...


I am an HPE Employee
Accept or Kudo
KishJ
HPE Pro

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

Dear Mr. Lawrence Fisher,

Indeed, there is difference in the way the VGID and PVIDs are saved in VG version 1.0 vs. 2.x

VG Version 1.0

# xd -An -j8200 -N16 -tx r_DSF

The first two set forms the PVID and the next two VGID. The first part (8 digits) of each is unique to the system they were cerated on i.e. CPU ID.

VG Version 2.x

For the PVID, please use:


# xd -An -j8208 -N17 -tc r_DSF


For the VGID, please use:


# xd -An -j8344 -N17 -tc r_DSF

Hope it helps. 

Warm regards,


I am a HPE Employee

Accept or Kudo

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

Kish,

 

I confess to being slightly confused by that - as it doesn;t match the information that comes from the internal ktools command dskinfo:

# lvmadm -l
--- Version 1.0 volume groups ---
VG Name /dev/vg00
PV Name /dev/disk/disk4_p2

--- Version 2.2 volume groups ---
VG Name /dev/vg01
PV Name /dev/disk/disk3

# xd -An -j8344 -N17 -tx /dev/rdisk/disk3
                41303030        30303030        30303030        30303030
                32000000

# xd -An -j8208 -N17 -tx /dev/rdisk/disk3
                41303030        30303030        30303030        30303030
                32000000

# dskinfo | grep disk3:LVM
disk3:LVM: HEADER struct lv_lvmrec 0x2000
disk3:LVM: lvm_id   : "LVMREC02"
disk3:LVM: pv_id    : 0x00020000  0x00000000 1970/01/01 00:00:00
disk3:LVM: vg_id    : 0x41303030  0x30303030 1995/08/15 06:27:12
disk3:LVM: pv_num   : 0x30303030

So are the locations in the xd wrong, or is the dskinfo command not displaying the correct data?

(happy to continue this on an internal email if more appropriate - sure you can figure out my email address)

Duncan


I am an HPE Employee
Accept or Kudo
Lawrence Fisher
Visitor

Re: disks in a VG have same 17 char PVID, where is the timestamp stored to differentiate? LVMREC02

For LVM2, We have always been using 8208 and 17 chars for the PVID:

-j8208 -N17

The problem is that occasioanally this is not unique enough to identify a PV, there may be two PVs in a VG with the same 17 char PVID, as internally HPE uses a longer key as I wrote above, including a timestamp and cpuid.

All I am looking for is for LVM2, where is this timestammp stored so that in conjunction with the @ 8208 N17 PVID I can get a unique identifier for the disk.

Thank you,

Larry