Strange problem using SAM to create a new filesystem

[Dave Johnson_1]

I am running HP-UX 11.00 and have an XP512 using open-9 in RAID-5 which gives me LUNs of size 7040MB each. I create a new vg using lets say 4 of this using command line programs. I then use SAM to create a single lvol1 on that new vg using 2 stripes, strip size 64K, VxFS, and Largefiles. If I try to create this new lvol with >7000MB, I get an error message about not enough extents available. I can create the lvol as say 7000MB, then increase the size in increments of 7000MB until finally able to reach final size of 28160MB.
Does anyone know of a patch I might have install to cause this, or am I missing some flag when I create the vg? I am getting tired of the work around and would like to find the root cause.

Thank you in advance
-Dave

[Patrick Wallek · Accepted solution]

It could be a SAM limit, though I'm not sure.

Why not just do this at the command line as well?

# lvcreate -L 28160 -i 2 -I 64 /dev/vg??
# newfs -F vxfs -o largefiles /dev/vg??/lvol1

[Todd McDaniel_1]

Looks like SAM is balking on the single disk size... thats another reason I dont use SAM for LVM work...it is to persnickety, imho.

I much prefer command line for LVM where I can specify everything. Of course you dont get the "are you sure?" message on the command line nearly as often as you can with SAM.

You might try to fool it by doing -l instead of -L and see if that works...

[Dave Johnson_1]

Well I have tried your suggestions and they did not help. It appears to be a problem with LVM. I was able to reduce an FS I was working with on Saturday from 56320MB to 28160MB, then issued the command:
lvextend -L 56320 /dev/vg06/lvol1
and received the following:
lvextend: Not enough free physical extents available.
Logical volume "/dev/vg06/lvol1" could not be extended.
lvextend: Couldn't retrieve the list of the physical volumes
belonging to volume group "/dev/vg06".
Failure possibly caused by strict allocation policy
Here is partial output of vgdisplay -v vg06
--- Volume groups ---
VG Name /dev/vg06
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 64
Cur PV 8
Act PV 8
Max PE per PV 1760
VGDA 16
PE Size (Mbytes) 4
Total PE 14080
Alloc PE 7040
Free PE 7040
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg06/lvol1
LV Status available/syncd
LV Size (Mbytes) 28160
Current LE 7040
Allocated PE 7040
Used PV 4

You can see there are free extents in the vg to be added to this lvol. The largest I was able to make it at one shot is 42240MB, which is the size of 2 LUNs. It seems funny this is the same as the stripe count.

[Todd McDaniel_1]

Unless I have badly misread this... You only have Free PE 7040... = 1 disk(LUN)

You cant add more than that till you add more disks to the VG... Also, since you appear to have the LVOL mirrored... you only show ONE free disk Free PE 7040 = 1 disk(LUN)... in the VG...

So there is no way you can even extend it at all untill you add at least one more disk to the VG...


When you reduced the Lvol the other day, did you remove the disks from the VG? It appears that you did...

[Dave Johnson_1]

Todd,
Yes you did misread it. 1 PE is not 1MB, it is 4MB. I have infact 4 free/unused LUNs and 4 inuse. I am able to increase the space form 28160MB to 56320MB but I have to do it in steps of 14080MB at most.

Thanks for trying.

Anyone else have an idea?
-Dave

[Dave Johnson_1]

Just bringing this back to the head of the list.
Anyone else see this before?
-Dave

[Todd McDaniel_1]

Dave,

I have seen problems with using the max extents...

Can you try to do lvextend with a few hundred less extents?

Do:
lvextend -L 55000 /dev/vg06/lvol1

Instead of:
lvextend -L 56320 /dev/vg06/lvol1

[Dave Johnson_1]

Todd,
I tried using 56000, 55000, 50000, 45000, and so on until I found it was in a maximum increment of 14080MB. I believe the magic number is the size of a LUN times the number of stripes.

Thanks for trying,
-Dave

[Todd McDaniel_1]

That does make perfect sense to me since you have a striped LVol...it should error if you dont have the increment of extents for a disk in your striped Lvol.

You should only be able to add increments that are full disks based upon what disks you have in the stripe.

It seemed wierd to me at first b/c I have never added disks to a stripe... I only ever created them once at inception, never adding disks later as you did.

[Dave Johnson_1]

Todd,
The truly strange bit is that I have 4 unused disks so I thought I should be able to add all 28160MB to the filesystem but I am tuck with a max of 14080MB. So if I was creating a new filesystem with 16 LUNs, I would have to allocate the space 8 times in increments of 14080MB to get to the full 112640MB. What a pain in the but. Image if I was creating a FS with 30 LUNs, or even 100 LUNs!!!!

Thanks again,
-Dave

[Todd McDaniel_1]

no points here...

I would think you could add in ANY increment that is (N x 14080)???

ie 24160, or 48320 etc...

If you just script it cut and paste your lines in vi... with Nlines/shiftY/P

[Dave Johnson_1]

Todd,
I did try all values. The current size is 28160MB which is 4 LUNs. I added 4 LUNs to the vg then tried to add the 28160MB to the lvol to get 56320MB but was only able to increase to 42240 which is 28160+14080. Then I was able to increase another 14080 to get to 56320 the full size of the vg.

I think I will open a case with HP to see if they know of some patch for LVM I might be missing or perhaps this by design (god I hope not).

Thanks,
-Dave

[Dave Johnson_1]

How did the rabbit in the hat get attached to this question. I do not recall setting it and yet there it is??

[Dave Johnson_1]

Ignore my last comments. It is because I assigned 8 points or more to one or more responses. Dah!

[Jeff Schussele]

Hi Dave,

Out of curiosity can we see
lvdisplay /dev/vg06/lvol1 ?

Also you should diskinfo EACH of those newly added 4 LUNs. I'm wondering if one of them is 14GB & the other is a "gatekeeper" LUN.
That would certainly cause the symptoms you're seeing because the striping would prevent using all four.

Rgds,
Jeff

[Dave Johnson_1]

OK,
Here is partial output:
lvdisplay -v /dev/vg06/lvol1 | head -30
--- Logical volumes ---
LV Name /dev/vg06/lvol1
VG Name /dev/vg06
LV Permission read/write
LV Status available/syncd
Mirror copies 0
Consistency Recovery MWC
Schedule striped
LV Size (Mbytes) 28160
Current LE 7040
Allocated PE 7040
Stripes 2
Stripe Size (Kbytes) 64
Bad block NONE
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c5t2d0 1760 1760
/dev/dsk/c7t2d1 1760 1760
/dev/dsk/c5t2d2 1760 1760
/dev/dsk/c7t2d3 1760 1760

--- Logical extents ---
LE PV1 PE1 Status 1
00000 /dev/dsk/c5t2d0 00000 current
00001 /dev/dsk/c7t2d1 00000 current
00002 /dev/dsk/c5t2d0 00001 current
00003 /dev/dsk/c7t2d1 00001 current

Note all these LUNs come from an XP512 disk array. The process that adds them to a vg forces me to provide 2 paths to each LUN and guarantee that each LUN is exactly 7040MB in size. Each LUN can only have one SCSI ID to any given host.
Here is the diskinfo output:
for i in /dev/rdsk/c5t2d*
> do
> diskinfo $i
> done
SCSI describe of /dev/rdsk/c5t2d0:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdsk/c5t2d1:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdsk/c5t2d2:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdsk/c5t2d3:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdsk/c5t2d4:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdsk/c5t2d5:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdsk/c5t2d6:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512
SCSI describe of /dev/rdsk/c5t2d7:
vendor: HP
product id: OPEN-9
type: direct access
size: 7211520 Kbytes
bytes per sector: 512

Note: I am able to eventually get the size increased to the full 56320MB in the striped environment and all is right with the world, it just takes multiple steps to get there.

-Dave

[Jeff Schussele]

Ok - I see it in the later post now.
I believe this is normal in a 2 PV striped LV.
I think you can only add a single stripe extension at a time.
IF you had created the LV as a 4 PV stripe, then you could have added the next 4 all at once.
Not 100% positive, but I'm fairly sure.
If you think about it, you'd *want* to control this. Let's say the 4 new LUNs had 2 on c5 & 2 on c7. If LVM was choosing you may get the 1st extension on c5 & c5 & the 2nd on c7 & c7 - not optimal at all. So I'd certainly want to insure the 1st extension is c5 & c7 & the second was the same.

Rgds,
Jeff

[Dave Johnson_1]

I see where you are going with that.

It has been my experience that LVM will allocate on a FIFO basis, or if you look at the LUN list of the vg, the order they are listed, top down. Since I purposely set up the primary link to alternate between c5 and c7 then the order in the listing is how I want it.

LVM might be designed to not assume an ideal configuration and then only allow smaller allocations. Why then can I add in say 5000MB chunks? While the first few chunks are all on a pair of LUNs, eventually they will cross one pair and continue on the next pair. So LVM can allocate space from more than one pair of LUNs in a 2 stripe setup.

The more I talk to people about this, the more it seems like a bug in lvcreate/lvextend.

-Dave

[Dave Johnson_1]

More information,
I just blew away the entire vg and rebuilt it from all command line commands. When I tried to:
lvcreate -L 56320 -i 2 -I 64 /dev/vg06
I got the following:
Logical volume "/dev/vg06/lvol1" has been successfully created with
character device "/dev/vg06/rlvol1".
lvcreate: Not enough free physical extents available.
Logical volume "/dev/vg06/lvol1" could not be extended.
lvcreate: Couldn't retrieve the list of the physical volumes
belonging to volume group "/dev/vg06".
Run the "lvextend" command to create space on the Logical Volume.

Then I tried to use:
lvextend -L 56320 /dev/vg06/lvol1
and got the following:
lvextend: Not enough free physical extents available.
Logical volume "/dev/vg06/lvol1" could not be extended.
lvextend: Couldn't retrieve the list of the physical volumes
belonging to volume group "/dev/vg06".

I was then able to:
lvextend -L 14080 /dev/vg06/lvol1
lvextend -L 28160 /dev/vg06/lvol1
lvextend -L 42240 /dev/vg06/lvol1
lvextend -L 56320 /dev/vg06/lvol1
each ran with out any errors. I was then able to newfs it and mount it with out any problems.

I will be opening a case with HP about this.

Thanks to all,
-Dave

[Todd McDaniel_1]

no points...

Please post back so we know how it went. :D

[Dave Johnson_1]

I opened the case a few minutes ago. I will update this thread when I have a solution.

Stay tuned,
same bat channel,
different bat time,
-Dave

[Jeff Schussele]

Holy Extents - Batman!
I can't wait!

Robin

[Tor-Arne Nostdal]

I just have to hang on to this thread :-))
I'd love to see the reply ... he-he

/2r

[Dave Johnson_1]

The way things are going with the support people so far, I will have to copy the entire case notes, you will not believe it.

Keeping my cool, (for now)
-Dave