topic Re: SET VOLUME/LIMIT failing in Operating System - OpenVMS

SET VOLUME/LIMIT failing

Malcolm Wade — Thu, 19 May 2005 19:49:34 GMT

I tried to set a DSA volume today for later expansion with the SET VOLUME/SIZE command and it is failing with:

%SET-E-NOTSET, error modifying _DSA224:
-SYSTEM-W-DEVICEFULL, device full; allocation failure

This was the second DSA volume that I did; the first, a shadowset like the first worked fine.

See attached log for process I used.

Thanks,
Malcolm

Re: SET VOLUME/LIMIT failing

Malcolm Wade — Thu, 19 May 2005 19:51:27 GMT

More info; 3 Alpha nodes SAN attached multisite cluster running OpenVMS v7.3-2 with v3 update applied. Storaage is all EVA based.

Re: SET VOLUME/LIMIT failing

John Gillings — Thu, 19 May 2005 20:50:07 GMT

Malcolm,

Probably worth logging a case for this...

That said, the most likely reason for the DECIVEFULL is insufficient, or excessively fragmented free space.

In order to set up the volume for dynamic expansion, the bitmap needs to be expanded. Since it's a primitive file, there are limits on how fragmented it can be (no extension headers allowed), so if there's not enough disk space to create it "contiguous enough", you'll fail with an allocation failure. A bit like the old HEADERFULL errors trying to expand INDEXF.SYS.

Obviously we need to have a closer look at the disk to confirm or deny this guess. Please log a case.

Re: SET VOLUME/LIMIT failing

Malcolm Wade — Thu, 19 May 2005 21:28:44 GMT

Call has been logged. It'll be in your queue by now :-)

Ironically, volume was 'shrunk' yesterday via image backup and restoer etc etc yet the one that worked wasn't!

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Thu, 19 May 2005 21:46:19 GMT

The problem is contiguous space.

Run DFU and look at your largest free chunk.
The expansion you are attempting to do of course means bitmap.sys has to expand and
expand into a single chunk. Parts of
a larger DCL as example follows.

The solution is to:

1) Write a wrapper that makes calls to DFU
, finds the largest chunk size

$ !
$ ! p1 = input DSA
$ !
$ get_largest_free_extent:
$ subroutine
$ dfu report 'p1' /output=dfu.tmp
$ type = "type"
$ pipe type dfu.tmp | search sys$pipe "largest free extent" | ( read sys$input a ; define/job/nolog tlog &a)
$ large_fe_line = f$trnlnm("tlog")
$ large_fe_line = f$edit(large_fe_line,"TRIM,COMPRESS")
$ largest_free_extent == f$element(0," ",f$edit(f$element(1,":",large_fe_line),"TRIM"))
$ delete/nolog dfu.tmp;
$!
$ endsubroutine

2) Do the right thing. Not too large,
not too small but just right:

$ call get_largest_free_extent 'shad_master_name'
$ else
$ freeblocks = f$getdvi(disk,"FREEBLOCKS")
$ cluster_size = f$getdvi(disk,"CLUSTER")
$ assign/user nla0: sys$output
$ assign/user nla0: sys$error
$ call get_largest_free_extent 'disk'
$ endif
$!
$ device = f$extract(1,f$length(disk),f$getdvi(disk,"FULLDEVNAM"))
$ maxblocks = f$getdvi(disk,"MAXBLOCK")
$!
$!
$ skip_set_vol_limit = "n"
$ if largest_free_extent .gt. 100000
$ then
$ calculated_limit = "UNLIMITED"
$ else
$!
$ working_free = largest_free_extent - 100
$ if working_free .lt. 1000
$ then
$ skip_set_vol_limit = "y"
$ else
$ two_billion = 2022113280 ! Using large but not max. DCL doesn't like larger
$ calc_bitmap_size = (two_billion / cluster_size) / 4096
$ if calc_bitmap_size .gt. working_free
$ then
$ calculated_limit = working_free * cluster_size * 4096
$ else
$ calculated_limit = two_billion
$ endif
$ endif
$ endif

3) I write it out for running later

$ write wlog "$ else"
$ write wlog "$ mount/over=(id,shadow) ''device'"
$ if calculated_limit .eqs. "UNLIMITED"
$ then
$ write wlog "$ set volume/limit ''device'"
$ else
$ write wlog "$ set volume/limit=''calculated_limit' ''device'"
$ endif
$ write wlog "$ dismount ''device'"
$ write wlog "$ endif
$ write wlog "$!"
$ else
$ write sys$output ""
$ write sys$output " Cannot set volume limit on ''device'"
$ write sys$output " The largest free extent on that volume is: ''largest_free_extent'"
$ write sys$output ""

What you see above is part of a larger
routine that is creating a routine.

When you are doing several hundred volumes
you kind of want to automate things.
Precious downtime and all that.

It is obvious there
are dangling elses and there is probably
a mismatch on endifs.

One final note, I picked an arbitrary 100000
blocks as a cutoff , bitmap.sys won't get
larger than 65535 allocated. You could tweek
the logic to be precise such that
if there was x contiguous free and bitmaps.sys calculated expansion is y
, etc...

I just wanted it to work.

Rob

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Thu, 19 May 2005 21:50:49 GMT

Gee - should have paid better attention.
I ran into failures on

$ set volume/limit $1$dgannn

So if anyone runs into that...

Rob

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Thu, 19 May 2005 21:59:54 GMT

I'm losing it.. I am distracted by surroundings..

You did this:

Allpriv> set vol/limit DSA224:
%SET-E-NOTSET, error modifying _DSA224:
-SYSTEM-W-DEVICEFULL, device full; allocation failure

That isn't correct. You have to have a physical disk mounted privately - see the example snippets I provided and search for
set volume/limit here:

http://h71000.www7.hp.com/doc/732FINAL/DOCUMENTATION/PDF/aa-pvxmj-te.PDF

Re: SET VOLUME/LIMIT failing

John Gillings — Thu, 19 May 2005 22:27:39 GMT

Rob,

>That isn't correct. You have to have a physical disk mounted privately - see the example snippets I provided and search for
set volume/limit here:

Sorry, not true! Yes the volume needs to be mounted privately, but you CAN mount shadow sets privately and you CAN SET VOL/LIMIT a shadow set (DSA device).

I've been able to reproduce Malcolm's symptom on a DSA device by fragmenting a disk.

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Thu, 19 May 2005 23:00:27 GMT

Hmmmm ... works with DSAs?
Maybe so. But it appears
the documentation has no hints or indications
it works that way:

dismounting a disk.
To use the SET VOLUME/LIMIT command to allocate extra bitmap space, the disk must be mounted
privately. However, once allocated, the volume can be expanded while the disk is mounted as shareable
(MOUNT/SHARE).
You can allocate additional bitmap space whether or not the physical volume has room for expansion. The
commands for allocating extra bitmap size and for expanding the volume size are available in OpenVMS
Alpha Version 7.3â 2. Volumes that use DVE can be used by any AlphaServer or VAX system running
OpenVMS Version 7.2 or later. The following command allocates extra bitmap size on a new volume:
$ INITIALIZE/LIMIT $1$DGAnnn: ! Allocates 1 TB bitmap
The following command allocates extra bitmap size on a mounted volume:
$ SET VOLUME/LIMIT $1$DGAnnn
The default /LIMIT size for both commands is 1 TB, which is also the maximum size currently supported on
OpenVMS. In special circumstances, you may want to specify less.
When additional physical storage is made available (either by adding a larger device to the shadow set and
removing the smaller member, or by increasing the size on the storage subsystem), you can then enter the

----

I'm seeing a number of references to "disk"
and physical storage, the examples revolve
around DGAs.. etc. Perhaps the shadowing
doc folks need to include examples of
DSAs? Or was that a deliberate oversight?

But you mention fragmentation... yes you
are correct - that's
the only time the set volume/limit fails as
the bitmap is fragmented, here is an
email from engineering:

"In the case of SET VOLUME /LIMIT, the XQP needs to find enough contiguous space on the volume to accommodate the new BITMAP.SYS. If the contiguous space is not available the operation fails with SS$_DEVICEFULL".

It looks like either workaround should work if you see this behavior again on a fragmented volume (compressing or using the procedure below [which we came up with
and polished]).

Name GoesHere
HP Services
Storage Technology Center
Mission Critical Solutions Center
Business phone (719) 59n-nnnn

As always, please let me know if you

---

Since it is a fragmentation problem, the
example DCL works around the problem.

There is another way of course - trial and
error. Been there - done that.

Ro

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Thu, 19 May 2005 23:02:19 GMT

Hey, can I get a hat now?
I don't want fig leaves - they are goofy
looking.

Rob

Re: SET VOLUME/LIMIT failing

Malcolm Wade — Thu, 19 May 2005 23:14:39 GMT

Output from DFU is attached; it doesn't look fragmented to me!

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Thu, 19 May 2005 23:19:22 GMT

Nope.. not at all.

Bust out one shadowset member and try
set volume/limit
with ust a physical mounted (like the
documentation shows).

Rob

Re: SET VOLUME/LIMIT failing

Malcolm Wade — Thu, 19 May 2005 23:23:34 GMT

Been there, done that, same problem!

hp are investigating.

Malcolm

Re: SET VOLUME/LIMIT failing

John Gillings — Thu, 19 May 2005 23:46:25 GMT

This case has been reported to engineering

It looks like the expansion limit on this volume HAS been increased to about the right ballpark given the volume and cluster sizes.

I suspect we've got a boundary condition in the arithmetic for this specific case that is resulting in a bogus message.

It was a "W" level message which means "the operation might have worked, but we're not certain". I'll update this thread when we find out for sure.

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Fri, 20 May 2005 00:02:06 GMT

Ah.. your show dev/full shows that
expansion size is already 2 billion. No
wonder it is failing.

Total blocks 104857600 Sectors per track 128
Total cylinders 6400 Tracks per cylinder 128
Logical Volume Size 104857600 Expansion Size Limit 2150449152

Here is what I think you are trying to
do.. expand to a larger drive. Are one
of the two drives in that shadowset
larger than the other?

Dismount the smaller (and skip
to set_vol: below).

If they are the
same size, dismount one, add a larger
drive. Copy completes, dismount the smaller
drive.

Now expand the shadowset to the size
of the remaining large drive (and watch
logical volume size go to total block
size):

$ set_vol:
$ set volume/size dsannn:

Rob

Re: SET VOLUME/LIMIT failing

Rob Young_4 — Fri, 20 May 2005 00:05:35 GMT

Ah ... modify that
last one, I'm missing a point.

Just dismount a drive and add a larger
one, copy completes, dismount the smaller
and set volume/size.

Re: SET VOLUME/LIMIT failing

Ian Miller. — Fri, 20 May 2005 04:18:16 GMT

The disk cluster size is also relevant to this problem. Try init the disk with a smaller cluster size.

Re: SET VOLUME/LIMIT failing

Jan van den Ende — Fri, 20 May 2005 05:27:07 GMT

Ian,

as I understand, a _BIG_ clustersize is NOT an issue!
Only clustersizes below 8 limits the LIMIT & SIZE values to (1/8 TB) * (clustersize).

Proost.

Have one on me.

jpe

Re: SET VOLUME/LIMIT failing

Ian Miller. — Fri, 20 May 2005 07:39:43 GMT

I am told that cluster size > 8 then there is a problem. Try with cluster size = 8.

Re: SET VOLUME/LIMIT failing

John Gillings — Tue, 24 May 2005 20:53:22 GMT

This has been reported to OpenVMS engineering who have confirmed this is a minor bug in XQP. It's in the process of being fixed.

The bug is purely cosmetic - the message is bogus.

The first SET VOLUME/LIMIT on a volume with a cluster size greater than 8 works correctly and no message is issued. The second and subsequent SET VOLUME/LIMIT commands don't change anything, since the volume is already at the maximum expansion limit. However, the commands issue the message:

%SET-E-NOTSET, error modifying _DSA224:
-SYSTEM-W-DEVICEFULL, device full; allocation failure

The command should just return (as it does for cluster size 8 and lower).

Note that there are circumstances where the command will legitimately fail with DEVICEFULL - that means there is insufficient contiguous space for the new BITMAP.SYS file. I imagine this would be rare in the real world. Simple to check... see SHOW DEV/FULL and check that the expansion limit. The exact value for a fully expanded volume will vary with cluster size, but for maximum expansion it should be a 10 digit number beginning with 21. If you're already there, you can ignore the message. If not, then BITMAP.SYS needs a contiguous extent larger than 65536*8/cluster-size blocks.