Failure extending beyond 2TB filesystem, HP-UX 11.23

[Don Mallory]

Hi everyone,

I seem to have found a bit of a wall. After perusing some of the other issues, I've pared down some of what it's not...


I'm extending a filesystem beyond 2TB and get an error, whether with fsadm or extendfs, or even a new filesystem.

Lots of LUNs and lots of free space. lvextend is okay.
Volumes are all present.
Nobody has reduced.
No stale extents.
All LUNs are 500GB in size, coming from an EVA.
SecurePath Manager is running (Autopath) v. A.3.0F.02F.02F
All LUNs show and are present with autopath display. Load balancing etc. is okay.

VG info:

--- Volume groups ---
VG Name /dev/vgbig
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 2
Open LV 2
Max PV 100
Cur PV 14
Act PV 14
Max PE per PV 65535
VGDA 28
PE Size (Mbytes) 256
Total PE 27986
Alloc PE 17991
Free PE 9995
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0


lv info:

--- Logical volumes ---
LV Name /dev/vgdbig/lvol2
VG Name /dev/vgdbig
LV Permission read/write
LV Status available/syncd
Mirror copies 0
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 2558720
Current LE 9995
Allocated PE 9995
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict
IO Timeout (Seconds) default

--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c6t9d3 1999 1999
/dev/dsk/c6t9d4 1999 1999
/dev/dsk/c6t9d5 1999 1999
/dev/dsk/c6t9d6 1999 1999
/dev/dsk/c6t10d2 1999 1999


extendfs of existing fs fails:

server2 # extendfs /dev/vgbig/lvol2
UX:vxfs fsck: ERROR: V-3-25316: could not read from block offset devid/blknum 0/2620129279 failed
UX:vxfs fsck: ERROR: V-3-20696: cannot extend aggregate
UX:vxfs extendfs: ERROR: V-3-20141: Invocation of the fsck program terminated abnormally.
The file system is marked bad. Run full fsck manually.
(e.g. fsck -F vxfs /dev/vgbig/lvol2)


New filesystems fail too:

server2 # newfs -F vxfs -o largefiles /dev/vgbig/rlvol3
version 6 layout
2560098304 sectors, 320012288 blocks of size 8192, log size 2048 blocks
largefiles supported
UX:vxfs mkfs: ERROR: V-3-25495: read failure at devid/blknum 0/320012287
UX:vxfs mkfs: ERROR: V-3-21080: vxfs: No such device or address

server2 # mkdir /test
server2 # mount /dev/vgbig/lvol3 /test
/dev/vgbig/lvol3: unrecognized file system


Any ideas?

[Steven E. Protter]

Shalom,

I have read on these forums that the size limit on filesystems was 128 TB, not 2 TB (wow this is however a whopper).

Therefore I must conclude its a bug. The question is whether its discovered (patch exists) or undiscovered.

Before event going to the response center I'd make sure a bi-annual patch set after September 2006 is installed and you do a search for this specific issue on the patch database.

SEP

[Don Mallory]

Hi Scott,

I'm patched:

QPKAPPS B.11.23.0703.059a Applications Quality Pack Bundle for HP-UX 11i v2, March 2007
QPKBASE B.11.23.0703.059a Base Quality Pack Bundle for HP-UX 11i v2, March 2007

Guess I'm off to the response centre...

Don

[Zinky · Accepted solution]

IIRC, 2TB Filesystems are only allowed with VxVM as your Volume Manager *and* VxFS (OJFS) versions greater than 3.5m (4.1 is preferred) and for VxFS Layout >4.0.

Can you send us output from:

fstyp -v /dev/vgbig/rlvol2

[Zinky]

You may want to read this as well:

http://docs.hp.com/en/5991-5853/5991-5853.pdf

And I stand corrected. You need to be at least VxFS (OJFS) 3.5 with Filesystem Layout version 5.0.

fstyp should show you the type of Filesystem layout.

[Zinky]

And assuming you are already at least on VxFS 3.5, the layout upgrade is easy...

Simply use the command vxupgrade. Read the manpage for usage and caveats..

[Sandman!]

Check out the thread below as it has the information relevant to your inquiry:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1120258

[Zinky]

Sandy,

I doubt the LVM LVOL size limitation for 11.23 is still true/current... Note that the post says he was able to expand his LVOL to 2.5TB.

But if indeed true still at 11.23, then I guess he needs to convert to VxVM...

I suggest he get's in touch with HP support whether LVM at 11.23 with the latest and greatest patches supports LVOLS greater than 2TB.. if such is the case, then I am pretty certain he uses Layout 4.0 in VxFS which has a limitation of 2TB...

[Duncan Edmonstone]

See the following whitepaper:

http://docs.hp.com/en/6054/Limits_wp.pdf

Note any given LV on 11.11 or 11.23 will reject IOs beyond 2TB.

I'm pretty confident this is 'up to date' as it also mentions 11iv3.

Looks like VxVM is the only option...

HTH

Duncan

[Don Mallory]

HP backline support also agrees that 2TB is the limit without using VxVM & OnlineJFS.

The next question is whether the Base VxVM is enough or the licensed version is required.

VxVM (vxupgrade) conversion docs:
http://docs.hp.com/en/B7961-90017/ch02s04.html

Good knowledge base article in reference to filesystem limitations attached (KBRC00017241.txt)

I'm going to close this off, but thought it would benefit everyone to have the other references.

Thanks for the help.