Very Large File System

[Michael Denney]

Hi,

Considering create a single very large logical volume and a single file system on this lvol. Large as in 3TB. The lvol would be extent stripped. The data usage would be for a database indexes and tables.

Looking for thoughts on pros and cons. I have my ideas about pros and cons, interested in your ideas.

Thanks for your time.
Thanks for the thoughts!

[Michael Denney]

Well, 2tb is largest supported.

[DCE]

Michael

I believe I ran into a 2 TB limit when I tried this last year.

I did use extent stripping on the 2 TB lvol, and it works fine.

[Steven E. Protter · Accepted solution]

Pro:

All the data is in one place.

It should be easy to find.

You can to 2 TB, though I thought the limit was 128.

Good performance is possible, depending on the underlying configuration of the disk array, as you can't get this much disk into a server.

Con:

If the filesystem is lost a whole lot is lost and it could take weeks to restore the data if your backup/recovery system does single threaded restore.

An awful lot of eggs in one basket.

It will probably take a long time to run newfs -F vxfs -o largefiles ...

SEP

[Alzhy]

Michael,

What Storage Array and what HP-UX release do you plan to do this on?

FYI, VxFS 3.5 (a.k.a Online JFS) can scale well beyond 2TB (in fact it can scale to 16TB)!

I am not sure if LVM canhave lvols in excess of 2TB but I am pretty sure VxVM has that ability.

[Michael Denney]

Array is emc dmx3000. This is just an idea, probably won't actually do this configuration. Stephen voiced my same exact concern. One very large file system will make a file system restore take a very long time. Course, could do logical volume backup intead..

[James R. Ferguson]

Hi Michael:

If you are committed to building a 3TB single filesystem, you will need to be running 11.23 with the September 2004 patch set; plus Online JFS version 3.5; and *not* LVM. You must use VxVM for the volume managment. See Table-2 here:

http://docs.hp.com/en/5971-2383/5971-2383.pdf

Regards!

...JRF...

[Alzhy]

If you are using "in-array" snapshot/backup solutions like Business Copy on XP/Hitachi or EVAs - then having a single large filesystem or several smaller volumes does not matter.

Backup solutions these days can multi-stream a single filesystem so that's a non-issue as well.

My practice has always been to settle for standard 200GB volumes -- for database on filesystems -- 200GB as it is sized right for my LTO-II media.

For my RAW storage, it's anything goes or as demands for data storage grows.

[jjh_1]

Pro:

easy to use.

easy to management & backup.

Con:

backup and restore time too long.

If any disk fault then lost the whole data.


* Strongly recommanded raid 5 or raid 1/0.

[Leahi]

I want to do the exact same thing, except, I need 3.5TB for one of my lv. We have an HP MSL 6030 2 Ultrium 960 Tape Library, HP rp4410 servers, and HP VA7110 disk array dual controllers using 2 GB scsi fibre channels.

I found that we have the Base VxVM 3.5 for HP-UX.

We're using HP-UX 11.11i v1.

What patches or upgrades do we need to get the 3.5TB logical volume to work? Are there any considerations in creating LUNs to create the 4 TB volume group?

Any assistance is greatly appreciated.

From a very green HP-UX sys admin.

[Alzhy]

You're cool to use the base VxVM but you will not have PATH protection with the base VxVM. Also - do not install AutoPath as AutoPath does not work with well with VxVM !

Simply patch it up to whatever you can find as latest for VxVM (I believe any latest GoldApps/GoldBase/SupportPlus bundle should contain them).

As far as how to build your mega stiped 3.5TB volume:

0. Present at least one small LUN (say 100MB) to serve as your "ceremonial" rootdg

1. Present however many "EVEN" number of however many sized VA LUNS that will total about ~3.5TB and configure into a diskgroup say named "oradg" (use VEA or vxinstall)

2. Create your volume:

vxassist -g oradg make oramega 3500G layout=stripe ncols=8 stwidth=64

3. mkfs .. vxfs...

4. mount

[Leahi]

Thanks for the reply.

I need autopath. Any other way of doing this?

We also have Command View SDM.

How about stringing 3 logical volumes to create the one 3.5TB logical volume? How can I do this?

[Alzhy]

Hmmm.. perhaps you may just be able to use LVM. You can certainly try creating a striped LVOL with the same properties as I've suggested earlier.

I am pretty sure you will need to upgrade JFS to 3.5 though.

http://h20293.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=BASEVXFS

Theorethical support of up to 32TB is allowed but still 2TB is supported.

For official support of more than 2TB on PARISC systems --- you will need HP-UX 11.23 aka HP-UX 11i Version 2.0. Official VxFS/JFS under said release is 8TB but it could already be 32TB or more (with VxFS 4.1).

[Leahi]

I found out that we have the Base-VxVM 3.5.

I did a swlist -l bundle and did not see VxFS whatever version it may be.

How do I find out which VxFS versions do I have?

The newest patch we have is the Gold Base Patches for HP-UX 11i v1, Dec 04.

[Alzhy]

You certainly still have JFS/VxFs 3.3. JFS 3.5 is only available with the OS at 11.23, For 11.11 systems.. you will need to apply the update -- which is a free download at:

http://h20293.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=BASEVXFS

If you have OnlineJFS .. then you are automatically updated to Online JFS 3.5 (aka Full VxFs 3.5). If you do not have Online JFS - then your base JFS (aks base VxFS) will be updated to JFS 3.5.

Either situation - after the update is applied (do not forget to apply the succeeding VxFS patches from ITRC relating to VxFS (do a search on VxFS patches) you should have:

root# swlist|grep -i vxfs
BaseVxFS B.03.50.0 VERITAS File System Bundle for HP-UX
PHCO_29897 1.0 VxFS 3.5-ga15 Command Cumulative Patch 03
PHKL_29896 1.0 VxFS 3.5-ga15 Kernel Cumulative Patch 05


Please let us know if you can indeed go beyond a 2TB volume and filesystem with Base VxVM and JFS/OJFS 3.5.

[Leahi]

Thanks a million!

I'll download patches tonight and patch tomorrow.

I'll let you know either way: success or fail.