Not-allowed-to-create-logical-volumes-larger-than- 2 TB

[hpuxzapango]

I have 40TB logical volume on HPUX but even after I installed the patch(PHCO_37939) as suggested in this link http://h30499.www3.hp.com/t5/LVM-and-VxVM/Not-allowed-to-create-logical-volumes-larger-than-2TB/td-p/Spell Check4834031 I still see it as 2TB. Any suggestions ? I installed patch but how can I check if it is installed correctly or not?

SCSI describe of /dev/rdsk/c29t0d1:
vendor: COMPELNT
product id: Compellent Vol
type: direct access
size: 2147483647 Kbytes
bytes per sector: 512

[Dennis Handly]

> I installed patch but how can I check if it is installed correctly or not?

Have you tried swlist and swverify on the patch?

swlist PHCO_37939

swverify PHCO_37939

[hpuxzapango]

Yes I verified as you said and it is installed correctly.

Other suggestions?

[Duncan Edmonstone]

According to this whitepaper:

http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c02019992/c02019992.pdf

on 11.23 you also need to install patches PHCO_36744 and PHKL_36745, but even then the maximum size is 16TB

[Matti_Kurkela]

In HP-UX 11.23, the maximum size of a single LUN is 2 TB. This is not a limitation of the filesystem, but of the underlying kernel mass storage driver architecture. (The LVM 1.0 also has a PV maximum size limit of 2 TB - I don't know for sure whether this is a separate limit or just a consequence of the LUN size limit.)

If you need a 16 TB LV in 11.23, you must build it out of 8 or more LUNs = PVs, each sized 2 TB or less.

For a 40 TB filesystem, you will probably need VxVM.

In HP-UX 11.31, there is a totally rewritten storage driver stack and LVM 2.x. As the storage driver stack was rewritten, the 2 TB architectural limit was removed: with 11.31, the LUN size limit is now (theoretically) 8 ZB. That should be enough for a while!

Here is a document on maximum number and size of LUNs in various HP-UX versions:

http://bizsupport1.austin.hp.com/bc/docs/support/SupportManual/c02029814/c02029814.pdf

Here is a document on LVM size limits:

http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02019992/c02019992.pdf

[hpuxzapango]

Hi Duncan,

I checked the patches you mentioned below. (PHCO_36744 and PHKL_36745) and they are also installed correctly. Any other suggestion?

[hpuxzapango]

Hi Matti, 

Our machine have HPUX 11.23.

When I formatted it, only 2 TB is seen. 

is there a guide how to format a 40TB raw disk on HPUX 11.23?

Maybe a guide in internet?

Thanks so much

[Matti_Kurkela]

Sorry, HP-UX 11.23 cannot use raw disks/LUNs larger than 2 TB.

This is a SCSI protocol limit: larger disks/LUNs would need a newer version of the SCSI protocols (also used on FibreChannel, iSCSI, SAS, etc.) to access more than 2 TB. But HP-UX 11.23 cannot do it; the LBA64 protocols required for >2TB disks/LUNs are implemented in HP-UX 11.31 only.

If you need a single disk device that is bigger than 2 TB, you must use LVM or VxVM to build it out of multiple disks/LUNs, each sized 2 TB or smaller.

With LVM, you could build up a 16 TB raw LVM LV out of 2 TB PVs:

• create and present 8 x 2 TB LUNs to the system
• use pvcreate to initialize ("format") the LUNs as LVM PVs
• use "vgcreate -s 256 -e 8192 -p 8 <vgname> <PVs...>" to create a 16 TB LVM volume group
• use "lvcreate" to create a 16 TB LVM LV
• use /dev/<vgname>/r<LVname> to access it as a raw LVM PV.

To have a 40 TB raw disk, you would have to:

• make sure the optional VxVM product is installed
• create and present 20 x 2 TB LUNs to the system
• initialize ("format") the LUNs as VxVM physical disks
• join the VxVM physical disks into a single VxVM diskgroup
• create a single 40 TB VxVM logical volume using that diskgroup
• access the logical volume as a raw VxVM disk /dev/vx/rdsk/<diskgroup>/<volume>

Unfortunately I am not very familiar with VxVM. Perhaps someone else can suggest a good VxVM basic commands guide?

[hpuxzapango]

Thanks Matti for the detailed answer. 

I installed VxVM and I need detailed instructions how to use it for 40TB installation.

Can you give me a link ?

Thanks again

[chris huys_4]

Hi,

NOTES:

1. HP doesnt support with HP-UX any dell diskarray, so also not the dell compellent diskarray.

HP only supports HP own range of diskarrays, xp , eva, msa, p2000, 3par,  and the 3party diskarrays that are in the following list.

http://www.hp.com/products1/serverconnectivity/mass_storage_devices.html

Symantec the developer of vxvm, supports only the dell compellent storage center  5 diskarray from HP-UX 11.31 and vxvm 5.0 onwards. 

So probably go to that hp-ux version and vxvm version, if you want to see something working.

procedure.

[after having installed hp-ux 11.31]

0. and having installed the asl if the dell compellent array is a dell compellent storage center 5 diskarray.

asl is a sort of library that is used by the multipathing of vxvm. called dmp. never use another multipathing product with vxvm .

asl can be downloaded from symantec.com / support & communities/ a-z products / storage foundation for unix/linux / asl and apm .. select hp-ux 11.31/ vxvm 5.0 .

# swinstall -s <asl>

1. configure vxvm

# vxinstall

[answer default answers on the questions. except on the questions of a default diskgroup there answer no.]

2. check if all dell compellent luns are listed 

# vxdisk -o alldgs list

3. change the disk access name to use agile, /dev/rdisk/<>, device names

# vxdiskadm

option 20. 

need to execute option 20 2 times, second time there is the option for specifying the "new" type.

4. result

# vxdisk -o alldgs list

[should now show /dev/rdisk/ disk access names. first column]

[should show the dell compellent luns in online invalid status]

5. vxdisksetup every dell compellent lun.

i.e. put a vxvm header on top of every lun

# /usr/lib/vxvm/bin/vxdisksetup -i <disk<>>

6. result

# vxdisk -o alldgs list

[should now show the vxdisksetup dell luns in online status]

7. create a diskgroup from all initialized dell luns

# vxdg init delldg delldg01=disk<X1> delldg02=disk<X2> delldg03=disk<X3> ... delldg20=disk<X20>

8. create a 40TB vxvm volume

# vxassist -g delldg make dellvol 40000m

9. newfs the 40TB filesystem

# newfs -F vxfs /dev/vx/rdsk/delldg/dellvol

10. mount the filesystem

# mkdir /dellfs

# mount /dev/vx/dsk/delldg/dellvol /dellfs

11. success

# bdf

Best Regards,

Chris