topic VxFS in Operating System - HP-UX

VxFS

Mark Huff II — Tue, 17 Sep 2002 17:36:53 GMT

Error when creating another file on this mount point "vx_noinode - /imgData file system out of inodes".

bdf -i /imgData/.
kbytes - 974585856
used - 652388376
avail - 319680320
iused - 8126464
ifree - 10068640

I installed the patch PHKL_26761 as suggested for hpux 11.11 which gives me the vx_ninode in the kernel parameters and it is set to 0 as suggested.

Why can I still not create another file?

Re: VxFS

James R. Ferguson — Tue, 17 Sep 2002 17:52:58 GMT

Hi:

The patch text notes "vx_ninode allows setting the maximum number of inodes that can be present in the VxFS in-memory inode cache at a time. If set to zero (the default) the size is tuned according to how much physical memory the system has."

Are you being limited by your system's memory? You might try setting a value other than zero, since I cannot find how the calculation is made.

Regards!

...JRF...

Re: VxFS

Ted Ellis_2 — Tue, 17 Sep 2002 18:03:35 GMT

do you recall the options that were used to create this logical volume:

lvcreate -L xxx -o /dev/vgxx

Re: VxFS

James R. Ferguson — Tue, 17 Sep 2002 18:05:17 GMT

Hi (again):

Hi:

The patch notes, if I read more carefully for PHKL_26761, note that the predecessor patch PHKL_24783, *removed* the 'vx_ninode' tunable, noting "vx_ninode is documented to be a tunable but it's not implemented in 11.11."

Regards!

...JRF...

Re: VxFS

A. Clay Stephenson — Tue, 17 Sep 2002 18:19:24 GMT

I've never seen this problem (at least in a vxfs filesystem above version 2.0). The filesystem is large enough that I need to ask is this a regular filesystem or is it perhaps an OmniStorage vxfs filesystem that migrates data to secondary storage. The other thing that occurs to me is that may someone reduced the filesystem size and the operation left the filesystem in a less than perfect state. I suppose that an fsck -o full is in order but that is going to take some time on this puppy.

Although I would not expect the above error message, I would check a two more things: ulimit and quotas.

Re: VxFS

Mark Huff II — Wed, 18 Sep 2002 11:08:19 GMT

I set vx_ninode 30,000,000
vx_ncsize is at the default of 1024.

I did a fsck -o full.

I am still recieving the same error.

As for how I created my Logical volume -- I just used the defaults for VxFS through SAM.

As for The Volume group it is set at 16mb extents.

Re: VxFS

Mark Huff II — Wed, 18 Sep 2002 11:15:28 GMT

And as for my staorage?

XP512

Re: VxFS

Dietmar Konermann — Wed, 18 Sep 2002 13:10:10 GMT

VxFS is able to allocate inode structures on disk dynamically. They are organized in Inode Allocation Units (IAU). Maybe it is impossible to allocate additional IAUs due to some fragementation?

I would recommend to try an extent defragemtation using fsadm.

Regards...
Dietmar.

Re: VxFS

Mark Huff II — Wed, 18 Sep 2002 13:36:20 GMT

This volume-group/logical-volume was just created within the past 2 weeks.

There can't be any fragmentation yet...

Re: VxFS

Mark Huff II — Wed, 18 Sep 2002 13:41:08 GMT

I have run one other test..

If I delete a single file(no matter what size) I can create a single file.

Re: VxFS

Mark Huff II — Wed, 18 Sep 2002 15:44:41 GMT

I have done a "find . | grep wc"
on the filesystem in question and have 8,126,461 files on it..

If none of you can help me, then can you suggest how I would start from scratch in building my file system 1tb large able to hold 5 times the number of files in my wc?

Thanks!!!

Re: VxFS

John Dvorchak — Wed, 18 Sep 2002 20:12:35 GMT

This may be a shot in the dark but was the filesystem created with largefiles or nolargefiles? To find out:

fsadm /dev/vgxx/ryourvol

Look for largefiles or nolargefiles on the "r"vol.

You might be simply running out of inode storage space. Try setting it to largefiles:

fsadm -o largefiles /dev/vgxx/ryourvol

Good luck.

Re: VxFS

John Poff — Wed, 18 Sep 2002 20:20:43 GMT

Hi,

I'm curious. How many directories do you have on that filesystem? What does 'find /imgdata -type d | wc -l' return?

JP

Re: VxFS

Mark Huff II — Thu, 19 Sep 2002 09:22:12 GMT

Thanks John Dvorchak,

I seem to be able to touch a few new files now, but how long will it last?

Thanks again!

Re: VxFS

John Dvorchak — Thu, 19 Sep 2002 21:32:59 GMT

Let me explain a little further. By enabling largefiles you allow the inode file to exceed the 2Gb limit, which it is probably at now. If my math is accurate with 8 million inodes the inode file would be about 2Gb now.

Re: VxFS

Dietmar Konermann — Fri, 20 Sep 2002 07:03:37 GMT

I'm learning interesting new things every day... :)

Indeed, John's assumption is perfectly correct... obviously the structural files (IFIAU) are also affected by the "largefiles" configuration.

This is the official Veritas statement, included in the VxFS 3.x release notes:

Inode Limitation on File Systems Without Large File Support

For a file system to have more than 8 million inodes, you must create it using the largefiles option of mkfs (the fsadm utility can also be used to set the largefiles flag on the file system). See the mkfs_vxfs(1M) and fsadm_vxfs(1M) manual pages for details.

Thanks, John! Well done. I will add a doc to HP's knowledge base also.

Regards...
Dietmar.

Re: VxFS

James R. Ferguson — Fri, 20 Sep 2002 11:30:30 GMT

HI (again):

John D. that was indeed *excellent* insight! Dietmar, addition of this into the TKB, as you have propsed, would be great.

The numbers shown here work well. Mark's 'bdf' showed 8,126,464 inodes used. At a VxFS inode size of 256 that means 2,080,374,784 bytes were (conservatively) consumed by inodes alone; almost the 2GB limitation of non-largefiles filesystems (2GB = 2,147,483,647 bytes). Very interesting!

Regards to all!

...JRF...

Re: VxFS

Dietmar Konermann — Mon, 30 Sep 2002 14:28:42 GMT

Just to make this thread complete...
http://www4.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000064129598