VxFS

[Mark Huff II]

Error when creating another file on this mount point "vx_noinode - /imgData file system out of inodes".

bdf -i /imgData/.
kbytes - 974585856
used - 652388376
avail - 319680320
iused - 8126464
ifree - 10068640

I installed the patch PHKL_26761 as suggested for hpux 11.11 which gives me the vx_ninode in the kernel parameters and it is set to 0 as suggested.

Why can I still not create another file?

[James R. Ferguson]

Hi:

The patch text notes "vx_ninode allows setting the maximum number of inodes that can be present in the VxFS in-memory inode cache at a time. If set to zero (the default) the size is tuned according to how much physical memory the system has."

Are you being limited by your system's memory? You might try setting a value other than zero, since I cannot find how the calculation is made.

Regards!

...JRF...

[Ted Ellis_2]

do you recall the options that were used to create this logical volume:

lvcreate -L xxx -o /dev/vgxx

[James R. Ferguson]

Hi (again):

Hi:

The patch notes, if I read more carefully for PHKL_26761, note that the predecessor patch PHKL_24783, *removed* the 'vx_ninode' tunable, noting "vx_ninode is documented to be a tunable but it's not implemented in 11.11."

Regards!

...JRF...

[A. Clay Stephenson]

I've never seen this problem (at least in a vxfs filesystem above version 2.0). The filesystem is large enough that I need to ask is this a regular filesystem or is it perhaps an OmniStorage vxfs filesystem that migrates data to secondary storage. The other thing that occurs to me is that may someone reduced the filesystem size and the operation left the filesystem in a less than perfect state. I suppose that an fsck -o full is in order but that is going to take some time on this puppy.

Although I would not expect the above error message, I would check a two more things: ulimit and quotas.

[Mark Huff II]

I set vx_ninode 30,000,000
vx_ncsize is at the default of 1024.

I did a fsck -o full.

I am still recieving the same error.

As for how I created my Logical volume -- I just used the defaults for VxFS through SAM.

As for The Volume group it is set at 16mb extents.

[Mark Huff II]

And as for my staorage?

XP512

[Dietmar Konermann]

VxFS is able to allocate inode structures on disk dynamically. They are organized in Inode Allocation Units (IAU). Maybe it is impossible to allocate additional IAUs due to some fragementation?

I would recommend to try an extent defragemtation using fsadm.

Regards...
Dietmar.

[Mark Huff II]

This volume-group/logical-volume was just created within the past 2 weeks.

There can't be any fragmentation yet...

[Mark Huff II]

I have run one other test..

If I delete a single file(no matter what size) I can create a single file.

[Mark Huff II]

I have done a "find . | grep wc"
on the filesystem in question and have 8,126,461 files on it..


If none of you can help me, then can you suggest how I would start from scratch in building my file system 1tb large able to hold 5 times the number of files in my wc?

Thanks!!!

[John Dvorchak · Accepted solution]

This may be a shot in the dark but was the filesystem created with largefiles or nolargefiles? To find out:

fsadm /dev/vgxx/ryourvol

Look for largefiles or nolargefiles on the "r"vol.

You might be simply running out of inode storage space. Try setting it to largefiles:

fsadm -o largefiles /dev/vgxx/ryourvol

Good luck.

[John Poff]

Hi,

I'm curious. How many directories do you have on that filesystem? What does 'find /imgdata -type d | wc -l' return?

JP

[Mark Huff II]

Thanks John Dvorchak,

I seem to be able to touch a few new files now, but how long will it last?

Thanks again!

[John Dvorchak]

Let me explain a little further. By enabling largefiles you allow the inode file to exceed the 2Gb limit, which it is probably at now. If my math is accurate with 8 million inodes the inode file would be about 2Gb now.

[Dietmar Konermann]

I'm learning interesting new things every day... :)

Indeed, John's assumption is perfectly correct... obviously the structural files (IFIAU) are also affected by the "largefiles" configuration.

This is the official Veritas statement, included in the VxFS 3.x release notes:

Inode Limitation on File Systems Without Large File Support

For a file system to have more than 8 million inodes, you must create it using the largefiles option of mkfs (the fsadm utility can also be used to set the largefiles flag on the file system). See the mkfs_vxfs(1M) and fsadm_vxfs(1M) manual pages for details.

Thanks, John! Well done. I will add a doc to HP's knowledge base also.


Regards...
Dietmar.

[James R. Ferguson]

HI (again):

John D. that was indeed *excellent* insight! Dietmar, addition of this into the TKB, as you have propsed, would be great.

The numbers shown here work well. Mark's 'bdf' showed 8,126,464 inodes used. At a VxFS inode size of 256 that means 2,080,374,784 bytes were (conservatively) consumed by inodes alone; almost the 2GB limitation of non-largefiles filesystems (2GB = 2,147,483,647 bytes). Very interesting!

Regards to all!

...JRF...

[Dietmar Konermann]

Just to make this thread complete...
http://www4.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000064129598