topic Re: what is the actual vx_ninode sizing rule in Operating System - HP-UX

what is the actual vx_ninode sizing rule

BIHAN — Tue, 15 Jul 2003 14:28:32 GMT

Hello,
what is the actual rule to size vx_ninode manually ?

Re: what is the actual vx_ninode sizing rule

Elena Leontieva — Tue, 15 Jul 2003 14:39:40 GMT

Hope this docs would help:

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000068403509
http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000062922494

Elena

Re: what is the actual vx_ninode sizing rule

A. Clay Stephenson — Tue, 15 Jul 2003 14:53:34 GMT

This is probably the best reference I have found on setting vx_ninode.

http://h21007.www2.hp.com/dspp/files/unprotected/devresource/Docs/TechPapers/UXPerfCookBook.pdf

Re: what is the actual vx_ninode sizing rule

BIHAN — Tue, 15 Jul 2003 14:56:24 GMT

Thank you but i don't have acces to this pages, i have an european ID.

Could you give them as attachement ?

regards,
Emmanuel

Re: what is the actual vx_ninode sizing rule

Elena Leontieva — Tue, 15 Jul 2003 15:04:36 GMT

Description of Tunable Parameters for HPUX 11.0 DocId: KBRC00002354 Updated: 5/5/03 11:44:00 AM

PROBLEM
Can you provide information on the following Kernel Parameters?
vx_ninode
vx_noifree

CONFIGURATION
HPUX 11.0

RESOLUTION

Patch PHKL_17953 adds VxFS tunables:

VX_NINODE
VX_NOIFREE

VX_NINODE allows setting the maximum number of inodes
that can be present in the VxFS in-memory inode cache.
If set to zero (the default) the size is tuned
according to how much physical memory the system
has. Decreasing the maximum size of the inode cache
reduces the fragmentation it causes.
To see what size the inode cache is on a running system:
# echo 'vxfs_ninode/D' | adb /stand/vmunix /dev/mem
So that this system will have a maximum of 8000 inodes
allocated in the VxFS cache.

VX_NOIFREE controls whether to free memory from the VxFS inode cache.
If set to zero (the default), inodes are eventually freed
back to the general memory pool if they are unused. If VX_NOIFREE
is non-zero, then memory is never freed from the
VxFS inode cache. It may seem counter-intuitive
to hoard memory to prevent memory problems, but
not freeing the 2KB buckets holding VxFS inodes to
the general memory pool prevents fragmentation of
the pool. Once the maximum size is reached for
the inode cache, VxFS will always re-use older inodes.

How to find the total used and free inodes on the system DocId: UPERFKBRC00009350 Updated: 3/22/02 5:43:00 PM

PROBLEM

How do I find the total number of inodes used on the system? bdf -i shows this
per filesystem but does not add up the total and the sar -v output for inode
cache does not show the number of JFS inodes used. CONFIGURATION
HP-UX 11.0RESOLUTION
First, some important notes here about inodes taken from DocId:
UPERFKBAN00000726 "Introduction to performance tuning for HP-UX":

Inode count as expressed by sar, only refers to the cache size for HFS inodes
not the actual inode count.

The kernel parameter ninode only effects HFS filesystems, JFS (VxFS) allocate
their own inodes dynamically (vx_ninode) based on the amount of available
memory. The number of inodes for a vxfs file systems is unlimited by default.
A JFS inode takes up 256 bytes.

To determine the amount of vxfs inodes allocated (these are not reported by
sar) run :

example:
echo vxfs_ninode/D | adb -k /stand/vmunix /dev/mem

vxfs_ninode:
vxfs_ninode: 64000

The following script simply takes the output of bdf -i and totals the numbers
for all the local filesystems (JFS and HFS). The output of the script looks
something like this:

Total fs space is 111026581

Total used space is 57482142

Total free space is 52289315

Total used inodes is 134232

Total free inodes is 5033952

The following script was written by Mike Saavedra at Hewlett Packard.
----------------------- cut here ------------------------------------
#!/usr/bin/sh

integer t u f iu if
#integer f

t=0
a=0
u=0
iu=0
if=0

bdf -i | grep "^/dev" | while read line
do
set $line
#echo "$2 $3 $4 $6 $7"
(( t = t + $2 ))
(( u = u + $3 ))
(( f = f + $4 ))
(( iu = iu + $6 ))
(( if = if + $7 ))
done

echo "\nTotal fs space is $t \n"
echo "Total used space is $u \n"
echo "Total free space is $f \n"
echo "Total used inodes is $iu \n"
echo "Total free inodes is $if \n"
------------------------------------------- cut here -----------------

Re: what is the actual vx_ninode sizing rule

BIHAN — Tue, 15 Jul 2003 15:20:43 GMT

(My previous answer was to Elena. )

Thank you Clay for the cookbook, great document.

I'm a bit confuse, it's one more value, and it gives me more questions.
(sorry for my syntax, i have difficulty with english)

- the cookbook says 20000
- veritas vxfs doc says ncsize+25% (its 16000 on my system )
- some forum threads says 90% of ninode
(7000 on my system)

Actually i'm triying to free memory on a 12GB system (lot of java).
----------
kmemifo gives
System = 632333
are this 4Kb pages ?

adb gives
vx_ninode: 0
vxfs_ninode: 296960
what is the size of a vxfs inode entry ?
-----------

Re: what is the actual vx_ninode sizing rule

Steven E. Protter — Tue, 15 Jul 2003 15:40:03 GMT

The default for vx_ninode is 0

That lets the system figure out what it should actually be. These are very expensive and you are much better off from a performance standpoint if you just pick a number and hard code it.

Thats what the author of this performance tuning doc told me. As to how I came to the number, we discussed it and agreed on 16000

http://www2.itrc.hp.com/service/cki/search.do?category=c0&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&searchString=UPERFKBAN00000726&search.y=8&search.x=28&mode=id&admit=-1335382922+1058287113302+28353475&searchCrit=allwords

I can't remember why. We did run some performance data collection and analyze it in advance. I'm attching that script.

SEP

Re: what is the actual vx_ninode sizing rule

BIHAN — Tue, 15 Jul 2003 16:01:16 GMT

Sorry steven i can't acces your link.
European ID, you know... :(

~~~~~~~~~~~~~~~~

Thank you elena for the attachement
my total used inode is 97000
i think the vx_ninode should not exceed this
value.

(it's not the mater of the discution but i changed the script because bdf gives sometimes the information on two lines.)

#!/usr/bin/sh

integer t u f iu if
#integer f

t=0
a=0
u=0
iu=0
if=0

bdf -li | grep "[0-9]%" | while read line
do
set $line
[ $# -gt 8 ] && shift
#echo "$1 $2 $3 $5 $6"

(( t = t + $1 ))
(( u = u + $2 ))
(( f = f + $3 ))
(( iu = iu + $5 ))
(( if = if + $6 ))
done

echo "\nTotal fs space is $t \n"
echo "Total used space is $u \n"
echo "Total free space is $f \n"
echo "Total used inodes is $iu \n"
~~~~~~~~~~~~~~~~

Re: what is the actual vx_ninode sizing rule

BIHAN — Wed, 16 Jul 2003 12:28:41 GMT

thank you to all, for responding, your links helps me a lot.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I did $e in adb to gather system values.

I found that vx_cur_inodes gives the number of cached inodes.

echo vx_cur_inodes/D | adb -k -w /stand/vmunix /dev/kmem

It is usefull to see the usage of the inode cache:

I did some test on some servers to see how it grows.

After a find / it reaches the total of vxfs inodes on the filesystems + D.
D is value between 50 and 1000 (it changes on different server). I don't know what D represent, some kind of special vxfs nodes ?

If VX_NOIFREE = 0 the inodes are freed and vx_cur_inodes decrease.

If vx_ninode is set to a low value (under the total of created inodes) then vx_cur_inodes reach (vx_ninode + d) after a find / .
d is a value who didn't exceed 3 on my working server. Maybe d correspond to old inodes ?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

My conclusion are :
tell me if i am wrong.

On my 12gb server, tuning vx_ninode to free memory is not an issue as the value of vx_cur_inodes is near 12000. And it can't exceed 97000, the total of created inodes. (even if vxfs_ninode is near 260000 when vx_ninode=0)

I think that the memory used by vxfs inode cache is near 256 * vx_cur_inodes ? Maybe more if it is a hash table.

Re: what is the actual vx_ninode sizing rule

Steven E. Protter — Wed, 16 Jul 2003 12:54:32 GMT

I apologize about the doc. Different links Europe and the US.

You might be able to find it like this.

http://itrc.hp.com

Search Knowledge Base

By Doc ID
UPERFKBAN00000726

This may not work, but I need to get a European ID to run doc searches I guess.

SEP

Re: what is the actual vx_ninode sizing rule

BIHAN — Wed, 16 Jul 2003 13:19:17 GMT

The doccument is not found...
Can you attach it ?
For european readers it is the only way to get this document.

I hope US and European servers will be synchronized in the future.