Operating System - HP-UX
1752687 Members
5402 Online
108789 Solutions
New Discussion юеВ

Re: what is the actual vx_ninode sizing rule

 
SOLVED
Go to solution
BIHAN
Frequent Advisor

what is the actual vx_ninode sizing rule

Hello,
what is the actual rule to size vx_ninode manually ?


10 REPLIES 10
Elena Leontieva
Esteemed Contributor
A. Clay Stephenson
Acclaimed Contributor
Solution

Re: what is the actual vx_ninode sizing rule

This is probably the best reference I have found on setting vx_ninode.

http://h21007.www2.hp.com/dspp/files/unprotected/devresource/Docs/TechPapers/UXPerfCookBook.pdf

If it ain't broke, I can fix that.
BIHAN
Frequent Advisor

Re: what is the actual vx_ninode sizing rule

Thank you but i don't have acces to this pages, i have an european ID.

Could you give them as attachement ?

regards,
Emmanuel
Elena Leontieva
Esteemed Contributor

Re: what is the actual vx_ninode sizing rule

Description of Tunable Parameters for HPUX 11.0 DocId: KBRC00002354 Updated: 5/5/03 11:44:00 AM

PROBLEM
Can you provide information on the following Kernel Parameters?
vx_ninode
vx_noifree


CONFIGURATION
HPUX 11.0



RESOLUTION

Patch PHKL_17953 adds VxFS tunables:

VX_NINODE
VX_NOIFREE


VX_NINODE allows setting the maximum number of inodes
that can be present in the VxFS in-memory inode cache.
If set to zero (the default) the size is tuned
according to how much physical memory the system
has. Decreasing the maximum size of the inode cache
reduces the fragmentation it causes.
To see what size the inode cache is on a running system:
# echo 'vxfs_ninode/D' | adb /stand/vmunix /dev/mem
So that this system will have a maximum of 8000 inodes
allocated in the VxFS cache.

VX_NOIFREE controls whether to free memory from the VxFS inode cache.
If set to zero (the default), inodes are eventually freed
back to the general memory pool if they are unused. If VX_NOIFREE
is non-zero, then memory is never freed from the
VxFS inode cache. It may seem counter-intuitive
to hoard memory to prevent memory problems, but
not freeing the 2KB buckets holding VxFS inodes to
the general memory pool prevents fragmentation of
the pool. Once the maximum size is reached for
the inode cache, VxFS will always re-use older inodes.

How to find the total used and free inodes on the system DocId: UPERFKBRC00009350 Updated: 3/22/02 5:43:00 PM

PROBLEM

How do I find the total number of inodes used on the system? bdf -i shows this
per filesystem but does not add up the total and the sar -v output for inode
cache does not show the number of JFS inodes used. CONFIGURATION
HP-UX 11.0RESOLUTION
First, some important notes here about inodes taken from DocId:
UPERFKBAN00000726 "Introduction to performance tuning for HP-UX":

Inode count as expressed by sar, only refers to the cache size for HFS inodes
not the actual inode count.

The kernel parameter ninode only effects HFS filesystems, JFS (VxFS) allocate
their own inodes dynamically (vx_ninode) based on the amount of available
memory. The number of inodes for a vxfs file systems is unlimited by default.
A JFS inode takes up 256 bytes.

To determine the amount of vxfs inodes allocated (these are not reported by
sar) run :

example:
echo vxfs_ninode/D | adb -k /stand/vmunix /dev/mem

vxfs_ninode:
vxfs_ninode: 64000



The following script simply takes the output of bdf -i and totals the numbers
for all the local filesystems (JFS and HFS). The output of the script looks
something like this:


Total fs space is 111026581

Total used space is 57482142

Total free space is 52289315

Total used inodes is 134232

Total free inodes is 5033952




The following script was written by Mike Saavedra at Hewlett Packard.
----------------------- cut here ------------------------------------
#!/usr/bin/sh

integer t u f iu if
#integer f

t=0
a=0
u=0
iu=0
if=0

bdf -i | grep "^/dev" | while read line
do
set $line
#echo "$2 $3 $4 $6 $7"
(( t = t + $2 ))
(( u = u + $3 ))
(( f = f + $4 ))
(( iu = iu + $6 ))
(( if = if + $7 ))
done

echo "\nTotal fs space is $t \n"
echo "Total used space is $u \n"
echo "Total free space is $f \n"
echo "Total used inodes is $iu \n"
echo "Total free inodes is $if \n"
------------------------------------------- cut here -----------------
BIHAN
Frequent Advisor

Re: what is the actual vx_ninode sizing rule

(My previous answer was to Elena. )

Thank you Clay for the cookbook, great document.

I'm a bit confuse, it's one more value, and it gives me more questions.
(sorry for my syntax, i have difficulty with english)

- the cookbook says 20000
- veritas vxfs doc says ncsize+25% (its 16000 on my system )
- some forum threads says 90% of ninode
(7000 on my system)

Actually i'm triying to free memory on a 12GB system (lot of java).
----------
kmemifo gives
System = 632333
are this 4Kb pages ?

adb gives
vx_ninode: 0
vxfs_ninode: 296960
what is the size of a vxfs inode entry ?
-----------
Steven E. Protter
Exalted Contributor

Re: what is the actual vx_ninode sizing rule

The default for vx_ninode is 0

That lets the system figure out what it should actually be. These are very expensive and you are much better off from a performance standpoint if you just pick a number and hard code it.

Thats what the author of this performance tuning doc told me. As to how I came to the number, we discussed it and agreed on 16000

http://www2.itrc.hp.com/service/cki/search.do?category=c0&docType=Security&docType=Patch&docType=EngineerNotes&docType=BugReports&docType=Hardware&docType=ReferenceMaterials&docType=ThirdParty&searchString=UPERFKBAN00000726&search.y=8&search.x=28&mode=id&admit=-1335382922+1058287113302+28353475&searchCrit=allwords

I can't remember why. We did run some performance data collection and analyze it in advance. I'm attching that script.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
BIHAN
Frequent Advisor

Re: what is the actual vx_ninode sizing rule


Sorry steven i can't acces your link.
European ID, you know... :(

~~~~~~~~~~~~~~~~

Thank you elena for the attachement
my total used inode is 97000
i think the vx_ninode should not exceed this
value.


(it's not the mater of the discution but i changed the script because bdf gives sometimes the information on two lines.)

#!/usr/bin/sh

integer t u f iu if
#integer f

t=0
a=0
u=0
iu=0
if=0

bdf -li | grep "[0-9]%" | while read line
do
set $line
[ $# -gt 8 ] && shift
#echo "$1 $2 $3 $5 $6"

(( t = t + $1 ))
(( u = u + $2 ))
(( f = f + $3 ))
(( iu = iu + $5 ))
(( if = if + $6 ))
done

echo "\nTotal fs space is $t \n"
echo "Total used space is $u \n"
echo "Total free space is $f \n"
echo "Total used inodes is $iu \n"
~~~~~~~~~~~~~~~~
BIHAN
Frequent Advisor

Re: what is the actual vx_ninode sizing rule

thank you to all, for responding, your links helps me a lot.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I did $e in adb to gather system values.

I found that vx_cur_inodes gives the number of cached inodes.

echo vx_cur_inodes/D | adb -k -w /stand/vmunix /dev/kmem

It is usefull to see the usage of the inode cache:

I did some test on some servers to see how it grows.

After a find / it reaches the total of vxfs inodes on the filesystems + D.
D is value between 50 and 1000 (it changes on different server). I don't know what D represent, some kind of special vxfs nodes ?

If VX_NOIFREE = 0 the inodes are freed and vx_cur_inodes decrease.

If vx_ninode is set to a low value (under the total of created inodes) then vx_cur_inodes reach (vx_ninode + d) after a find / .
d is a value who didn't exceed 3 on my working server. Maybe d correspond to old inodes ?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

My conclusion are :
tell me if i am wrong.

On my 12gb server, tuning vx_ninode to free memory is not an issue as the value of vx_cur_inodes is near 12000. And it can't exceed 97000, the total of created inodes. (even if vxfs_ninode is near 260000 when vx_ninode=0)


I think that the memory used by vxfs inode cache is near 256 * vx_cur_inodes ? Maybe more if it is a hash table.


Steven E. Protter
Exalted Contributor

Re: what is the actual vx_ninode sizing rule

I apologize about the doc. Different links Europe and the US.

You might be able to find it like this.

http://itrc.hp.com

Search Knowledge Base

By Doc ID
UPERFKBAN00000726

This may not work, but I need to get a European ID to run doc searches I guess.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com