inode table overflow

[Jerry L. Sims]

Hello All,

We were monitoring the application server, and
when I executed SAR I noticed the "inode table"
seems to be overflowing, at least that's what I assume is happening. Has anyone seen this before? , and what was the solution ? Thanks

********************
HP-UX it-hp54 B.11.11 U 9000/800 ( rp7410 )
********************

( sar -v ): SIZE AND LOAD ON THE SYSTEM TABLES


proc-sz The current-size and maximum-size of the
process table;

inod-sz The current-size and maximum-size of the
inode table (inode cache);

file-sz The current-size and maximum-size of the
system file table;

proc-ov The number of times the process table
overflowed (number of times the kernel
could not find any available process
table entries) between sample points;

inod-ov The number of times the inode table
(inode cache) overflowed (number of
times the kernel could not find any
available inode table entries) between
sample points;

file-ov The number of times the system file
table overflowed (number of times the
kernel could not find any available file
table entries) between sample points.

HP-UX it-hp54 B.11.11 U 9000/800 03/23/04

16:00:11 text-sz ov proc-sz ov inod-sz ov file-sz ov
16:05:01 N/A N/A 176/2064 0 2632/2632 0 3151/8202 0
16:10:00 N/A N/A 177/2064 0 2632/2632 0 3146/8202 0
16:15:11 N/A N/A 189/2064 0 2632/2632 0 3178/8202 0
16:20:00 N/A N/A 177/2064 0 2632/2632 0 3150/8202 0
16:25:01 N/A N/A 178/2064 0 2632/2632 0 3280/8202 0
16:30:05 N/A N/A 184/2064 0 2632/2632 0 3320/8202 0
16:35:01 N/A N/A 182/2064 0 2632/2632 0 3343/8202 0
16:40:01 N/A N/A 181/2064 0 2632/2632 0 3298/8202 0
16:45:06 N/A N/A 182/2064 0 2632/2632 0 3300/8202 0
16:50:06 N/A N/A 179/2064 0 2632/2632 0 3127/8202 0
16:55:00 N/A N/A 179/2064 0 2632/2632 0 3119/8202 0
17:00:21 N/A N/A 178/2064 0 2632/2632 0 3075/8202 0
17:05:01 N/A N/A 176/2064 0 2632/2632 0 3042/8202 0
17:10:00 N/A N/A 176/2064 0 2632/2632 0 3050/8202 0
17:15:05 N/A N/A 178/2064 0 2632/2632 0 3044/8202 0

[Jerry L. Sims]

also, what actually is an "inode" ?

[Jeff Schussele · Accepted solution]

Hi Jerry,

Well technically your not overflowing - you're maxxing out. IF you were the next column would be >0

I suspect the app detects the limit & doesn't toe over the line.

None-the-less you should increase ninode either with a hard set value or if still at default, by upping the maxusers kernel parm.
Don't step it way up as it is one of the heavier kernel memory loads, but step it up a little & see if the app toes up to that as well. Then go a little higher until it stops maxxing it out.

Rgds,
Jeff

[Rita C Workman]

Take another look it is not overflowing...

your inode 2632/2632 shows the overflow as= 0

So there were no overflows...and remember that had it overflowed you'd also have gotten a message in syslog.

The inode table is constantly being recycled with the system taking the next available inode as it becomes freed up. So that is why you see that all of them are in use....you just can't tell from this view that some of those numbers are actually now in-active and ready for the next request.

Rgrds,
Rita

[Jeff Schussele]

From the ninode man page

http://www.docs.hp.com/cgi-bin/fsearch/framedisplay?top=/hpux/onlinedocs/B3921-90010/B3921-90010_top.html&con=/hpux/onlinedocs/B3921-90010/00/21/2147-con.html&toc=/hpux/onlinedocs/B3921-90010/00/21/2147-toc.html&searchterms=ninode&queryid=20040324-124322
=========
DESCRIPTION

ninode defines the number of slots in the inode table, and thus the maximum number of open inodes that can be in memory at any given time. The inode table is used as a cache memory. For efficiency reasons, the most recent ninode (number of) open inodes is kept in main memory. The table is hashed.

Each unique open file has an open inode associated with it. Therefore, the larger the number of unique open files, the larger ninode should be.

Related Parameters

The default value of ninode is defined in terms of maxusers and npty. The value of nproc should be considered.
======

It's a huge performance boost if the inode address (file location) is actually in memory.

Rgds,
Jeff

[RAC_1]

Kernel parameter ninode applies only to HFS files systems. that brings question to my mind dfo you have hfs file systems?(apart from /stand)

Anil

[Rita C Workman]

Looks like the definition has been given...

..here's a little tip on the ninode parm.

It get's it's value based on a calculation of of other parms (like maxusers and maxproc or something) ... in other words it creates this huge number. The fact is 'generally' you don't need that much. So you may consider lowering it. I start at 2048 and if I see that I truly do ...overflow...then I increase it a little, until I don't overflow. And the difference between this figure and the calculation that SAM sets up is .... well amazing. Makes me wonder why they came up with that formulae.

Rgrds,
Rita

...just my 2 cents worth....

[Jeff Schussele]

Hi RAC,

You're confusing the *actual* inode table on the disk for each FS with the inode table cache the kernel creates in memory as it's "shortcut" to open files. IF a file is open & access is requested by a process or thread AND the inode address of that file is cached the kernel can go straight to the block on the disk. IF it's not in the cache then the kernel has to do a physical read(s) of the physical inode table on the disk to get that info - much, much, much, MUCH slower.

Actual inodes are used by both HFS & VXFS, the difference being that they are a fixed value for HFS & dynamic for VXFS. And the kernel's "virtual" inode table can contain entries from both.

Rgds,
Jeff