topic Re: Kernel parameter maxfiles_lim showing more than 400% in Operating System - HP-UX

Kernel Parameter mafiles_lim showing more than 400%

Kamaleshan — Thu, 12 Sep 2013 12:10:55 GMT

Hi ,

In our DB server kcusage for maxfiles_lim always showing more than 400% .

Server Model - rx6600 , OS- 11.31

We are having 2 servers which is configured in RCA Oracle clustering. We set both servers kernel parameters are same, (maxfiles_lim = 4096) But in one server maxfiles_lim always showing more than 400% , in another its showing less than 40%. I am not getting any performance issue on servers.

My Questions:

1. What is use of maxfiles_lim ?

2. How kernel allows to use more than configured settings for maxfiles_lim ?

3. What is the resolution?

Regards,

Kamaleshan

Re: Kernel parameter maxfiles_lim showing more than 400%

Dennis Handly — Thu, 12 Sep 2013 22:17:18 GMT

>1. What is use of maxfiles_lim?

Limit run away processes from opening too many files.

>2. How kernel allows to use more than configured settings for maxfiles_lim?

Possibly root processes can bypass the limit?

Re: Kernel parameter maxfiles_lim showing more than 400%

Kamaleshan — Fri, 13 Sep 2013 03:57:47 GMT

>1. What is use of maxfiles_lim?

> Limit run away processes from opening too many files.

See I have two servers both are configured in RCA Clustering with Load balancing , So why only one server is showing high utilization of maxfiles_lim ? Why not another server?

Regards,

Kamaleshan

Re: Kernel parameter maxfiles_lim showing more than 400%

Dennis Handly — Fri, 13 Sep 2013 13:35:24 GMT

>So why only one server is showing high utilization of maxfiles_lim?

Are both configured the same are they running the same programs? What is the max usage number instead of the percentage?

Re: Kernel parameter maxfiles_lim showing more than 400%

Kamaleshan — Sat, 14 Sep 2013 05:52:57 GMT

> Are both configured the same are they running the same programs?

Yes , Oracle 10G is configured & Running on Both Servers with RCA Clustering.

> What is the max usage number instead of the percentage?

Tunable Usage / Setting
======================
maxfiles_lim 49873 / 4096

usage is - 49873

Regards,

Re: Kernel parameter maxfiles_lim showing more than 400%

Dennis Handly — Sat, 14 Sep 2013 08:46:25 GMT

>usage is - 49873

On both?

Re: Kernel parameter maxfiles_lim showing more than 400%

Kamaleshan — Sat, 14 Sep 2013 11:02:25 GMT

Server 1 Usage is - 49873

Server 2 Usage is - 1366

Setting is 4096 for both servers,

Regards,

Re: Kernel parameter maxfiles_lim showing more than 400%

Dennis Handly — Sat, 14 Sep 2013 20:37:44 GMT

>Server 1 Usage is - 49873 Server 2 Usage is - 1366

Looks like Server 1 is different, busier or running different applications.

I'm not sure this is worth worrying about.

Re: Kernel parameter maxfiles_lim showing more than 400%

Kamaleshan — Mon, 16 Sep 2013 04:16:37 GMT

Both the servers are running same applications & I checked Server utilization that is also almost same.

Already informed to my database team they also checking with their end.

Thanks for your replay . If I ll get any update from my team , I ll post.

Regards,

Re: Kernel parameter maxfiles_lim showing more than 400%

Santosh Abraham — Thu, 19 Sep 2013 13:56:31 GMT

One possibility is that maxfiles_lim may have been configured to a higher value previously (maybe 64k), and the files opened by a process could have grown to that value. This value is cached in a per-process data structure and the process can open files upto that value.

Later on when you changed the maxfiles_lim to a lower value, i.e 4096 it will not apply to processes created prior to the new changed value.

kcusage -t maxfiles_lim can be used to determine which are the top 5 processes and their pids, consuming files.

kclog -n maxfiles_lim will tell when the tunable change was applied.

Maybe you can co-relate the two to find out when the process was created. If it was created prior to the tunable change you have your answer.

Re: Kernel parameter maxfiles_lim showing more than 400%

Kamaleshan — Sat, 21 Sep 2013 10:31:41 GMT

Hi Santosh,

Thanks for your Replay,

I checked with kcusage -t maxfiles_lim , i got a below O/P

Tunable Usage / Setting Usage Id Name
=============================
maxfiles_lim 60209 / 4096
60209 5496 racgimon
60207 4505 racgimon
60205 3938 racgimon
60198 4774 racgimon
1633 3845 diaglogd

kclog -n maxfiles_lim - I am not getting any output , there is no tunable was done previously.

Regards,

Re: Kernel parameter maxfiles_lim showing more than 400%

Santosh Abraham — Sun, 22 Sep 2013 06:11:18 GMT

The other possibility is that a privileged user has raised hard limit's via the setrlimit() system call.

From the "Restrictions" section specified in the maxfiles_lim man-page (See http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02261153/c02261153.pdf)

"Restrictions on Changing
The maxfiles_lim tunable is dynamic (tuning will take effect immediately on the running system).
Dynamic changes affect all existing processes in the system except:
· Processes that have more ﬁle descriptors allocated than allowed by the new limit,
· Processes that have speciﬁcally set their limits through a call to setrlimit() or ulimit()."

The 1st one was ruled out, so it's likely the second one.

From the Security Restrictions section of setrlimit() manpage

(See http://h20000.www2.hp.com/bc/docs/support/SupportManual/c02254049/c02254049.pdf)

"Security Restrictions
Raising hard limits with the setrlimit call requires the LIMIT privilege (PRIV_LIMIT). Processes
owned by the superuser have this privilege. Processes owned by other users may have this privilege,
depending on system conﬁguration."

It's quite likely that the 'racgimon' process you listed out has these privileges and has specifically raised it's hard-limits to a much higher value than the system defined limit. It sounds like it's part of the Oracle RAC suite -- you might probably want to check with your Oracle contacts/forums whether the process really needs those many open files or there is a bug in the RAC suite.

From the HP-UX kernel point of view, things are working as designed.

Re: Kernel parameter maxfiles_lim showing more than 400%

Kamaleshan — Tue, 24 Sep 2013 06:16:33 GMT

Hi Santosh,

Two days before we done database switch over activity from one server1 to another server2 ( server1 is having maxfiles_max issue ). After switchover the database to another server & i checked the maxfiles usage on server1 its in normal . Now again we return the database on original server1 , After done this activity i checked maxfiles_max its getting reduced on server1 ,

see the output of maxfiles_max -m & -d.

Even i rebooted the server also this problem not solved then how its getting reduced automatically?

Regards,

Kamaleshan C