Page Out Problem in Linux

[Sajeesh O.K]

Hi

Please help to identify the bottleneck in one of the RHAS Linux server. Attach find the measureware report. There is very less Memory utilization. Still the Pageout Rate is very high. Only bottleneck can see from the report is File Lock Percentage is always 100%, whereas File % is less than 1%. Is this be the reason ? What parameter has to be changed to reduce this value.

Also find the /etc/sysctl.conf and the ulimit output

# ulimit -a
core file size (blocks) 0
data seg size (kbytes) unlimited
file size (blocks) unlimited
max locked memory (kbytes) unlimited
max memory size (kbytes) unlimited
open files 1024
pipe size (512 bytes) 8
stack size (kbytes) 8192
cpu time (seconds) unlimited
max user processes 15231
virtual memory (kbytes) unlimited

# cat /etc/sysctl.conf

# Disables packet forwarding
net.ipv4.ip_forward = 0
# Enables source route verification
net.ipv4.conf.default.rp_filter = 1
# Disables the magic-sysrq key
kernel.sysrq = 0
kernel.msgmax = 16384
kernel.msgmnb = 32768
kernel.msgmni = 2052
kernel.shmmax = 2147483648
fs.file-max = 417145
kernel.sem=250 32000 128 128
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_keepalive_time=300
net.core.rmem_default = 262143
net.core.rmem_max = 262143
net.core.wmem_default=262143
net.core.wmem_max = 262143
net.ipv4.tcp_rmem = 4096 262143 262143
net.ipv4.tcp_wmem = 4096 262143 262143
net.ipv4.tcp_sack = 0
net.ipv4.tcp_timestamps = 0
vm.freepages = 1024 2048 3072
net.ipv4.tcp_max_syn_backlog=8192

Thanks
Sajeesh

[Ranjith_5]

Hi Sajeesh,

I have seen the the excel file report you attached. Only file-lock is showing 100%. Are you running Oracle on on this machine. Let it it be any data base I feel that your the number of file-handlers are less. File-handlers means number of files that the max. Number of open files that you can have on the system.

Pls read the following:-
==========================

Setting File Handles
==========================
The maximum number of file handles denotes the maximum number of open files that you can have on the Linux system.

Setting System Wide Limit for File Handles

The value in /proc/sys/fs/file-max sets the maximum number of file handles or open files that the Linux kernel will allocate. When you get error messages about running out of file handles, then you might want to raise this limit. The default value on RH 2.1AS is 8192.

For an Oracle server it is recommended that the file handles for the entire system is set to at least 65536.

To determine the maximum number of file handles for the entire system, run:
cat /proc/sys/fs/file-max
To determine the current usage of file handles, run:
$ cat /proc/sys/fs/file-nr
1154 133 8192
The file-nr file displays three parameters:
- Total allocated file handles
- Currently used file handles
- Maximum file handles that can be allocted (see also file-max)

The kernel dynamically allocates file handles whenever a file handle is requested by an application, but the kernel does not free these file handles when they are released by the application. The kernel recycles these file handles instead. This means that over time the total number of allocated file handles will increase even though the number of currently used file handles may be low.

The maximum number of file handles can be changed in the proc file system without reboot:
su - root
echo "65536" > /proc/sys/fs/file-max
Alternatively, you can use sysctl(8) to change it:
sysctl -w fs.file-max=65536
To make the change permanent, add or change the following line in the file /etc/sysctl.conf. This file is used during the boot process.
echo "fs.file-max=65536" >> /etc/sysctl.conf

Apart from that you can have a view on the following too and just compare the tuning parametres with your system. I assume here that your machine is running with Oracle database.

http://www.puschitz.com/TuningLinuxForOracle.shtml

Pls reply with your observations.
Have a nice day.

Regards,
Syam

[Ranjith_5]

Sajeesh,

Since this is a linux related issue. If you post this in Linux section of itrc, you can get more results.

regards,
Syam

[Sajeesh O.K]

Shaym

Thanks. But in my system, the file-max is 417145 and only 0.2 % is using currently
I will post this in Linux Forum. We are using informix as databse.

Thanks
Sajeesh