Problem with root processs

[Karthika_2]

Hi all,
I am facing this in HPUX 11.0 server.
Most of the CPU memory is occupied by root processes and because of this we are not able to work on our application.So , we are planning for hardware upgrade. Here comes my question , after changing the hardware also will the root processes occupy the memory of the server.

Here I am pasting the root processes:
fenton:/home3/oonadm5
=> ps -lu root
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
1003 S 0 0 0 0 128 20 83da28 0 - ? 00:14 swapper
141 S 0 1 0 0 168 20 41536e00 131 400003ffffff0000 ? 00:17 init
1003 S 0 2 0 0 128 20 42128c00 0 101bc58 ? 02:52 vhand
1003 S 0 3 0 0 128 20 42128f00 0 76a060 ? 11:23 statdaemon
1003 S 0 4 0 0 128 20 41510200 0 830ee8 ? 01:12 unhashdaemon
1003 S 0 8 0 0 100 20 42128000 0 b8af60 ? 00:00 supsched
1003 S 0 9 0 0 100 20 42128300 0 830858 ? 00:00 strmem
1003 S 0 10 0 0 100 20 42128600 0 983f68 ? 00:00 strweld
1003 S 0 11 0 0 100 20 42128900 0 830a08 ? 00:00 strfreebd
1003 S 0 12 0 0 -32 20 41510500 0 82d6f8 ? 49:08 ttisr
1003 S 0 18 0 0 147 20 41777600 0 8300c8 ? 00:16 lvmkd
1003 S 0 19 0 0 147 20 41777900 0 8300c8 ? 00:16 lvmkd
1003 S 0 20 0 0 147 20 41777c00 0 8300c8 ? 00:16 lvmkd
1003 S 0 21 0 0 147 20 41777f00 0 8300c8 ? 00:16 lvmkd
1003 S 0 22 0 0 147 20 415df200 0 8300c8 ? 00:16 lvmkd
1003 S 0 23 0 0 147 20 415df500 0 8300c8 ? 00:16 lvmkd
1003 S 0 24 0 0 148 20 415df800 0 611bd0 ? 00:00 lvmschedd
1003 S 0 25 0 0 100 20 415dfb00 0 - ? 00:00 smpsched
1003 S 0 26 0 0 100 20 415dfe00 0 - ? 00:00 smpsched
1003 S 0 27 0 0 100 20 415e7100 0 - ? 00:00 smpsched
1003 S 0 28 0 0 100 20 415e7400 0 - ? 00:00 smpsched
1003 S 0 29 0 0 100 20 415e7700 0 - ? 00:00 smpsched
1003 S 0 30 0 0 100 20 415e7a00 0 - ? 00:00 smpsched
1003 S 0 31 0 0 100 20 415e7d00 0 - ? 00:00 smpsched
1003 S 0 32 0 0 100 20 415f0000 0 - ? 00:00 smpsched
1003 S 0 33 0 0 100 20 415f0300 0 - ? 00:00 sblksched
1003 S 0 34 0 0 100 20 415f0600 0 - ? 00:00 sblksched
1003 S 0 35 0 0 148 20 415f0a00 0 402e4fe0 ? 00:43 lvmdevd
1 S 0 1709 1 0 156 20 426de400 10 4051f450 console 00:00 getty
1 S 0 818 1 0 154 20 43040600 302 832198 ? 00:00 sshd
1 S 0 567 1 16 154 20 4243be00 57 82ec40 ? 01:45:31 syncer
1003 R 0 40 0 0 152 20 415f5900 0 - ? 01:09:55 vxfsd
1003 S 0 56 0 0 148 20 4236f100 0 402f4fe0 ? 00:00 lvmdevd
1003 S 0 57 0 0 148 20 42404100 0 402e6fe0 ? 00:00 lvmdevd
1003 S 0 58 0 0 148 20 417c1e00 0 402e8fe0 ? 00:00 lvmdevd
1003 S 0 59 0 0 148 20 424c2300 0 402ecfe0 ? 00:00 lvmdevd
1003 S 0 60 0 0 148 20 42404400 0 402eefe0 ? 00:00 lvmdevd
1003 S 0 61 0 0 148 20 4264c100 0 402f0fe0 ? 00:00 lvmdevd
1003 S 0 62 0 0 148 20 4264c400 0 402f2fe0 ? 00:00 lvmdevd
1 S 0 837 1 0 154 20 42f9d200 211 832198 ? 00:00 rpcbind
1 S 0 665 1 0 154 20 42991100 21 832198 ? 00:06 syslogd
1 S 0 1299 1 0 154 20 432fce00 96 4324452e ? 00:38 cron
1 S 0 668 1 0 155 20 42b84a00 12 a547ac ? 00:00 ptydaemon
1 S 0 681 1 0 127 20 42971100 8 7a5c9c ? 00:00 nktl_daemon
1003 S 0 842 0 0 152 20 43049800 0 82d100 ? 00:00 nfskd
1 S 0 691 1 0 127 20 42fcc000 9 42fd126c ? 00:00 ntl_reader
1 S 0 692 691 0 127 20 400bab00 13 4bd00500 ? 02:21 netfmt
1 S 0 863 1 0 154 20 4264c700 0 830d58 ? 00:01 biod
1 S 0 881 1 0 154 20 42fa2200 234 832198 ? 00:00 rpc.lockd
1 S 0 864 1 0 154 20 42d45000 0 830d58 ? 00:01 biod
1 S 0 865 1 0 154 20 42431800 0 830d58 ? 00:01 biod
1 S 0 866 1 0 154 20 42d45100 0 830d58 ? 00:01 biod
1 S 0 929 1 0 154 20 42d45200 31 832198 ? 00:44 inetd
1 S 0 875 1 0 154 20 4274b300 262 832198 ? 00:00 rpc.statd
1003 R 0 896 0 0 152 20 43049600 0 - console 00:41 autofs_proc
1 S 0 1246 1 0 64 20 43183400 62 832198 ? 00:00 rbootd
1 R 0 891 1 0 152 20 4302bd00 452 - ? 05:56 automountd
1 S 0 1691 1 0 154 20 43555600 590 435b1068 ? 00:06 discagnt
1 S 0 1234 1 0 154 20 43121f00 757 832198 ? 01:00 rpcd
41 S 0 1259 1 0 120 20 42d26600 97 400003ffffff0000 ? 01:29 xntpd
1 S 0 1280 1 0 154 20 4303b200 62 832198 ? 01:38 pwgrd
1 S 0 1681 1 0 134 20 40084f00 772 - ? 01:28 cdpmgr
1 S 0 1785 1 0 154 20 435d5200 85 832198 ? 00:05 sysstat_em
1 S 0 1509 1 0 154 20 42f9d400 541 832198 ? 00:22 swagentd
1 R 0 1830 1427 0 152 20 42472c00 1143 - ? 01:37 rep_server
1 S 0 1638 1637 0 154 20 4330e600 16 402bda00 ? 00:00 nfsd
1 S 0 1639 1637 0 154 20 43400600 16 402bdbc0 ? 00:00 nfsd
1 S 0 1859 1331 0 154 10 487db100 162 832198 ? 07:47 psmctd
1 S 0 1519 1 0 154 20 4347aa00 69 832198 ? 00:25 emsagent
1 S 0 1340 1 0 154 20 4303b700 11 43399b2e ? 00:00 envd
1 S 0 1336 1 0 154 20 4313a200 65 832198 ? 00:00 psmond
1 S 0 1331 1 0 154 10 42f95400 92 832198 ? 07:01 diagmond
1 S 0 1866 929 0 154 20 487a6200 48 832198 ? 00:00 registrar
1 S 0 1858 1331 0 154 10 487d0900 82 832198 ? 00:47 memlogd
1 S 0 1389 1 0 154 20 43406400 123 832198 ? 00:00 ttd
1 S 0 1503 1 0 127 20 4340b000 806 487d92c0 ? 10:28 scopeux
1 R 0 1427 1 0 152 20 43498800 365 - ? 00:02 perflbd
1 S 0 1504 1 0 154 20 430ca600 415 832198 ? 00:18 pvalarmd
1 S 0 1857 1331 0 158 10 434ba100 73 82ed30 ? 00:21 diaglogd
1 S 0 1711 1 0 168 20 435de700 47 4237ed00 ? 01:10 p_client
141 R 0 1475 1 0 -16 20 42761c00 4644 - ? 02:30:25 midaemon
1 S 0 1528 1 0 154 20 43521400 7 832198 ? 01:13 hpnpd
1 S 0 1672 1 0 154 20 434f0400 153 43357468 ? 00:00 esmd
1 S 0 1689 1 0 154 20 4351e500 180 43535468 ? 00:00 sqlexecd
1 S 0 1637 1 0 154 20 42757b00 16 402bdac0 ? 00:00 nfsd
1 S 0 1624 1 0 154 20 43516c00 138 832198 ? 00:00 rpc.mountd
1 S 0 1640 1637 0 154 20 43572700 16 402bdb40 ? 00:00 nfsd
1 S 0 1641 1637 0 154 20 43578200 16 402bda40 ? 00:00 nfsd
1 S 0 1642 1637 0 154 20 4357ed00 16 402bda80 ? 00:00 nfsd
1 S 0 1643 1637 0 154 20 43585800 16 402bdb80 ? 00:00 nfsd
1 S 0 1644 1637 0 154 20 4358f300 16 402bdb00 ? 00:00 nfsd
1 R 0 1842 1427 0 152 20 42757300 566 - ? 01:03 agdbserver
1 S 0 1742 1 0 154 20 435ece00 87 832198 ? 00:09 disk_em
1 R 0 1845 1842 0 152 20 4878c100 587 - ? 02:32 alarmgen
1 S 0 1771 1 0 154 20 4860d600 85 832198 ? 00:02 dm_core_hw
1 S 0 1764 1 0 154 20 43391600 149 832198 ? 00:30 ha_disk_array
1 S 0 1792 1 0 154 20 434a0b00 77 832198 ? 00:01 scsi123_em
1 S 0 1779 1 0 154 20 435fde00 77 832198 ? 00:01 dm_memory
1 S 0 16445 16402 0 154 20 434fce00 53 53981468 pts/te 00:00 ksh
1 S 0 1799 1 0 154 20 434c4700 85 832198 ? 00:28 dm_stape
1 S 0 16401 929 0 154 20 434e1f00 56 531c1c68 pts/te 00:00 telnetd
1 S 0 16258 929 0 154 20 488feb00 56 52540c68 pts/td 00:00 telnetd
1 S 0 17636 929 0 154 20 50e40200 56 4eb81068 pts/ta 00:00 telnetd
1 S 0 20097 929 0 154 20 48d42d00 56 4dec1c68 pts/tb 00:00 telnetd

[Sagar Sirdesai]

Hi
It looks like the server heavily short of memory and that is the reason you have swapper and vhand process running .

[Ganesan R]

Hi,

In general, root process by means system processes will not consume more than 25% to 30% of system resources. In your case, I would suspect you are running sort of physical memory. That's why swapper and vhand processes are activated.

What happen is when the system running out of memory, these two processes will be activated and consumes most of cpu time to swap in and swap out the data to allote enough space on memory atleast for running processes. So system will not do anything useful other than swapping.

So you need to investigate in deep to find out which process consumes more memory. Based on that you need to work out with application vendor whether those process really need that much memory. Otherwise you need to increase the physical memory.

There are many tools like glance, top, kmeminfo to help you out on this.

[Karthika_2]

Hi Ganesh,

Thank you for response...

My question is , you can see in the list that other the swapper and vhand process many processes are running that too same processes running more than once.

We are planning to change the hardware after changing the harware will these root process will get reduced or do we need to manually kill some processes which is occupying more memory.

[Ganesan R · Accepted solution]

Hi Karthika,

You should not kill any root process unless or until you are sure about that process. I don't think the root process listed here consumes more cpu cycles. For example, for every telnet connection you make, a new telnetd process will spawn. This is the same applicaple to nfsd process or lvm related process.

You can see the amount of cpu used by system process in glance or top. If it is around 25 to 30% then you don't need to kill any root process. it is quit normal. If user process consumes more cpu cycles, then you need to either upgrade the cpu's or kill some process.

Once you upgrade system, your issues will be solved. You don't need to kill any root process. Also make sure that your system is tuned properly. Especially look at the values of dbc_min_pct and dbc_max_pct kernel parameters.

[Karthika_2]

Hi Ganesh,

Thank you very much :)

[Avinash20]

Hi Karthika,

Only the above output is not sufficient to come to a conclusion.

You need to find our if there is CPU/Memory Bottleneck exists on the server, and find out which processes are occupying the memory/cpu

With regards to the above top output, this is normal.

You need to mainly look into the processes which are occupying more physical memory / CPU cycle.

Hardware upgrade might help .. but it always better to investigate.

H/W upgrade will help if the number of users/processes or load on the server have increased.

If the load on the server has not increased, then I would advice you to analyse this further. You could check the top output for CPU taken by System / User, the Free memory available, from kmeminfo the memory which is utilizing more memory.

I have posted the above points, since sometimes, if the application processes are consuming more amount of physical memory then you need to check with the application team if this is normal. Might be tuning of the application might help to reduce the amount the memory to be consumed. If you don't investigate it, then the application process might keep on looping by consuming more memory in which case the upgradation will be of not much use.

[smatador]

Hi,

I have some hpux server, I have no problem of performance and I have quit the same output

Look at this thread for some explanation of the flags of the process http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=844472


Like Avinash say, try so see with glance what is the bottleneck.
If you could not work with glance/gpm, try the good vmstat and I suggest you to take a look to the dbc_max_pct/dbc_min_pct parameter which are fix by default to much on 11.00.