how many processes can your system handle

[Marc Ahrendt]

are their HP boxes out there that can handle efficiently 5 or 6 thousand processes in the process table?

basically i am wondering if HP-UX starts to degrade non-linearly in performance after getting beyond a certain threshold of processes. i know processes have varying demands and systems have varying hardware ...just do not have the experience to tell my team much since the largest system we have is a K460 that has never gone beyond 2K processes in the proc table (and at 2K it has a performance degradation that the users can notice). if i say faster CPUs to them (which is what i think we need ...now at 180MHz x4), then i would like to be able to back it up with experiences from others and any ways to verify my quick assumption.

[Michael Tully]

Hi Mark,

Handling the shear number of processes is one thing, but what are the processes actually doing is something yet again.
Items such as the CPU usage, memory usage, disk IO can paint a completely different picture about your system. In answer to your question about the number, 5 or 6 thousand is not an abnormal number os processes.

HTH
Michael

[Bill Hassell]

The simple answer is that about 32,000 processes can be handled by HP-UX (limit on PID numbers). Since performance is subjective, the rest of the answer is: "it depends"

If you have a 4-processor system and run 4 programs that compute for 8 hours with no I/O, then the computer is saturated. Add 4 more identical processes and the work will take more than twice as long as the opsystem switches between each process. Add 10x more processes than will fit into memory and swap space will handle the overflow but performance will degrade by 1000:1 or more.

So if you rewrite the majority of your processes to be more efficient, you can easily run 10,000 processes or more...so it all depends.

[doug hosking]

Following up on Bill's comment, the limit
is currently 30,000 processes - not 32768
or some 'intuitive' number. (See
MAXPID or PID_MAX defines in the header
files.)

One mistake I have seen system managers make is to increase maxusers to a value that indirectly causes nproc to exceed 30000. This isn't always noticed by the kernel, and can lead to wasted memory. (You tie up memory
but can't actually use it after the first
30000.)

[Dave Chamberlin]

About 10 months ago we upgraded from a 4 way K460 to a 4-way N (440 MHz). I was routinely seeing CPU performance problems with the K - I have yet to get one on the N. Oracle jobs run 4-6 times faster now. We typically have about 1000 processes on the machine. Your performance problems may not be related to the number of processes - but the resouces those processes consume - namely RAM and IO.

[Shannon Petry · Accepted solution]

As a consultant, I see these things quite a bit. You have to look at the complete picture to get a good answer. The number of processes really means little, as there is no indicator of what those processes are doing.
I.E.
Web/App servers are usually more network bound. If your network utilization is high, then invest here.

FEA(Finite Element Analysis) is not only very CPU and Memory intensive but disk intensive as well. (hard to get the balance). However disk I/O peaks at runs in 90-100%, and CPU peaks at 90-100%. A lower number in either side at peak time means the other is having problems.

Database servers are usually more disk intensive than cpu intensive. CPU can peak very high but will normally be a few percentage points behind disks in utilization. If your running a database server, and CPU usage is high but disk utilization is low, then adding CPU's can help.

Food before investing. Here is where you have to think quite a bit. Is your money better spent on outdated technology(I.E. 180Mhz Processors) or are you better off to get better/faster I/O, and newer processors/RAM.

In most cases, the investement in old CPU's will buy you a bit of time, but not much. You will see a fractional (at best) performance difference going from 4way-6way, but never enough to make a DBA/Programmer or engineer happy with the upgrade.

I have seen 2way L-Class machines put 4way K-Class machines literally in the dirt when it comes to both FEA and Database applications.

Remember that not only has the CPU Clock cycles increased with time, but I/O paths streamlined/enhanced, disk drive speeds have increased by 140-200%(or more in FCAL), memory is faster and better, etc...

BTW: You should be able to get a demo box from HP before buying. this may help you descide on the N or L class before purchasing one. The L does not expand past 4way, and may not fit your plans for growth.

Regards,
Shannon

[harry d brown jr]

Seriously, drop the K class servers. The N-class and up servers beat the crap out of the K's. The IO backplane of the new servers is far superior, the cpu's are faster, and the memory is faster. The cost of refurbished servers is CHEAP, and HP now remarkets servers.

I benched a K580 (4cpus) against an A500 and an N550 (single cpu ~1.5gb memory - virtually partitioned), of which I found the A and the N to be more than 6 times faster.

Good luck!

live free or die
harry

[John de Villiers]

Just to give more motivation for going to an N-class

We have a V-class with one sap instance and one oracle instance. Our test report runs for 4 hours on it. It has 8 220MHz processors and 4 gig ram

Our N has 11 sap instances, 11 oracle instances and one livechache instance. It has 4 550MHz processors and 12 gig ram. It runs the test report in 55 minutes. I am very impressed by the way the N performs. Our only bottleneck at this stage is Physical RAM. we have 49 of swap and it shows an ave of 70-80 % usage during normal running. Physical memory is always 100%, but thats good. At least you use what you paid for.

As far as processes go, i dont know how many we use at any given time. Its not something i look at regularly.

Also, we dont let users loose on our system, ie no shell accounts. SAP connections only.

John

[Marc Ahrendt]

thx 2 all

1) got a feel about maximum PIDs on a system and some numbers people have seen on their systems
2) the game plan has been to replace our K with an N ...now i feel even better about recommending this to the powers that be