script to drive cpu utilization

[Shabu Khan_1]

Hi,
I've been tasked with writing some scripts to drive cpu utilization, the idea is to run some tests and see how the database server performs under peak load.

The management decided that I write the scripts, I can write some decent scripts, but I need some direction on how do I go about doing this ...

This is the oracle 9i db running on solaris 9

Here is the plan:
. The script would generate CPU utilization and/or load average that's consistent with production (User, System and WIO percentages as reported by SAR output
. We would then run another (script) process that has a lower priority than the DB, having it consume all available CPU while monitoring the DB to see how it responds.
. We would repeat this, but run the script at the same priority as the DB while again monitoring the DB to see how it responds in that environment.
. Modify the environment and run same tests (double CPU busy, half CPU busy, vary # of connections, etc.)
. Analyze the data that we get from these tests

Please advise.

Thanks,
Shabu

[Michael Tully]

Why not get something like loadrunner to generate CPU activity.

You could use 'glance' to monitor and capture performance.

[Shabu Khan_1]

Thanks Michael for your suggestion.

Most of our testers around here use loadrunner extensively, I can check with them to see if it can accomplish what we are looking for ...


-Thanks,
-Shabu

[Muthukumar_5]

Hai,

Let us come to script to do this requirement.

We can get the CPU utilization rate with sar(1M) command.

We make the process with user specified priority using the rtprio(2) command.

To allocate the processor utilization use the mpsched(1) call.

Create a environment file with your own specification such as priority level for check script and all.

You will get the exact requirement completion with script on using these.

Regards,
Muthukumar.

[Shabu Khan_1]

Thanks Muthu !

But, I guess my question is related to writing the script itself to generate load on a system rather than measuring the performance of the system. We have some decent tools to gather metrics etc after the load has been generated.

I think loadrunner is one option.

Are there any opensource tools available to perform such tasks ?

Thanks,
Shabu

[Bill Hassell]

load is a very ambiguous term in the world of databases. To generate 100% CPU load, you can simply type this at a shell prompt:

while :
do
:
done

If you have 2 processors, then one processor will be at 100% and the other will run otrher tasks--until the opsystem allows a context switch to another program. Now if the database is not compute-bound, this high load may have no effect. You can demonstrate this by running 2x or 3x as many of the busy-loop scripts above as there are processors. A typical database generates a lot of I/O mixed in with computations. Because these are heavily intermixed, the opsystem tends to favor the mixed CPU and I/O programs, so the impact of a 100% CPU-busy program is minimal. The opsystem will typically degrade the priority of compute-bound programs the more they run.

A true load needs to be defined in terms of CPU, memory and I/O (primarily disk). For instance, a bunch of memory hog programs can just allocate RAM and sit there...no CPU activity. But the effect on Oracle while the programs are grabbing RAM will be massive as programs and shared data start paging. So a mix of CPU and disk may be the true measure of a competing load. I would charcaterize the nature of the additional load first (RAM, I/O, CPU) and then design the load generators appropriately.

[Muthukumar_5]

hai,

I am not known about loadrunner tool. But your requirement will be easily solved and progressed by using the preferred calls of sar,rtprio,mpsched using scripts.

Get the processor status with sar and make the script with the user defined priority with rtprio. Schedule the process to access the processor(s) with mpsched. You can measure the performance of your d/b here.

Measure the cpu usage. It will be easy to handle your requirement with this design.

I did not come across on open source tools for this requirement. Let you know if I find the one.

Regards,
Muthukumar

[Shabu Khan_1]

Bill,
Thanks for the details.

I was thinking about the 'while true' loop to generate cpu load but as you said it might not be a valid test in this context.

So do you think writing shell scripts to perform cpu/load, memory and disk I/O would be tedious but a third party tool like a loadrunner type solution would be the best route to take ?

I appreciate your feedback.

Muthu,
Like I said earlier I am not concerned about measuring the performance of the load with sar etc but in generating the load (see Bill's answer)

Thanks,
Shabu

[Bill Hassell]

It would be tedious to duplicate a specific load with scripts. While you could create a loop to generate a CPU load, and then start some disk I/O, the location of the disk I/O may have no effect or it may unfairly saturate a disk channel with sequential I/O, which may not be a fair load.

I would look at loadrunner since it is a specialized tool for exactly this purpose. However, you'll still need to define the test load in terms of I/O, CPU time, RAM, etc.

[Muthukumar_5]

Hai.

You can give the load to CPU with out any memory usage as

cat /dev/urandon >/dev/null

in a infinite loop

Assign the CPU for process to access the 100% with mpsched

mpsched â c /usr/bin/cat /dev/urandom > /dev/null &

Regards,
Muthukuma

[Muthukumar_5]

It is
mpsched -c /usr/bin/cat /dev/urandom > /dev/null &

Problem because of copying line.

[Shabu Khan_1]

Excellent ! Thanks Bill.

I would start looking at Loadrunner to accomplish this task ...

BTW, just curious what are the unix commands that I could use to generate disk I/O apart from the general read/write programs that can be written.

Thanks for your assistance.


Thanks,
Shabu

[Ted Buis]

Even with Loadrunner, I have seen some results, that are not very accurate. The problem, I saw, was that the queries generated with Loadrunner, accessed the same data repeatedly. Since the system has a buffer cache and, in this case, the disk array also had a read cache, after the initial few queries, the real disk I/O was basically done, giving a false high value to the system's scalability. To get good results, it takes time and study. I would certainly recommend testing, but you should convey to management that this is a complicated task. Also, Measureware (OpenView Performance Agent?) is available for Solaris. It is a great tool to use to collect what is happening on the system under test.

[Shabu Khan_1]

Thanks Ted ! I value your opinion.

I placed my proposal with inputs from you guys to management and they understand that this is a complex task.

I'll let you folks know the final decision and the route that we will take to accomplish this requirement.

Man ! I love this forum.

Thanks again guys !

-Shabu

[Sundar_7]

Hi Shabu,

Here is how I do it.

Compile this C program and run with nohup.

Beleive me this will make ur CPU go crazy spinning down begging for mercy. :-)

#include
#include
main()
{
int i=0;
float X[1024],Y[1024];

while (1)
{

for(i=0;i<1024;i++)
{
X[i]=1;
Y[i]=X[i] * M_PI; # M_PI is defined in /usr/include/math.h
}
}
}

[Shabu Khan_1]

Thanks Sundar !

I compiled the program on my sun blade workstation and executed the binary and the cpu load was around 70%-80% range.

I had to define M_PI inside of your 'c' program for some reason it didn't like it being in math.h ...

a while true loop like Bill mentioned performed the same way like the 'c' program ...

Thanks for your input anyways ..

-Shabu