topic Re: Strange SAP Performance Issue in Operating System - HP-UX

Strange SAP Performance Issue

Geoff Wild — Thu, 24 Mar 2005 11:53:15 GMT

Basis has just finished doing a kernel upgrade to our DEV landscape. They have their own test box - in which we just refreshed the data from DEV to it. They did the kernel upgrade on their box first. Then, after testing, they did it to our DEV server.
Both servers are HP-UX 11i with same patches.

Here's the strange thing - a job that takes 2 hours to run on their test server - takes 4 hours and 15 minutes on our DEV server.

What is different -well, their test server is a lowly J5600 with 2 552 Mhz cpus and 4 GB of ram and single ended scsi disks.

The DEV server - N4000-44 - with 4 x 440 Mhz cpus, 10 GB of ram and Fibre attached disks on DMX1000.

No swapping on either box.

The DEV server is a "shared" box - but SAP does most of the work - other APPS barely tax the system - so much so - we tend to wonder if they are used at all...

The oracle db is just over 100GB...

I've checked glance and perview data - see nothing to indicate a performance problem...

Any ideas?

Rgds...Geoff

Re: Strange SAP Performance Issue

TwoProc — Thu, 24 Mar 2005 12:06:22 GMT

Geoff, would this new kernel have changed the kernel parameter shmmax ? If the shared memory is cut up into more than 6 segments (from what I've been told), you'll get delays as the database will handle only 6 segments actively at a time, the rest of the requests queue up. Just a thought as to what kernel parameter could drastically change the performance of an Oracle database - but, I'm sure there are others. You should see a large amount of buffer_waits if that is the case.

While I'm thinking of it - have you run statspack yet to see if the large delays can be identified?

Just a thought...

Re: Strange SAP Performance Issue

Geoff Wild — Thu, 24 Mar 2005 12:10:41 GMT

Sorry - unclear - the kernel upgrade is a SAP kernel - not OS kernel.

Rgds...Geoff

Re: Strange SAP Performance Issue

TwoProc — Thu, 24 Mar 2005 12:16:37 GMT

:-)

Oh... Well Then... Nevermind...
- Rosanne Rosanna Danna

Re: Strange SAP Performance Issue

Geoff Wild — Thu, 24 Mar 2005 12:17:30 GMT

No statspack - don't have it...

See attached txt of O/S kernel parms...

Rgds...Geoff

Re: Strange SAP Performance Issue

TwoProc — Thu, 24 Mar 2005 12:37:06 GMT

Statspack - You've more than likely already have it - it's just not installed.
Look in $ORACLE_HOME/rdbms/admin/spdoc.txt to get the instructions. All of the scripts to install it/run it, etc are in the same directory. Suggest creating a separate user with a separate tablespace for this purpose as it can get sizeable after a while (there are purge tools though, so just purge more often if this is an issue).

Re: Strange SAP Performance Issue

Geoff Wild — Tue, 29 Mar 2005 09:28:11 GMT

ping...

Re: Strange SAP Performance Issue

Kenneth Platz — Tue, 29 Mar 2005 13:23:06 GMT

Geoff,

Do you see any "weird" behavior while viewing the systems in glance? Are the systems using higher-than-expected amounts of time in system mode or context switching?

Re: Strange SAP Performance Issue

Geoff Wild — Tue, 29 Mar 2005 13:38:08 GMT

Nothing weird in glance...

The N-class server is using a bit more system time - but we are talking an average of 3.1 % with a high of 9.8 % and a context switch rate average of 0.2 and a high of 0.8 whereas the J5600 has an average system time of 0.8 % and a high of 3.4% and a context switch rate average of 0 with a high of only 0.2...

Rgds...Geoff

Re: Strange SAP Performance Issue

Tony Scully_2 — Tue, 29 Mar 2005 15:29:44 GMT

Geoff,

What is this job - is it a single process that attachs to the database, gets data, crunches it up and writes it back?

If its all in one process and it doesn't have lots of threads of execution then you may not be getting much benefit from the additional CPUs, and the J5600 CPUs are a bit faster.

Would not account for such a massive difference though.

Tony

Re: Strange SAP Performance Issue

Tony Scully_2 — Wed, 30 Mar 2005 17:01:55 GMT

Geoff,

Just wondering if this is resolved? I'd be interested in what happened if it is.

Tony

Re: Strange SAP Performance Issue

TwoProc — Wed, 30 Mar 2005 17:20:34 GMT

Sorry Geoff, I don't understand "ping..." - but I gather that you didn't want to hear more about statspack - please excuse...

Re: Strange SAP Performance Issue

Tim D Fulford — Wed, 30 Mar 2005 17:51:58 GMT

super late...

so If I get this right your test server has 2 550MHz CPUs and your DEV server has 4x 440 MHz CPUs...

I'm guessing here, & this does not account for the whole difference... you have configured the dev as a 2 CPU system?? this would be about 20% performance dfference.

The next thing, the disks, with a job that takes that long sometimes a dub JBOD disks works faster than cleaver fibre/cache because it avoids doing read cache misses. (I'm guessing the job is read intensive). Also if the DMX has less spindles or slower spindles this too can cause slow down

regards

tim

Re: Strange SAP Performance Issue

Sanjay Kumar Suri — Thu, 31 Mar 2005 00:35:23 GMT

SAP-Basis itself offers some very useful transaction codes such as:

ST04
ST02
ST06
ST03

which provide vital clue on performance.

Has the Basis team taken care of requred kernel, SAP, Database parameters as per the SAP notes.

sks

Re: Strange SAP Performance Issue

Geoff Wild — Thu, 31 Mar 2005 08:54:41 GMT

I think Tony was close - the job is single threaded - Basis is happy with that answer.

DEV is a 4 CPU system...with all cpu's running.

Basis has taken care of requred kernel, SAP, Database parameters as per the SAP notes.

Thanks for the help.

Rgds...Geoff

Re: Strange SAP Performance Issue

Geoff Wild — Thu, 31 Mar 2005 08:54:59 GMT

Closing...