ias 10G application server, how do you do it?

[Steven E. Protter]

Environment.
rp5450 server 8 GB RAM, tuned by me.

Databse 9.2.0.4 9i Rel 2 database server, patched current and often.

ias 10g as (application server) No patches from oracle yet. All required OS patches and kernel parameter changes implemented.

Problem:

Poor performance.

as 10g has two major components. A database for the as server and the application server.

Our dba set up the database to be owned and started by user oracle, the owner of the 9.2.0.4 database binaries. We change the ORACLE_SID and the ORACLE_HOME but we start with the oracle user.

Peformance is miserable for the AS server.

I was asked to write a startup script for the product suite. I made a mistake, possibly a fortitous one. I accidently used the ias10g user, the owner of the application server binaries, and the psuedo as database server binaries to start the database.

So, we do this:
oracle user starts the as database
ias10g user starts the application user.

Bad peformance.

ias10g user starts both, good performance, but performance degrades over time.

How do you do it? Anyone had a similar issue? If so, what did you do to solve it.

Note, this case is specific to the product sets listed above. OS is 11i v1, also patched to near perfection (so I hope).

Peformance monitoring shows an ocean of shared memory and no resource issues. The box has swap reserved but is not paging at all.

SEP

[Steven E. Protter]

Preliminary results say (duh), gotta start the applicaton servers "psuedo" database with the owner of the binaries.

Current theory has the perfomance problems arising from confusion at just what user owns the shared memory.

Anyone have experience or insight to add?

Have a nice weekend.

SEP

[Victor BERRIDGE]

Hi Steven,
I havent installed IAS10g yes so I dont know what to say with 9 you will have to wait till I return to work and look in my notes...
But I can say this:
We never use oracle to launch anything it is the keeper UID of the software...
we create individual dba who is owner of instance and apps... since we had issues on servers running many instances I struggled to find in shared memory semaphores etc who was who...
this helps alot when it comes to ipcrm...
I remember seiing a lot of open sockets not beeing cleaned properly...

Good luck
Have a nice week-end
All the best
Victor

[Gary Cantwell]

updated subject line as requestewd,

Cheers,

Gary

[Steven E. Protter]

Bumping with prejudice.

Here is additional information. The problem seems to get worse over time. Just letting the box sit there seems to cause issues.

10G seems to have a web server based on Apache 2.0.x

HP-UX 11i has a built in web server based on Apache 1.3.x

Can these two web servers be corrupting each other shared memory? Should i upgrade and replace the Apache 1.3. server. I do actually have some internal docs sitting on the Apache server.

Also, anyone have trouble with shared memory/semaphores slowing down or getting corrupted and have a patch they can recommend.

This is getting annoying.

We seem to be able to duplicate the problem like this:

reports server starts right after the rest of the 10G architecture.

We stop the reports server according to Oracle guidelines. We start it again, via the web interface or command line.

Then we have the problem.

Someone must have experienced this before. I've never thought myself bleeding edge.

SEP

[Alzhy]

SEP,

If you've Measureware.. call it to action. You should be able to track resource use (memory, cpu, IPC, etc..) over time and see which application group (yes you need to group them) is exhibiting the obvious.

With Oracle9i Rel. 2, we've uncovered a number of slow leakers that have been remedied by patches from Oracle.

Also, can't you avoid have both Application Server and Database server on the same boxen so you can tune separately?

I've always opposed Application and DB serving to be handled by a single OS instance (ie. single server). Can't you vPar your box so one instance is DB server and the other the app server?

[Steven E. Protter]

Note: App server and Database server ARE on the same box.

I've been running glance/gpm for days. When the application server slows to a crawl the system blissfully reports no performance problems whatsover.

Quite vexing. Current theory is Apache 1.3 which shipped with the OS is bothering the Apache 2.0 based httpd server that shipped with ias 10G.

Getting ready to open a software call. Lets see if ITRC can beat the response center.

Using a modified version of HP's performance data collection scripts (the one i post all over itrc forums). Not too much I know how to do to collect data on shared memory performance and corruption.

SEP

[Steven E. Protter]

Nelson note: rp5450 servers can't vpar.

SEP

[Alzhy]

Oh.. I thought it was an 5470.. ;-(


Same other suggestions though.


You can try using WLM/PSETs if you've the license..

[Victor BERRIDGE]

Hi Steven,
I unfortunatly cleaned up my mailbox so I have no info left and have to wait till a DBA will be kind enough to spare some time with me...
but here are some perf I collected with your famous script...
You will not see anything of the swinstall because using my own account (that still works... yes hard times lately...)