How to reduce memory requirement.

[John Flanagan]

I have a D270 with 512MB RAM and have recently upgraded it to HP-UX 11i and the database to Oracle 8.1.7.

This is the output of swapinfo
# swapinfo
Kb Kb Kb PCT START/ Kb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 1048576 63784 984792 6% 0 - 1 /dev/vg00/lvol2
reserve - 364352 -364352
memory 389064 248116 140948 64%

I have 25 users using a telnet client to access a terminal application.

I would like to reduce the memory requirement so that the swap space used is reduced to a minimum.

I think I can get a little bit more from my oracle server but I need to test this. Where can I save ram on the HP-UX side?

Thanks,

John.

[Leif Halvarsson_2]

Hi,
The first thing to do is to disable unused functionality (for ex. nfs, system monitoring, CDE) but what you can disable depends on what you use.

[John Flanagan]

What Do I use?

For an Oracle Install:
This requires an graphical interface but not the CDE.
I think nfs is required for the pfs_mount for the ridge rock oracle CD's.

I need vt funcionalaty and telnet server for my application.
Send mail and corn are used to all system check info to my mail account.
rcp is used to copy files between systems.

I have attached my rc.log file

[Kevin O'Donovan]

John,

if you're pushing the envelope a bit on it you could have a look at your system file and remove any drivers that aren't being used. That would reduce the amount of memory used by the kernel. Be careful with that though, only take out drivers you're sure aren't being used.

After shutting down any services you don't need (NFS server/client maybe? NIS?) and removing any drivers you're not using it would be getting down to kernel parameters, but unless you're really stuck for memory I'd leave those alone, could upset Oracle/applications by changing some of these.

Depending on how big the database is and what exactly the users are doing that swap space use doesn't look too bad to me, especially with 25 users on 512MB RAM.

hope that helps,
Kevin.

[John Flanagan]

The problem is that I am getting client terminal screens locking for upto 30 seconds for no reason. I thought this may be due to a memory shortage.

Products on the system include
Oracle 8.1.7
Cognos Powerhouse
Visibility (ERP system)

Visibility is Written in Powerhosue.

here is a sample from top

System: PTIHP1 Fri Sep 5 12:08:46 2003
Load averages: 2.72, 2.46, 2.34
216 processes: 208 sleeping, 8 running
Cpu states:
LOAD USER NICE SYS IDLE BLOCK SWAIT INTR SSYS
2.72 22.6% 0.0% 1.8% 75.6% 0.0% 0.0% 0.0% 0.0%

Memory: 208132K (30536K) real, 503536K (95200K) virtual, 14968K free Page# 1/16

TTY PID USERNAME PRI NI SIZE RES STATE TIME %WCPU %CPU COMMAND
pts/tt 25835 smsk 154 20 11168K 6016K sleep 5:43 16.50 16.47 quick
pts/tc 23858 fnsb 154 20 9104K 3880K sleep 0:39 6.14 6.13 quick
? 25837 smsk 154 20 30476K 1384K sleep 0:59 4.60 4.60 oraclepaulive
pts/tb 23786 dohb 154 20 9216K 3972K sleep 0:58 0.60 0.60 quick
? 0 root 127 20 0K 0K sleep 57:28 0.57 0.57 swapper
pts/tl 29210 root 178 20 580K 564K run 0:00 0.54 0.54 top
? 23860 fnsb 154 20 30348K 1192K sleep 0:06 0.51 0.51 oraclepaulive
? 31 root 152 20 0K 2272K run 19:11 0.39 0.39 vxfsd
? 1447 root 152 20 1964K 1060K run 0:50 0.25 0.25 dmisp
? 22777 oracle 156 20 30556K 1552K sleep 0:07 0.20 0.20 ora_dbw0_paulive
? 22779 oracle 156 20 30556K 1504K sleep 0:06 0.12 0.12 ora_lgwr_paulive

[Massimo Bianchi]

Hi,
you can gain memeory also from kernel parameters a lot, expecially from


dbc_min_pct set to 5
dbc_max_pct set to 8
(since you have oracle)

ninode 4096 or your value
vx_ninode 90%*ninode


this two are to prevent some known memory leak.


Paste your /stand/system for other suggestions :)


Massimo

[Massimo Bianchi]

Hi,
load looks almost correct.

Check also netwok settings (FD, HD, 10, 100) with ladnadmin, and make sure that there is no autonegitiation, this can hurt very much on telnet sessions.

Massimo

[Kevin O'Donovan]

Hi John,

thanks for assigning the points!

Pauses like that I've mostly come across when NFS is involved. Are you mounting an NFS filesystem from another server? The thing I've found with NFS is that the NFS client makes a request to the server and just waits until it gets a response, doing nothing else, which needless to say is a pain if you get a slow server.

Doesn't sound memory related to me, I'd look at either NFS or maybe the database for cause of the pause.

Is it a case of ALL the client terminals lock up at the same time or individually? or just at login time? If all together I'd suspect NFS, if individually I'd suspect the database.

The Oracle/database side I wouldn't be sure about what would slow that down, apart from inefficient joins, table indexes, that kinda stuff but I haven't done anything with it.

Kevin.

[John Flanagan]

Hi,

Here is my /stand/system

I will try sugestions this weekend when I can do a reboot.

Thanks,

John.

[Bill Hassell]

512 megs is way too small to run Oracle. The minimum should be 2,000Mb, more for even better performance. With only 512 megs, your DBA is struggling with a very small SGA which translates into very slow performance. Reducing drivers and a few kernel parameters will only save a few megs, so why bother? The slow performance you are seeing (including the 30 second lockups) is exactly what you would expect with small RAM. There is nothing left in your server to improve without adding RAM and increasing SGA for Oracle.

It is very easy to justify the additional memory expense--just multiply the number of users times the number of seconds per day that they spend waiting for the computer to respond and translate that into lost labor dollars. I believe that Oracle may state 512 megs is a minimum but what is probably missing are statements like: Oracle barely functions, supports only two users at the same time, etc...

[John Flanagan]

Sorry, forgot the attachement

[Massimo Bianchi]

Missing /stand/system :)

Massimo

[Massimo Bianchi]

I see no change in the system, so i suggest to take in consideration my previous hints:

dbc_min_pct set to 5
dbc_max_pct set to 8

ninode 4096
vx_ninode 90%*ninode
(this are only cache)

Massimo

[Bill Hassell]

Also, you no longer need PFA with 11.0 and 11i. There are 3 patches for 11i that add a new option to mount Oracle (and other industry standard CDs) with a new option: -orr. The patches are:

PHKL_28025
PHKL_25760
PHKL_26269

PFS is too unstable to be used on a production machine and NFS is not far behind unless properly managed. You can turn off all the rpc (NFS) services in /eetc/rc.config.d/nfsconf.

[John Flanagan]

I should point out that prior to the upgrade I was running HP-UX 10.2 and Oracle 8.0.5. With this configuration I had 2 datebases on this server of identical size. One for the live system and One for the test system. I was happy with this configuration and only upgraded due to the unsuported versions of Hp-UX and ORACLE.

My test system DB is now on an NT server.

The DB contains about 4GB of data.

[Leif Halvarsson_2]

Hi,
One thing I wonder about. Your users are logged in with telnet. Does this mean that the client is running local. Is it not possible to run this application in client/server mode, acessing the database via sqlnet. This would save a lot of memory.

[Mark Grant]

I have to agree will Bill here. This amount of RAM is just not fair on Oracle and your users. I notice you have Cognos products on your machine and I know that these applications generally consume far more memory than is reasonable. At our office we ended up moving most of our Cognos products over to Windows which is really what they are designed for.

[Pete Randall]

John,

It's a given that newer release of the O/S and DB will have more capabilities and more features. This translates directly to increased resource usage - particularly memory. As others have suggested, I'm amazed you can get Oracle to run at all with just 512MB of RAM. Normally 2GB is the minimum to have a usable system. Tuning, at this point, is not going to save enough memory to make any significant difference. Your two choices, as I see them, are to add RAM to the system, or to regress back to HP-UX 10.20, if possible (though, for support reasons, I would hesitate to recommend this).


Pete

[Jeff Schussele]

Hi John,

As usual Bill is spot-on & I agree 100% with him. 512MB is not near enough for Oracle 8.1.7
It may have sufficed for Oracle 7.x & lower, but is woefully shy for newer Oracles vers. Frankly, I'm surpised that you were even able to install Oracle with that memory total.
You're going to have to bite the bullet & spend some $ to get that system to at least 2GB if you want to see decent performance.

My 2 cents,
Jeff

[Kent Ostby]

John --

Two other suggestions (sorry if the first one already got suggested and I missed it on my read through other responses).

#1) Make sure that you have enough swap space.
I would guess that you do based on only having used 6% above. But if you do add a lot of memory, make sure that you also have at least 2* swap.

#2) The following is not fool proof, but could add some insight into what is going on.

q4 is a tool that we normally use to look at crash dumps, but it could be helpful here as well.

When you see the client's hanging, run q4 as follows:

q4 -p /stand/vmunix /dev/kmem

At the q4 prompt type:

include analyze.pl

then:

run Analyze AU > ana.out

then

include whathappened.pl

then

run WhatHappened -HANG > what.out

then

exit

In the ana.out file, you can do the following:
vi the file and do a forward search for:

^proc

until you find one of your processes that is hanging (it will ahve the run string).

After you find one do a search for FUNC

This will bring you to the stack trace of the process. Post that here and I or someone else might be able to discern what sort of resource you are hanging on.

In the what.out, search down in the file for "Descending List of most frequent waitchannels".

Post that table here (it should be about 10 to 15 lines long) and again, that might point to a specific resource issue.

I'll flag this thread so I'll get email whenever it gets updated and take a look.

Best regards,

Kent M. Ostby

[John Flanagan]

Hi,

I have looked at q4 and it appears I need to run q4pxdb on the kernel first. I have never used these tools and so I am a bit nervous about it.

What impact do these tools have on the kernel and do I have to be in single user mode to run q4pxdb.

also

I have disables nfs and I will see the results either later today or Monday. I will try changing the kernel paramaters after this (possibly Monday evening).

[Kent Ostby]

John -- I'm surprised its asking you to do that on an 11i box.

Just to make you feel a bit better about this, do the following:

cp /stand/vmunix /tmp/vmunix
(or a different filesystem if you dont have enough room in /tmp)

Then do your q4pxdb against this copy and run:

q4pxdb /tmp/vmunix

then:

q4 -p /tmp/vmunix /dev/kmem

Then the commands I posted earlier.

That should work the same way.

Best regards,

Kent M. Ostby

[John Flanagan]

Kent,

When I got q4 to work the start of both files contained file table full errors.

sar -v gives the following

16:37:33 text-sz ov proc-sz ov inod-sz ov file-sz ov
16:37:34 N/A N/A 140/276 0 0/700 0 1323/2010 0

related kernel params:
nfile 2000
fnlocks 300
ninode 700

I will post the other info on Monday.

Have a good weekend all,

Regards,

John.

[John Flanagan]

Kent,

When I got q4 to work the start of both files contained file table full errors.

sar -v gives the following

16:37:33 text-sz ov proc-sz ov inod-sz ov file-sz ov
16:37:34 N/A N/A 140/276 0 0/700 0 1323/2010 0

related kernel params:
nfile 2000
fnlocks 300
ninode 700

I will post the other info on Monday.

Have a good weekend all,

Regards,

John.

[John Flanagan]

Kent,

When I got q4 to work the start of both files contained file table full errors.

sar -v gives the following

16:37:33 text-sz ov proc-sz ov inod-sz ov file-sz ov
16:37:34 N/A N/A 140/276 0 0/700 0 1323/2010 0

related kernel params:
nfile 2000
fnlocks 300
ninode 700

I will post the other info on Monday.

Have a good weekend all,

Regards,

John.