Cache mem hit is always 100%

[chindi]

Hi  we have hpux 11iv3 servers where Cache Memory hit is always 100% .

We are experiencing tremendous performance issues.

Filecache_max = 5%

filecache_min= 3%

Oracle 11g

Checked it from Perfware tool .

PFA log

[Dennis Handly]

>Cache Memory hit is always 100%

This cache hits are usually a good thing.  Unless you want to blame applications for always reading/writing the same thing.

How much memory do you have?  What kind of applications are you running?  What does "swapinfo -tam" show?

[chindi]

Hi Dennis,

root #/ >sw
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 204800 0 204800 0% 0 - 1 /dev/vg00/lvol2
reserve - 58890 -58890
memory 249227 37490 211737 15%
total 454027 96380 357647 21% -

We are getting performance issue since we have migrated from 9i to 11g also from 11iv2 to 11iv3 .

Oracle guy saying he has chekd all parameters , sga , auto mm is disabled .

We are having EMC VNX5500 with fast cache 200 GB SSD  + 24GB internal cache .

We have 2 app servers with 16cores / 256GB RAM .

2 DB Servers with with 16cores / 256GB RAM .

What we have observed is when all batch jobs start at same time , the system just chokes .

Jobs that used to take 40 min to complete , now are taking 120 min .

Everything has gone for toss in this new setup .

EMC analysis showing SP's are unbalanaced , apart from that nothing .

We have installed powerpath in those servers.

[Dennis Handly]

dev 204800 0 204800 0%
reserve - 58890 -58890
memory 249227 37490 211737 15%
total 454027 96380 357647 21%

This says you have way too much device swap.  And plenty of memory.

What is your size of shared memory for your SGA?

http://h30499.www3.hp.com/t5/Databases/shmmax-and-oracle-sga/td-p/3963396

Do you have glance installed?

[chindi]

Hi Dennis ,

We have glance installed.

We have 20 databases on on server.

All SGA mounting to approx 60 GB .

 Can we check something from OS end , some sar statistics .

[chindi]

Hi Dennis ,

when we changed max_queue_depth to 128 , we got desired performance.

We changed it for only 1 lun as of now.

Hi is it safe to change this kernel parameter online ? Can we change it ?

Why this issue was not faced in 11iv2 ?

 PFA sar -H 5 5 which shows the write MB/s is very very low , though its of during off hrs , i will collect same when system is busy and share here .