System buffer cache issue

[Hoang Chi Cong_1]

Dear all,
First of all, I would like to thank you for all of your supports....
I have some stucks with my system:
We have 4 servers and runing with MC/SG.
All of them connected to a VA 7400.
Now we have a plan to restructure the VA 7400 to encrease the Disk performance.
I have some questions:
Question 1:
We use Oracle 8i database, somebody said to me that, to encrease performance of disk I/O, I need to by pass the system buffer cache...
Because, data has already buffer in Oracle memory so that no need to buffer it once more in system buffer cache. What is experts idea?
And if it is correct, how to by pass the system buffer cache? It is good for Oracle database structure?

Question 2:
We have total 28 disks in VA 7400. Which is the best queue Full Threshold number?
The array is operating in AutoRaid mode (We can not change to RAID 1+0 becsause we don't have enough space to change RAID mode :( !
Question 3:
How many size for a LUN in VA 7400 with optimal performance?
I know that lots of small LUN will encrease VA performance but I don't want to create too much LUN...it is difficult to manage in Host level.
Question 4:
Does anyone have a tool that measure the VA 7400 performance? For example: If we have 28 disk module (spindle) 36GB FC 10K, Block size is 8KB...
How can I get IO/sec, the Respond time, MB/sec...


Please advice me!
All advices welcome!

[A. Clay Stephenson]

1) This will depend upon your data. My experience with HP-UX 11.11 and up is that Oracle actually performs better using fully cooked files on boxes with large amounts of memory. Earlier versions of HP-UX performed well when the buffer cache was bypasswd using convosync=direct,mincache=direct,nodatainlog vxfs mount options
but I have not found that to be true on 11.11.
Keep your buffer cache size reasonable -- somewhere between 800-1600MB for a box with lots of memory.

2) Nothing you are going to do will improve you as much as going to RAID 1/0. That will probably be your best option even if it means buying more arrays.

3) In general, I find that most arrays perform best when each VG contains as many LUN's as there are separate I/O to the host. For example, if you have 2 I/O paths then make 2 equal size LUN's with the primary path to each over a different SCSI path. You then stripe each LVOL in this VG over both LUN's in 64-128 KB or so stripes.
Host based performance tools will APPEAR to
spot an i/o bottleneck but that's only because all the host can know is that a lot of i/o is going to what it thinks is one disk --- nevermind that this LUN might be spread over many physical disks.

[Hein van den Heuvel]

Hmm,

These questions are not very unique. They have alll been answerred many times before. Please google around for the basic asnwers, and then ask for the specifics.

Q1:
Like Clay said, you can use convosync=direct, or the old RAW devices to avoind double buffering. Please not that does NOT make IO faster. It potentially reduce CPU time needed for an IO. Your topic suggests to me you are worried abotu IO time, not CPU time.
RAW, or direct-IO may help CPU time a little if you get it just right. If you get it wrong, it may hurt IO performance terribly, as you just gave up on your safety net!
Just google for: convosync=direct
Check out our ixora friends writings at: http://ixora.com.au/notes/direct_io.htm

Q2: Don't worry. You have bigger problems.

Q3: I like larger luns involving many disks, within the constraint of maximum outstanding IOs/lun. Liek you say... it is just too hard to manage many small luns and balance the load and space over those.


Q4: Again, just google for "disk performance tools" or some such. Toss in an hpux for good measure.
One hit I got found:
http://forums2.itrc.hp.com/service/forums/questionanswer.do?threadId=337566
Be sure to check out 'iozone', but don't forget the a simple 'time dd bs=8k count=100000 ... '
Of course any and all such measurements will only measure the benchmark which may or might not be close to the actual usage. More llikely it is not close. Still, a simple test like dd will give you some good boundary indications.

Hope this helps some,
Hein,

[Ted Buis]

I believe you need OnLineJFS for some of the mount options needed to bypass the OS buffer cache. See attached cookbook.

HP has tools to tell you the VA7400 performance you can expect given all the data on your configuration. If you have much true random I/O, you will be limited by the RAID level and the number of spindles. Rough rule of thumb would be 100 to 120 IO/sec per spindle, and if you mirror, you should cut the number of spindles in half for writes. If you use RAID5 cut the number of spindles by 4. Reads can come from either mirror so you can use all the disks for it.

Latency will vary on writes depending if the cache is full or not. With cache not full you can get burst IO/sec rates much higher than from just the spindle counts, but if you are doing more writes that the back end spindles can do, sustained performance will drop to that of the backend spindles once the cache is full.

[Eric Antunes]

Hi,

From Oracle side, make sure you have good buffer cache ratios:

select (bbc.total_waits*100/(cg.value+dbg.value)) "Buff busy ratio ind. <= 0,007"
from v$system_event bbc,
v$sysstat cg, v$sysstat dbg
where bbc.event='buffer busy waits'
and cg.name ='consistent gets'
and dbg.name='db block gets';

select round(((1-(sum(decode(name, 'physical reads', value,0)) /
(sum(decode(name,'db block gets', value,0))+
(sum(decode(name, 'consistent gets', value, 0))))))*100),2)
|| '%' "Buffer Cache Hit Ratio > 95%"
from v$sysstat;

select round(100 * sum(getmisses) / sum(gets), 2) "Misses Pct < 5%"
from v$rowcache
having sum(gets) > 1000;

select (1-(sum(getmisses)/sum(gets))) * 100 "Hit Ratio"
from v$rowcache;

Best Regards,

Eric Antunes