Oracle IO Problems VA7400 or MOD12H

[Aziz Zouagui]

Greetings,

We are having some IO performance bottlenecks running our Oracle Database on top of Autoraid MOD12H.

We have 2 MOD12H(s) connected to an N Class Server. Every AutoRaid unit has 5 Luns, one unit is dedicated to datafiles and the other unit to indexes. All of this but we still have IO problems.

The vendor is talking us into switching to a VA7400 subsystem. I don't know if this will solve our problems but I surely don't want to spend the money and the time to migrate and find out that we have not done anything.

Has anyone here been in the same situation ? any feedback would be greatly appreciated.

Aziz.

[Ray Brewer]

We currently have 2 databases running on a single Autoraid and are not experiencing any I/O problems at all. We are using only a K580 which should be much slower than your "N" class. I wuold suggest that you look at kernel tuning. Oracle has a document that outlines minimum kernel requirements for running a database on HP-UX. You can find it on their web site or your Oracle DBA's should be able to get ahold of it. There are two parameters that are not in that document that should also be modified, dbc_min_pct and dbc_max_pct. We run them at min=3 and max=5 but you may need to adjust that for your enviornment. Hope this helps.

Ray

[Aziz Zouagui]

These are the kernal params you mentioned.

dbc_max_pct 2
dbc_min_pct 2

How many users are connected to your database on average ?

Thank you for the feedback.

Aziz

[A. Clay Stephenson]

Your 12H could definitely be a bottleneck. It almost certainly is a bottleneck if you have allocated over about 60% of the capacity as LUN's and 50% is my usual limit for optimum performance. As long as you stay below 50% capacity as LUN's, the 12H will stay in RAID 1/0 mode. If you exceed this by much, part of your data will be migrated to RAID5 and you will see performance degradation. The other 'trick' is to fully utilize both external SCSI buses. There is absolutely no benefit to exceeding two LUN's per VG on your AutoRAID. Ideally, you configure each VG as two LUN's of identical size. LUNA's primary path is through Controller X (alternate Y) and LUNB's primary path is through Controller Y (alternate X). You then stripe each LVOL within this VG across both LUN's typically in 64K stripes. Finally, you need at least 8 or so drive modules per 12H for best performance. If you follow these steps, you will be plesantly surprised at how well these old guys will perform.

Now that I've said all this there is a very good chance that nothing hardware or tuning related is going to fix your problem. The vast majority of Oracle performance problems reside in the application's SQL code itself. It is not uncommon for an 10x-20x increase in throughput to result from the addition of a single index. Long before you consider switching hardware take a very close look at the code.

[A. Clay Stephenson]

You used that dumb Tuned Parameter Set that sets timeslice (incorrectly) to 1; set it to 10 immediately.

[Aziz Zouagui]

Could you please explain the timeslice parameter you mentioned, I would really appreciate it.

Thank you for your comments.

[A. Clay Stephenson]

The tuned parameter set that incorrectly set the timeslice value to 1 has been around for a very long time and despite the best efforts of several people it still persists. I think it's finnaly been changed in 11.11 but since I don't use it, I don't.

The best definition of timeslice that I've ever heard was given by Stephen Ciullo. He describes it as "the maximum number of ticks that a process ain't gonna get". What this means is that when timeslice is set to 10, your process will get no more than 10 ticks (9) before voluntarily going to sleep so that other processes
can run. Now it timeslice is set to 1, how many ticks does that allow the process to run?

In this state, the system does context switches and very little else. You will see very strange semaphore behavior and in one extreme case, I actually saw that cursor keys could not be recogized because before the ESC [ C (or whatever) sequence could be processed, the process would sleep and the sequence would be read as three distinct characters because curses could not process the sequence.

Set timeslice to 10 before you do anything else on this box.

[Aziz Zouagui]

I value your input, but it is not as easy as 123. The system is 24/7 and a downtime to recompile the kernel and reboot is not another week away.

Bear in mind that I have used this system for over 12 months now with these parameters. nothing like what you mentioned above.

Is this responsible for the IO problems I am suffering from when the Oracle databases are heavily accessed ?

Again, I can't thank you enough for the feedback you're providing here.

Aziz.

[A. Clay Stephenson]

The best I can tell you is that you are wasting your time doing anything until you rebuild the kernel with an increased timeslice. The problem can be well masked until the system begins to load and then it degrades very, very rapidly.

[Ray Brewer]

Sorry I forgot to mention about the timeslice parameter and to make sure it stays at 10. We have around 200 to 300 users at any given time on our databases. I feel you pain in not having time to recompile the kernel.

[Aziz Zouagui]

I will definitely change the timeslice parameter in the next Kernel rebuild.

Is there any other parameter in the list above, that need to be modified, Ray mentioned the DBC_MIN and DBC_MAX values.

Thank you very much everyone.

Aziz.

[harry d brown jr]

Aziz,

First, I'd set these so that the max of dbc_mac_pct would result in around 400M of you memory:

So if you have 4GB, set
dbc_max_pct 10
dbc_min_pct 2


Secondly, the va7400 is light years ahead of the 12H. I had one for 8 months as an evaluation and it rocked. Basically I set it up and walked away from it until I had to give it back to HP! Plus, you'll be switching from SCSI to fibre!


live free or die
harry

[Aziz Zouagui]

I have 8 Gig of RAM and 6CPU.

Do I double the values of dbc_min and max ?

[A. Clay Stephenson · Accepted solution]

If you have 8GB and you want to limit buffer cache to 400MB - a good value for 11.0 - then dbc_max_pct should be set to 5; 2 is okay for dbc_min_pct. In 10.20 and 11.0, you are actually going to see better performance in most cases by setting bufpages
to somewhere between 80000-100000 (320-400MB) and thus disabling dynamic buffer cache.

All of this pales, including faster disk arrays,
into insignificance compared to the effect of your current timeslice setting.

[Aziz Zouagui]

You all convinced me, I will change the following:

dbc_min_pct=2
dbc_max_pct=5
timeslice=10
bufpages=100000

Please let me know if there is anything else.

Thank you all.

aziz.

[A. Clay Stephenson]

There is no need to change the
dbc_xxx_pct values because when bufpages is set to a non-zero value, these parameters are ignored. I would also suggest that you consider installing PHKL_21678 (if applicable to your OS); the disk sort fairness algorithm could be one source of your problems. I wouldn't do the patch at this time because I think you are going to see a marked improvement from the timeslice setting. When that is done, you have a much better chance of correctly spotting other bottlenecks.

[Aziz Zouagui]

What about the nbuf parameter ?

I read somewhere that if you change the bufpages you also need to change the nbuf parameter.

[A. Clay Stephenson]

I had looked at your kernel parameters earlier. Your default setting of 0 for nbuf with a non-zero bufpages settings results in a a reasonable buffer header and pool allocation. It is rather rare that nbuf needs to be also adjusted.