topic Re: Disk Immediate Reporting in Operating System - HP-UX

Disk Immediate Reporting

Tim Medford — Thu, 06 Jun 2002 14:59:15 GMT

Does anyone have an opinion or horror story they'd like to share on the kernel parameter default_disk_ir?

I have always had this parameter set to zero (off) since that is the recommended setting from HP. Yesterday a group of consultants who were looking at our kernel config dinged us in a report to mgmt. They claim this should be turned on to maximize database performance.

At first we were defensive about our config, but upon reflection, maybe they are right? We are on an L3000 server, HPUX 11.0, connected to an AutoRAID 12H. The AutoRAID has dual 96mb controllers.

I set up everything in LVM very carefully to balance across the controllers. I'm using extent-based striping and a few other tricks to balance the load. All of our statistics look excellent, no hot-spots. The Oracle statistics show zero wait times.

I guess my real question is, why have 96mb of cache on the controllers if you are waiting until everything is written to disk anyway?

Thanks in advance for any comments.

Tim

Re: Disk Immediate Reporting

Pete Randall — Thu, 06 Jun 2002 15:06:00 GMT

Tim,

From Sam's help on default_disk_ir:

With Immediate Reporting ON, disk drives that have data caches
return from a write() system call when the data is cached, rather
than returning after the data is written on the media.

To me it sounds dangerous to turn it on, especially in a database application environment.

Pete

Re: Disk Immediate Reporting

Martin Johnson — Thu, 06 Jun 2002 15:17:38 GMT

You are not experiencing performance problems, why would you want to change things? "If it ain't broke, don't fix it!"

It appears to me that you would be compromizing data integrity for speed. Speed which you don't really need right now. Don't do it!

Marty

Re: Disk Immediate Reporting

Jeff Schussele — Thu, 06 Jun 2002 15:32:51 GMT

Hi Tim,

Straight from the man pages

\Quote

Cached data can be lost if a device power failure or reset occurs before the device writes the cached data to media. Because of these risks, the recommended value for this parameter on servers is Immediate Reporting disabled (0).

When Should the Tunable Be Turned On?
When a third party application vendor recommends it. For normal use, HP strongly recommends you don't enable this tunable.

What Are the Side Effects of Turning the Tunable On?
Since enabling this tunable circumvents the protections provided by LVM or RAID, there is a strong risk of filesystem corruption and data loss in case of a device power failure or reset.

When Should the Tunable Be Turned Off?
HP recommends you always disable this tunable. This is especially true if you don't want to take the risk of filesystem corruption and data loss in case of a device power failure or reset.

What Are the Side Effects of Turning the Tunable Off?
This might reduce the performance of disk write (not read) access while eliminating the risk of filesystem corruption and data loss in case of a device power failure or reset.

\EndQuote

Sure sounds straight-forward to me!

I'd say if mgmnt wants it turned on, get them to sign off on it. No sign-off - No turn on.

Rgds,
Jeff

Re: Disk Immediate Reporting

harry d brown jr — Thu, 06 Jun 2002 15:37:01 GMT

KEEP IT AT ZERO!!!!!!!

default_disk_ir

The (ir = immediate report)

is explicitly set to 1 on HP workstations. It is 0 by default on HP servers. When it is set to 1 a disk write is considered complete when it is transferred into the cache memory (RAM) in the hard-disk drive. This improves filesystem performance significantly at the expense of a small decrease in data integrity on power failure. When this parameter is "1" it also causes HFS filesystems to be mounted, by default, with the "behind" option active. (see: man 1m mount_hfs) In general, keep set to 1 on workstations.

http://www.comet.ucar.edu/strc/unix/textfiles/SystemTune.txt

And then tell your PSEUDO-expert CON-sulk-Ants to get a life and maybe another job! Maybe they should take a KomPewter class?

live free or die
harry

Re: Disk Immediate Reporting

S.K. Chan — Thu, 06 Jun 2002 15:37:26 GMT

The 12H controllers are not just there for cache, they are the CPUs behind keeping the AutoRaid running efficiently, figuring out the appropriate RAID configuration, it's internal bus IO, etc, etc. The default_disk_ir parameter (if enabled) does seems like it would improve performance because from what I understand it does not wait till the data is physically written to the disk, it assumes the disk controller knows how to handle that part, hence the system calls cycle is much faster. Having said that I have not got around enabling it, simply because ..
1- For now the IO performance is good (no complaints)
2- I want the to make sure anyway the data is pysically written.

Re: Disk Immediate Reporting

Pete Randall — Thu, 06 Jun 2002 15:48:19 GMT

Geez, Harry - I wish you wouldn't be so wishy-washy with your advice.

;^)

Re: Disk Immediate Reporting

harry d brown jr — Thu, 06 Jun 2002 15:59:00 GMT

Pete,

I had to go though an audit yesterday of a huge project with E&Y, one where I designed the technical specs, and it ended up being a waste of my time because they didn't have a question that hadn't already been answered.

I could have done that and saved us money!

I probably could have said that we were going to implement the new java flavored jumping beans, and they would have said, oh yeah that sounds good :-))

live free or die
harry

Re: Disk Immediate Reporting

Pete Randall — Thu, 06 Jun 2002 16:02:02 GMT

Harry,

I think you and I are on exactly the same wave length when it comes to auditors - they're a bunch of time wasting fools that should stick to counting their beans and leave us alone.

Pete

Re: Disk Immediate Reporting

Steve Lewis — Thu, 06 Jun 2002 16:17:23 GMT

Tim,
The best way to speed up i/o to an Autoraid 12H is to replace some of the disks (or plug into empty spaces) some new drives which spin at 10k rpm. They REALLY make a difference. I did this to one of my customers and their data capture job suddenly took half the time it used to. One word of warning with this is that you have to replace the disks one at a time, allowing the rebuild on each to complete before you replace the next one.
Also, if the new disks are bigger than the previous disks then make sure that you plug in at least 3 of them (one for hot spare, 2 for RAID). With a full array (ie no free LUN space) I was given no extra space and no more performance until I plugged the 3rd one in, then it flew along at the speed of jbods.

Keep your immediate reporting parameter at zero. If you are using a database then the i/o is probably going to be asyncronous anyway, so changing this parameter will make no difference. If you are just talking about filesystem i/o then use fs_async=1 and buy a UPS.

Re: Disk Immediate Reporting

Tim Medford — Thu, 06 Jun 2002 16:30:26 GMT

Thanks everyone for your opinions.

Steve - Our AutoRAID is fully loaded with 36 gb 10k drives. I have less than 50% of it allocated so I know that everything is running Raid 1/0. The weekly stats that I run on the array relect this since I can see that nothing ever get migrated to Raid5.

I think these consultants were just following some 10yr old boilerplate recommendations. They also made a bunch of Oracle tuning recommedations which were totally bogus.

The default_disk_ir setting is bugging me though. The AutoRAID has battery backups, plus the entire room is on a UPS system. Seems like several redundant systems would all need to fail simultaneously for this to corrupt a file system.

But just to be on the safe side I'll leave it at zero. I have no statistics which indicate it should be set otherwise.

Thanks again,
Tim

Re: Disk Immediate Reporting

Jeff Schussele — Thu, 06 Jun 2002 17:12:03 GMT

Hi (again) Tim,

Yes you may never lose power on the array....BUT....can you guarantee that the SCSI bus won't lbolt, or reset, so to speak?
You want to guarantee that a chip won't fail, a resistor won't fry, a cable connector won't oxidize, hell the roof won't leak?
If it's production data, it's PURE $.
Now who wants to quibble over a few measly microseconds...that seems like right now, you don't even need.
That cache is fairly large, you could miss quite a few transactions if something that *never will* happen - does.

My 2 cents,
Jeff

Re: Disk Immediate Reporting

A. Clay Stephenson — Thu, 06 Jun 2002 17:14:42 GMT

Hi Tim:

You should have asked "Have you measured this?".

By a remarkable coincidence my Sandbox server (an old K260/1 with a 12H attached) is running HP-UX 11.11.

I put together this test:

timex dd if=/dev/zero of=/u01/tempdata bs=64k count=10000

Here are the averaged results over 5 trials for each setting:

default_disk_ir real user sys
0 31.35 .10 9.93
1 31.50 .11 9.57

Note that the total time actually increased slightly with default_disk_ir set to 1 although
the system time decreased by about 3.6%.

I also used arraymgr to set the apparent read and write cache state to on and that made no significant change either.

Re: Disk Immediate Reporting

Martin Johnson — Thu, 06 Jun 2002 18:11:50 GMT

In My Humble Opinion (IMHO),

Always take a consultant's advice with a grain of salt.

We had a group of consultants who recommended that we upgrade from ITO v4 to VPO V6 without the intermediate upgrade to v5 (HP's recommendation). Since they seemed to know what they were talking about we had them do the upgrade. It has been almost a year and we are still trying to straighten things out!

Marty