Auto RAID 12h: Performance

[Rao Uppuluri]

I have a production server is a L2000 with 2 CPUs and an AutoRAID 12h for storage. We are using this as an Oracle db server. My test env is a A500 with bunch of disks. I regularly update my test env with export and imports. Lately I am finding that the query response times are much faster on my test machine than the Prod box. All my schemas are analyzed. I always see a disk bottleneck(sar -d) on my prod box. Disk util close to or equal to 100%. Is it possible 12h is slower than just bunch of disks? I know I have provided very little information here, but can provide more if you have any questions. Thanks!

[Eugeny Brychkov]

First of all, AutoRAID 12h is HVD device, capable running 40MB/s max (bus limitation). Second, if SCSI bus is having problems (not terminated/overterminated, bad device on the bus, bad cable/terminator) this can cause failures on SCSI bus and thus additional delays. And of course, if you use fibre channel FC10 as a bunch of disks, 12h will not look good.
Check if 12h is healthy - simply from its control panel. Check if bus is properly terminated and cables are ok, not exceeding its lenghts (max 25m as whole bus). There're no devices on the bus which can load bus preventing 12h of winning SCSI bus arbitration
Eugeny

[Rao Uppuluri]

I checked my 12h and found no apparent errors. arraydsp -i shows everything as ok. My test env doesn't have any fibre channel connections. Also, I am not sure how to check for bad termination or over termination etc. and I don't think there any other devices on the bus. HP engineers set up the machine here so I am hoping it is setup correctly. Are there any setup parameters on 12h I sould look for? Thanks for the response.

[Eugeny Brychkov]

Attach to your next reply zipped (attach, not post!):
- 'arraydsp -a'
- 'ioscan -fn'
- 'logprint'
Bus termination rule is simple: bus should be terminated at both ends with correct type of terminators: in your case HVD ones
Eugeny

[Rao Uppuluri]

Eugeny ,
Here are the files you requsted. Thank you

[Eugeny Brychkov]

Rao,
file is damaged. Although ITRC states that max file size is 1MB, in reality it's 256K. File attached is 717MB in size, and I guess logprint in a large file there. So please use '-s' (start date) option with logprint (starting, let's say, from Feb 1) to cut earlier AutoRAID events.
You have to manage to get file to 256K limit
Eugeny

[Vincent Farrugia]

Hello,

If an AutoRAID is not configured well, then, yes, it can be slower than JBOD, it can be slower than anything in this world.

First, how much data is in your AutoRAID? If it is approching 90% of the total disk space, then this is your problem. AutoRAID works by putting the least-used data in RAID 5, the most-used data in RAID 0/1. If your data is close to 90% of disk space, then your data will be getting from RAID 5 to RAID 0/1 all the time, bringing it extremely slow.

Recommendation is to always keep AutoRAID having 50% or 60% of total disk space unallocated, so that all data is in RAID 0/1, which is much faster than RAID 5.

HTH,
Vince

[Rao Uppuluri]

Hi All,
please take look at the attached files. As suggested I am including logprint output from Feb 1st. The arraydsp output shd give you info about the usage of the 12h. Thanks

[Eugeny Brychkov]

AutoRAID is ok. Only note that rebuild priority is high (perfomance impact during rebuild, I found no rebuilds in log) and disk in slot A5 sometimes timeouts (approx once per 2 weeks, last time Apr 1st at 9:47). LUN allocation also good enough. So I do not see any config problems.
Please note that latest firmware for 12h is HP62. I do not remember its improvements, but it was recommended to roll to HP62
Eugeny

[Rao Uppuluri]

Eugeny
Thanks for your help. So now going back to my problem, I guess I have to totally concentrate on Oracle, sql, exec plans etc. for the slow query executions on my prod machine.
I have one question. How did you exactly figured that the disk in slot A5 has timeouts? As a matter fact, I do have a open call to HP about this timeout message i got in my syslog.log file. (Funny, they never asked me about the files/output(logprint, arraydsp) you asked!) I know how to figure the physical path. but does it actually point to a disk on a RAID? I will also look into upgrading the firmware. Thanks again for your help. - Rao