Hi All, Has anyone had the following scenario on a Disk Autoraid on HP9000/K580.
HPUX 11.00 Been running happily for over a year and then get the "Disk Balance in progress" logged fairly regularly (1-2 times a day ), completes within a a couple of minutes. Autoraid stats show data being relocated from Raid 5 to Raid 1. AT the moment not affecting peformance but could get worse!
LUN sizes and number has not been changed. System running Oracle Database and database size is not changing, but contents changing frequently. 12 * 17GB disks
Total physical = 208397 MB * Allocated to LUNs = 124928 MB * Used as Active Hot spare = 17366 MB * Used by non-included disks = 0 MB * Used for Redundancy = 32313 MB * Unallocated (avail for LUNs) = 33790 MB *
Question is why would the data Balancing startup out of the blue with no changes in the disk configuration. Oracle database has its own LV's and so does the data within the LV's also get relocated from Raid 5 to Raid 1 as well?
I think this actually means that your LUNs are getting full.
AutoRAID puts the least-used data in RAID 5, and the most-sed data in RAID 0+1. When the AutoRAID is becoming full, there is more need to redistribute data between RAID 0+1 and RAID 5, because space is becoming tight and so it needs to put more into RAID 5 (which uses less space) then RAID 0+1. That is why you are having those message more often.
You are right: if the data keeps increasing, there might be a massive performance loss because your AutoRAID will redistribute from RAID 0+1 to RAID 5 and back all the time.
In order to solve your problem, increase space in your AutoRAID and leave it unallocated. Currently, you only have 33Gb out of 270Gb to play with. For optimal performance, this needs to be half or at least 40% of your total space.
Vince is correct - there's a difference between LUNs allocated to the server/database, and what the server actually uses of the LUN.
When databases change data in the database, they often use whatever space is available and convienient; which means that it may use previously unused areas of the disks to insert records, even though there's space from record deletions.
This can cause the AutoRAID to rebalance the data load on the drives.
So, there are two things you can do to ensure the best performance - add more drives so there is more free space in the array, or reorganize your database (which will recover that space from deleted records.)
One more thing to check. Run 'logprn' for this disk array. Check its output. Is will contain events happened to array. If you'll see anything wrong there (especially at the time array balances), then post it here. You can zip whole output and attach to your next reply Eugeny
I agree with Eugeny, review the event logs. Balance is not an AutoRAID policy. It is used to re-distribute the data to a newly added disk. The fact that it is triggered may indicate you are having a problem with a disk. Also, review the armperf OPAQUE logs. These contain more information about the execution of the RAID migration activity.
You should also upgrade to HP18. HP18 has number performance and availability enhancements.
Roger meant HP18 firmware for VA7x00. As soon as VA has capability working in autoraid mode same rules apply to VA as well. Autoraid 12 latest firmware is HP27; Autoraid 12h latest firmware is HP62 Eugeny
If there'll be anything wrong you'll see it and understand immediately that someting's wrong. Relocations are usual activity pushing old data to RAID5 and pulling requested data to RAID1 space. If you configured active hotspare or have free unallocated space then it's used for data chaching. All LUNs are spread across all drives, and autoraid managing data layout basing on available free LUN space, activity, cache settings etc. I do not think LUN reserved space can be used for data from another LUN. Could you please attach whole 'arraydsp -a' output to your next reply? Eugeny
I can not understand why my latest reply was inserted before Suri's reply... It should be after? Time is 16:15 now. Looks like ITRC server timing problem! Eugeny
All replies very useful for diagnosing the problem. One other question before allocating points. Besides the database, does the unused space in one LUN get used in the DATA RAID5/1 manipluation, for data from other LUN's.
Eugeny,
I have run logprint on the system, Compared with other systems, see nothing wrong and do get ONLY the "Raid 5 to Raid 1 relocation" figure a few times a day (normally 0). But does not seem to correspond to the time stamp in the syslog.log file. Must say that occassionally the figure is in the hundreds but that is normally at off peak times. thanks
Some notes on it: Autorebuild is disabled. If disk will fail array will not rebuild. If one more disk will fail data will be lost. So please better turn on autorebuild and set rebuild priority to low. BTW, I remember 12h can hold 36GB disks, so you have ability to increase your storage space. But maybe better (and cheaper) to buy VA71x0 disk array? Eugeny
In /opt/hparray/bin are some tools, specifically logprint. Syntax is
logprint -t perf logprint -t ctrlr
This will give you performance data and controller errors. The controller error include disk errors so you will know if a disk is going south. The performance data will have to be analyzed by HP in all probability.
Yes, the unallocated space is used for RAID 0+1/RAID 5 allocation. In fact, that is why it is recommended to be at least 50% of your whole space. That way, all data will be in RAID 0+1, which means performance gain by:
1. RAID 0+1 is much more performant than RAID 5. 2. No balancing is performed.
Anothr thing. If you want to increase space, increase disks in PAIRS, not 1. This applies if the disks you are entering are larger than your current disks. If you enter just 1 36Gb disk, and you have 9Gb disks, for example, the AuotRAID will only use 9Gb from the 36Gb disk. But if you enter 2 36Gb disks, you will use all the space provided by the disks.
Also, if you are entering 36Gb disks, the fans must be high-spec. If you don't have the fan logo on your fan on a "bubble", but instead on a flat surface, then you need to change the fans in order to install 36Gb disks. This will provide the cooling necessary for these kind of disks to operate.
