EVA4400 Performance problem (FATA)

[JorisK]

Hi,

We recently added 24 FATA disks to our EVA4400 system. The system has some 300GB/450GB and now 24 FATA disks on it.

Diskgroup one contains all 300GB disks, diskgroup two contains all 450GB disks and diskgroup three contains all 1TB disks.

We have our production environment running on the 300/450GB disks, and we use the 1TB diskgroup as 'cheap' storage to store old backups, ISO's etc.

Now what happens: When i start running a few copies at the same time to the FATA disks, the production disks get slow. I see no heavy CPU load or bandwith usage whatsoever.

What can cause the all the diskgroups to start lagging when we only perform some serious I/O on one disk group? Could this be the cache? Is it wise to turn off Read/Write caching for the vdisks in the FATA diskgroup?

Best Regards,

J.

P.S. This thread has been moved from General to Storage Area Networks (SAN) (Enterprise). - Hp Forum Moderator

[Johan Guldmyr]

Hi, where do you not see heavy CPU load or bw usage? On the EVA?
Are you using evaperf?
Do you have any BC features activated?

[JorisK]

Hi,

What exactly do you mean with BC enabled?

I do not see any high CPU loads on the EVA system, it avarages around 15-20%.

[Johan Guldmyr]

I meant business copy / snapshots, clones, etc. Maybe if you're using those and they have the copy stored on the SAS disk group that could be the issue.

Is the FATA disk group doing any other things like reconstruction? Maybe that's the cause for the low performance.

If you collect some evaperf data into .csv and send to HP they have a tool that can visualize the data into graphs.

[JorisK]

Nothing special running, just plain storage, no snapshots whatsoever.

I create a lot of graphs using Cacti and i don't see any heavy spikes, except for read IO during the backup.

But my question: can i just on-the-fly disable read/write-caching on the vdisks?

[Johan Guldmyr]

It says in CV 9.4 user guide it says below, it doesn't mention if it's on offline operation. I can't recall if I've done this while using the storage but I'd say you can do it online.


"The array automatically flushes the write cache before copying data to a snapclone or snapshot.

You can reduce the performance impact of creating a snapshot or snapclone by presetting the write cache on the source disk to write-through mode. The performance benefit is greatest when there are multiple snapshots or snapclones for a single source disk. Be sure to re-enable write caching after the copy is initiated"

[Amar_Joshi]

Are you using RAID6?

EVA has limited cache and FATA drives are extreme slow for random IOs. Multiple writes can severely impact performance. Writing from hosts can overload the cache on EVA HSV controllers and specially if it's coming from mis-configured hosts. Unfortunately EVAPerf will not show you cache utilization and you need to judge it based on host-port queue-depth. In my experience RAID6 is very bad in conjunction with FATA drives. Also you must try these -

1. Windows hosts should be configure correctly wtih ALB enabled.

2. Unix/Linux hosts using FATA drives should have queue-depth of 8.

3. Any ESX3.5 hosts should be configured to use single path only.

All the above can impact improperly load-balanced EVA controllers and will result in slow performnace (especially on FATA disk_groups).