topic Re: SAN IO problems in HPE EVA Storage

SAN IO problems

Schroders — Thu, 01 Jul 2010 08:03:44 GMT

We recently migrated from Win2k3 SQL2005 to win2k8R2 SQL2008R2, but are now experiencing IO queues to our EVA4400 SAN. The hardware for both servers are the same BL460C G6.

We are using MPIO v4.01.01 on the Win2k8R2 server and MPIO v3.x on the Win2k3 server. The Win2k3 server runs queries in seconds, but the Win2k8R2 server runs them in minutes - any ideas?

The load balance policy is set to DLQD for 2008 and SQST for 2003.

Re: SAN IO problems

Johan Guldmyr — Thu, 01 Jul 2010 08:11:33 GMT

Hi, have you looked at the queue depths/targets for the FC HBA? I believe the default ones are different between 2003 and 2008. Is the block size different as well on the partitions?

Re: SAN IO problems

Schroders — Thu, 01 Jul 2010 08:19:51 GMT

Where can I find the setting please?

Re: SAN IO problems

Johan Guldmyr — Thu, 01 Jul 2010 08:45:22 GMT

I am not sure actually, I think this differs depending on what FC HBA you are using.

Qlogic has sansurfer and Emulex has hbanywhere for configuring/diagnostics on the FC HBAs.

Blocksize etc can be found with the diskpart tool from windows.

See http://support.microsoft.com/kb/300415

Re: SAN IO problems

Schroders — Thu, 01 Jul 2010 09:55:54 GMT

Found it, it was called hbacmd and is a cmdline util for 2008. I saw some articles recommend 64 or 128 QueueDepth values for SQL, but this hasn't made a difference to the performance - though the figures look better in PerfMon.

I need to check the block sizes next.

Re: SAN IO problems

Schroders — Thu, 01 Jul 2010 10:07:20 GMT

Using wmic partition get blocksize, I can see that both servers' disks are set to 512.

Re: SAN IO problems

Johan Guldmyr — Thu, 01 Jul 2010 11:33:37 GMT

Are your vdisks/luns on the EVA balancedb etween the controllers (you can check owning controller on the vdisk in CV)? If they are all owned by one.. maybe that's the reason?

Unsure where to look further, you can use the evaperf utility to gather stats/metrics from the EVA.

Re: SAN IO problems

Schroders — Thu, 01 Jul 2010 11:39:39 GMT

They appear to be evenly distrubuted.

Some HP engineers rand the evaperf command and collected the data, but it wasn't conclusive as to why this is happening.

Re: SAN IO problems

Uwe Zessin — Thu, 01 Jul 2010 11:43:38 GMT

That 512 (bytes) is most likely the SCSI blocksize as presented (and not changeable) by the EVA. I would guess that Johan meant the NTFS cluster size:
>fsutil fsinfo ntfsinfo C:

Also check the operating system type of the Windows 2008 servers - there is a separate entry for W2008.

Re: SAN IO problems

Schroders — Thu, 01 Jul 2010 11:56:36 GMT

NTFS Volume Serial Number : 0x12528c60528c4a85
Version : 3.1
Number Sectors : 0x000000007fffe7ff
Total Clusters : 0x000000000ffffcff
Free Clusters : 0x00000000057a5ebf
Total Reserved : 0x0000000000000000
Bytes Per Sector : 512
Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 1024
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 0x0000000004b40000
Mft Start Lcn : 0x00000000000c0000
Mft2 Start Lcn : 0x0000000000000002
Mft Zone Start : 0x00000000000c4b40
Mft Zone End : 0x00000000000cc840
RM Identifier: 614E60F6-2542-11DF-9D9E-D8D38565FC5C

The host is defined as a 2008 server on the EVA if that's what you mean.

Re: SAN IO problems

Uwe Zessin — Thu, 01 Jul 2010 12:03:08 GMT

> Bytes Per Sector : 512

The SCSI block size.

> Bytes Per Cluster : 4096

The file system "block size". I am not an expert in MS SQL, but I've read somewhere that one should use 64KB clusters.
But I don't beleive that this is the problem.

>The host is defined as a 2008 server

Correct.

Are the database really set up the same way and do the same queries? A missing index, for example, can really ruin the day.