SQL I/O issues on EVA 8000 SAN

[Peter Davies_5]

Our SQL Server has its mbd/ldf files located on the SAN. Over the last while we having been getting 2 main errors:
->SQL Server has encountered 5329 occurrence(s) of IO requests taking longer than 15 seconds to complete on file [K:\MSSQL$INSTHCM\Data\tempdb.mdf] in database [tempdb] (2). The OS file handle is 0x00000698. The offset of the latest long IO is: 0x0000009bd90000
->BobMgr::GetBuf: Bob write not complete after 60 seconds

I've checked in with Microsoft and they have mentioned to check in with storage as this may be a disk subsystem issue. This issue happens between 1-1:30am every morning.

I'm not a storage specialist but are there common diagnostic disk tools that can help pin point this issue. Are there disk maintenence tasks recommended by HP? Anyone else had this type or similar issue here?

Appreciate insight.

[Rob Leadbeater]

Hi Peter,

If this is happening at a specific time overnight, I would guess that there are some backup jobs scheduled at that time that are making the EVA busier than usual...

Cheers,

Rob

[Peter Davies_5]

Hi Rob:

Actually there are no backups (internal or Veritas) or any SQL jobs running between 12:30am->2:00am.

Peter

[IBaltay]

Hi,
the good help is to run the evaperf util on the e.g. FP ports and also the VDISK level.
Also check the LB (ALB) on the host side (SQST is preferred).

[Clint Placette_2]

I would like to add what IBaltay mention.

You should run Evaperf for a week. You run Evaperf and set the length of time by using the duration command.

When you run evaperf use the all or MOF in conjuntion with the CSV command. By using these command you will log the data to a file or files in a csv format. You then can graph the performance based on the counters.

Given that you know when the issue is occuring I would use the Evaperf HPS command to identify if your problem lies within a particular server which is what I suspect is the problem. Another issue which seems be igored is the I/O load balance between EVA controllers. A quick way to find out if you are balanced is to run the Evaperf VD command and look at the controller assignment per server. If you have never run Evaperf you must run the FN (friendly name) command so you indentify the server name vs WWNN.

[Peter Davies_5]

Thnxs for the information. Our Storage person is currently logging the EVA and hope to see what comes about.