LSI_SAS Errors. Related to P4000 SAN?

[Peter J West]

Despite spreading out VM's over mutliple OS VMFS volumes we are still seeing a range of event's in the event logs.  These range from the LSI_SAS errors mentioned above to others pertaining to MPIO problems and others.

I've now opened a case with HP Support as I don't feel we're a particularly large environment and we shouldn't be seeing these issues.

I'll report back with more news on what we're doing just in case it helps anyone else solve their issues.

Pete

Edit: Just to clarify.  The new configuration is 5 x 250GB VMFS volumes and on each of these we have no more than 3 virtual servers.  Typically we allocate 60GB for the OS of each VM.

[Peter J West]

Just a further update.

We had a live support session yesterday and as a result of this it was recommended that we turn on flow control on the storage nodes and switch ports.

The guy doing the support session mentioned that our IOP load on the SAN wasn't significantly high but felt that the enabling of flow control might be a good starting point.

We've done that today and will monitor over the coming days.  I'm not that hopeful that it'll fix the issue but we'll see.

One thing I have noticed was that when looking at the storage latency performance graph there seems to be a lot of high values reported by HBA33 which is a physical iSCSI port on the Broadcom card.  This struck me as a little strange as i'd have expected the latent HBA to be HBA37 which is the Softwared-based iSCSI Initiator.  Maybe this points to a problem with my configuration?

[Aart Kenens · Accepted solution]

Please use the software iscsi initiator.

Performance is better and it has proven itself.

Some users at vmware forums were complaining that iscsi luns got disconnected under heavy load. 

After they changed to swiscsi the problems were gone.

I too use the swiscsi instead of the broadcom hw iscsi.

regards,

[Peter J West]

Thanks Aaart, i'll give that a try.

The max read and write latency figures for vmhba33 (which is the Broadcom) are 109,475 and 49,500 respectively.  These figures dwarf those of the iSCSI Software Initiatoy (vmhba37) which has values of 91 and 89.

Fingers crossed this change will make some impact on the issue.

[Peter J West]

Hi again,

It's a little early to say for sure yet but at the moment the changing of the iSCSI configuration appears to have fixed the problem.

I'm now seeing max latency values of 50ms for both read and write operations - i'm not sure how this fits with recommendations but it does mean that all fo the events in the event logs on the servers have vanished.

I'm going to continue monitoring for now, but it looks very much like we've got the bottom of the issue.

Thanks

Pete

[Aart Kenens]

I am glad you've sorted out the issue.

greetings,

Aart