HP LH P4500 strange latency

[Chrisb2k]

Hi

I have an issue where all vms periodically and seemingly independently lock up, denying all access. Even pinging the vm shows these symptoms by taking many seconds to complete a number of pings over about a minute, then it reverts to normal and pings as you would expect. The behaviour on the VM is as if it has hung, then eventually returns

I have been onto HP support who suggest that the number of iscsi sessions open is extremely high, 7258 at the last count, where my configuration can support 4000 at most.

So, my first question is how do we free up iscsi sessions? Rebooting ESX servers makes no difference or increases the number. Warm rebooting the storage nodes made no difference(as I did when installing a patch for memory issues last night which were thought to be a cause - it wasn't). HP suggested disconnecting the volumes on the ESX hosts then rebooting but this would take many days and the customer is already screaming ... is there another way to free iscsi sessions?

Secondly, is the number of iscsi sessions likely to cause the problem we are seeing? Has anyone seen this before?

Any advice greatly appreciated

Thanks in advance

Chris

[Bryan McMullan]

I've seen this behavior in my environment when testing failovers with ESX. We actually saw it more in our pilot tests when the implementation engineer didn't enable load balancing on the VMWare Hosts in the CMC.

You have 7258 iSCSI sessions? I think something must be wrong in that count. You can see the number of sessions by clicking the cluster in CMC and selecting the iSCSI Session tab. It just seems grossly overstated unless your install dwarfs mine (about 60 NSM's).

Have you fully patched your units? My guess is the patch you installed was 10065. You may want to look at the RAID Firmware patches (10055/10056) or if you are using SATA drives, the backplane patch (10047).

The other issue could be your network is not setup correctly for use with ESX and the SAN.

[Chrisb2k]

Interesting thanks very much Bryan. The issue was actually traced back to a VMWare problem which a series of patches seems to have resolved. Although the iscsi sessions are extremely high, it doesn't seem to affect the environment adversely ... YET.

My env certainly doesn't dwarf we have 15 NSMs :)

I will look at getting these patches installed

Thanks again