c7000 bl680c hang

[Mel Nugent]

Two c7000 blade chassis each with 4x680c blades running ESX 3.5
All 6 power supplies replaced in both chassis on Tues. All 4 ESX servers in one of the chassis hung on Wed. Coincidence?
I see this on the console on all 4 esx servers in the chassis that hasn't had a problem "bmc returned incorrect response, expected netfn 5 cmd 27, got netfn 5 cmd 35" also in logs on other that did reboot. Starting to investigate if HP or VMware problem.

Any one had any problem after replacing power suppliies?

[Mel Nugent]

Sorry posted into ML/DL by mistake, maybe mods can move to blade system. thanks

[Mel Nugent]

Through HP and VMware support we have found out that cisco switch (MDS9124) failed due to cisco field notice cscsu80534. Although in our case it did not reboot but stayed down. Prior to this the ESX hosts had dropped the paths to the SAN via the second data switch for some reason as yet unknown. So when this switch went down the ESX hosts had no access to storage. This caused all VMware guest to fail until the ESX hosts were rebooted.