Windows 2003 server blue screens with MSA-1500

[Pete Cousins]

I'm a SQL DBA, not a SAN expert and so would value any ideas on investigating our current problem. We have a DL585 G2 with an Emulex FC2243 HBA connected to an MSA-1500. The server is running SQL Server 2005 and under extreme load it is blue-screening. No memory dump files are being created, but the event log is indicating a problem with communicating with the SAN. One event is always:
Application popup: Windows - EXCEPTION : Possible deadlock condition.
With a variety of other events - different for each blue screen e.g.
Elxstor event ....The following information is part of the event: \Device\RaidPort0.
Additionally, just before the blue screen, SQL Server complains that there are I/O's taking longer than 15 seconds.
The drivers and firmware for the Emulex FC2243 HBA are fairly up to date (storport 5.2.10.901, Firmware 2.72.A2). MSA controller firmware 5.10.
It's almost as if the SAN is disappearing, or no longer talking to the HBA. The server has also blue screened when it was under no load!!
Anyone got any ideas of where I should start looking?
Regards,
Pete Cousins
SQL DBA
RIAS

[David Tapper_1]

Hi Pete,

I'll think you have problem with the storport driver according to this document: http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?lang=en&cc=us&objectID=c00715130&jumpid=reg_R1002_USEN

I'll suggest you to download the latest storport-frix from Microsoft: http://support.microsoft.com/kb/932755/en-us

But be aware, If you have an old version of Smart Array Controller driver you can still be suffering from BSOD: http://www13.itrc.hp.com/service/cki/docDisplay.do?docId=emr_na-c01068337-58&docLocale=en&admit=109447627+1244182629499+28353475

I also recommend you to upgrade your controller firmware, smart array controller driver and latest supported driver for your HBA. Also make sure you have the latest supported MPIO installed.

Regards,
David Tapper

[Pete Cousins]

David,
Many thanks for your reply. I have already upgrade the storport driver to 5.2.3790.4021 and the HPCISSS2.SYS driver to 6.14.0.32 (which I believe is used by our P400 Smart Array Controller - that was causing lots of blue screens, but not anymore!). We've also upgraded the firmware and drivers for the Emulex FC2243 controller. We're not using MPIO (yet!), and the next plan is too upgrade the MSA Controller firmware (from 5.10 to 5.30). We may even go active/active on the MSA, as it's perfoming about 4 times slower than the local disk on our development server.
Thanks again - and I'll keep you posted whether the firmware patch fixes the problem
Pete