MSA + Oracle RAC + Odd behavior

[Rex McCracken]

I'm reaching out to the HP Support community as I'm tired of trying to deal with this headache internally. 

Environment:

Windows 2003 Server SP2

p-class Blades, BL20p G3 and BL40p

MSA1500 CS, firmware 5.30

 - 2x 4TB SCSI shelves

 - 1x 2TB SCSI shelf

 - 2x 9TB SATA shelf

2/16v SAN switches running 4.4.0c OS

There are 3 BL40p servers with fiber channel cards running Oracle 10g RAC Database.  The issue we have been experiencing for a fairly long time is that of the RAC database servers "losing" the SAN disks provisioned to them and as such, restarting.  I have tried firmware updates, lowering activity on the SAN, shifting backups to other times, looking for tuning settings within Oracle, and at this point I'm pretty well out of ideas save blowing everything away and starting from scratch, though doing so in a pseudo-production system tends to make lots of people jumpy.

My questions to the community:

 - Has anyone else encountered this behavior?

 - Are there any suggestions outside of updating all drivers, firmware, etc. to resolve this problem?

 - Would lowering the LUN count (we're at 27 now due to "scope creep" over 5 years) help reduce this problem?

Also, 2003 was a requirement, so please save the "change your operating system" suggestions.

Any help is greatly appreciated.

Sincerely,