ProLiant Servers (ML,DL,SL)
1748130 Members
3612 Online
108758 Solutions
New Discussion юеВ

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

 
MisoVranes
Occasional Contributor

Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

Hi friends.

 

I am experiencing serious problems with HP Proliant Servers DL380 G7.

These servers reboot or crash much more then any proliant I have ever seen before. After reboot I often see this error:

"Fatal PCI Express Device Error PCI ? B00/D00/F00 " Also, there are false power supply #1 errors. On one of these servers couple of times I also experienced uncorrectable ecc memory errors which were not solved by replacing memory and happen again and again .

 

The Integrated Management log :
Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 0, Function 0, Error status 0x00000000)

 

I have upgraded all firmwares to current version (today is 26.08.2011), except smart array controller. Fixes and release notes for BIOS and ILO3 firmwares are pointing out that  behaviour regarding these problems is improved, but for my affected servers it is still NO GO.

 

I am aware of advisory from last year:

 

SUPPORT COMMUNICATION - CUSTOMER ADVISORY

Document ID: c02178619

Version: 1

Advisory: HP Integrated Lights-Out 2 (iLO 2) - "Fatal PCI Express Device Error" Message May be Displayed After an AC Power Cycle When Memory Size Is Reduced to Below 4 GB

https://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/kb/docDisplay/?javax.portlet.tpst=ba847bafb2a2d782fcbb0710b053ce01&javax.portlet.prp_ba847bafb2a2d782fcbb0710b053ce01=wsrp-navigationalState%3DdocId%25253Demr_na-c02178619%25257CdocLocale%25253Den&javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken

 

and I am aware of the post

 

http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-SL/DL380G7-win2k8r2-w-sp1-fatal-pci-express-device-error/m-p/4769378#M110670

 

My servers have two aditional NC364T network cards in PCIe slots 2 and 3.  Servers have two Xeon 5640 processors.

Servers are running VMWare ESXi 4.1 U1.

 

Does anyone experience similar problems or have a solution?

 

Best regards to all.

 

 

11 REPLIES 11
Sirajul haque
Frequent Advisor

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

I rememember this error with P410 controllers and I suppose you use the same..

Uncorrectable PCI Express Error (Embedded device, Bus 0, Device 7, Function 0, Error status 0x00004000)

But in your case, the error message is not exactly same.. However, you may try below said action plan:

1. Update the firmware of Storage controller, i think you have P410 controller, download Version:     5.06 (C) (29 Jul 2011) from HP.com

I wrote something about it here:

http://www.tricksguide.com/blue-screen-error-hardware-malfunction-pci-express-error-hp-proliant-server.html

Also try:

To disable тАЬNo-Execute Memory ProtectionтАЭ on your server, enter in to Server BIOS (RBSU) by pressing the F9 key at Server Boot -> under System options -> Select Processor options тАУ> Now select No Execute Memory Protection тАУ> Set to Disabled


Note: The erros message has some difference "Device 7" and "Device 0", so this is just a recommendation from my side.. :),  But I think its worth a try :)

Though I work for HP, I do not represent HP here :)
AkiraX
Advisor

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

I see the same issues with my bl460c G7's even with the latest 5/5/2011 BIOS......

 

Sometimes, like 5 minutes ago, when you reboot the server it just comes up with a black screen without the PCI Express Error. A coldboot via Ilo resolves it until next time.

AkiraX
Advisor

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

also, because its a hyper-v server I cant disable the No-Execute memory option.

 

At the advice of HP, I toggle the dip switch 6(I think), power-on, re-toggle and still no luck as it eventually comes back.

MisoVranes
Occasional Contributor

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

I have tried to analyze situation using bootable linux but I can not find out whait PCIe device is under incestigation. It must be ILO3. I am not sure, I will try to upgrade the smart array, and Iwill conduct offline Insight diagnostics, but stability is ridicilous.

DL380 series were my favourites before. Hey, HP are we becoming beta testers?

MisoVranes
Occasional Contributor

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

I will try.

 

Thanks.

MisoVranes
Occasional Contributor

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

All updates installed and still no go. Servers crash very often. I set at the end "NO c-states" in BIOS and set power management to maximum performance. I am trying to avoid that stupid mbedded PIe device error by setting PCIe compliance to PCIe Gen #1. I will inform what happened.

Sirajul haque
Frequent Advisor

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

Hows it going now? Did that work?

Though I work for HP, I do not represent HP here :)

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

i solved this problem..
dame error messages. :)
my case is... sata cable problem.. :(
sata cable replace..
Daniel Lepak
New Member

Re: Proliant DL380 G7 Fatal PCI Express Device Error PCI ? B00/D00/F00

Based on in the field experience always check the FW (Firmware) first. Unless you are using a special BIOS setup. Otherwise refer to the latest & greatest FW to resolve most issues w/ ProLiant Servers G7 and above using SPP2013020B...

 

http://h18004.www1.hp.com/products/servers/service_packs/en/index.html

 

 

and utilize the HP Support Center on the HP Support Site; as noted below...

 

http://h20566.www2.hp.com/portal/site/hpsc/template.PAGE/public/psi/mostViewedDisplay/?javax.portlet.begCacheTok=com.vignette.cachetoken&javax.portlet.endCacheTok=com.vignette.cachetoken&javax.portlet.prp_efb5c0793523e51970c8fa22b053ce01=wsrp-navigationalState%3DdocId%253Demr_na-c03261617-2%257CdocLocale%253Den_US&javax.portlet.tpst=efb5c0793523e51970c8fa22b053ce01&sp4ts.oid=4091412&ac.admitted=13...

 

 

Hope this resolves your issue or anyone else experiencing similar issues w/ there G7 or above.

 

Please note: ProLiant G6 & below Servers may or may not work using SPP2013020B (Service Pack for ProLiants 2013) they may have to utilize the old school PSP's & SmartStart FW DVD/CD's.