ProLiant Servers (ML,DL,SL)
1748289 Members
3074 Online
108761 Solutions
New Discussion

Re: HP Proliant DL580 Gen8 and GPU's

 
GertDekkers
Occasional Contributor

HP Proliant DL580 Gen8 and GPU's

Dear all,

I'm a PhD student at KU Leuven university.  I have a HP Proliant DL 580 Gen8 server with 3 GPU's at my disposal. Lately we've been experiencing full system shutdowns. The main thing that has changed is that I added an additional GPU. I'm certain that is is related but I don't know the exact cause. 
In the iLO we get the following message at 'Integrated Management Log':
System Power Fault Detected (XR: 10 A2 MID: BF 0F F0 00 00 FF 3F 0B 0B 04 04 0B 0B 0B 0B 04 04 ....)

The server setup:
HP PROLIANT DL580 Gen8 4X I-XEON E7-4850v2 12-CORE 2.30GHZ 512GB DDR3 RAM P830i 2GB RACK RAILS
4x 1500W Power
2x Geforce RTX2080Ti, 1x Geforce GTX1080Ti
CentOS

We think it's related to the GPU causing power issues/heat issues (although measured values are within accepted values).
By removing one GPU and limiting the average power to 220W for each CPU we haven't experienced any outages so far. 
Does someone have any experience with this server along with GPU's? I was also wondering what the 10-pin MOLEX auxiliary power connectors can provide? Now a single MOLEX is used to power one GPU. Also, does it matter which connector I use?

All information is welcome.
Thanks!

Best,
Gert

18 REPLIES 18
KSINGH_2018
HPE Pro

Re: HP Proliant DL580 Gen8 and GPU's

Geforce RTX2080Ti & Geforce GTX1080Ti  are not listed under the compatible list of GPUs for DL580 Gen8 as per the server quickspecs.

https://h20195.www2.hpe.com/v2/getdocument.aspx?docname=c04199329

Kindly coordinate with NVIDIA and check the Hardware Compatibility List of the Vendor to confir if the GPU is supported by DL580 Gen8 Server or not.

If it is identified as supported by as per Vendor Hardware Compatibility List then please log a case from HPE Support Case Manager or call us to log a ticket for the same. For any service the server must be under Valid Active Contract/Warranty/Carepack and we'll require AHS Log for analysis as well


I work for HPE

Accept or Kudo

nikolaandelic
New Member

Re: HP Proliant DL580 Gen8 and GPU's

Hi, I am also wondering the same thing. Did you make it work? It should as the DL580 Gen9 at least supports 5 full size, full hight GPU's. 

Wishgranter042
Advisor

Re: HP Proliant DL580 Gen8 and GPU's

Hi   @KSINGH_2018  

Im get in next few days USED ( Refubrished )   HP Proliant DL580 Gen8 and planning to sue Nvidia RTX 2070 Super ( 2 GPUs now, 2 wil be added later )  
So the question is it is supported on the server or it needa BIOS update or etc ? to work properly ? Older card like GTX 980 wil work in meantime ?  

pdworzynski
New Member

Re: HP Proliant DL580 Gen8 and GPU's

Hi,

I'm on the same boat. We've been using our DL580 Gen8 with Tesla K40 for a while but after increasing amount of RAM past 1TB we run into address limitation of K40 which makes it useless. I'm thinking about getting a couple of RTX 2080Ti's but i'm wondering about what issues may occur. Could you let me know if you run into any?

Thanks

KSINGH_2018
HPE Pro

Re: HP Proliant DL580 Gen8 and GPU's

As per DL580 Gen8 Quickspecs, we have the following Graphics Cards tested & recomended by HPE's Engineering Team (Please refer to "HPE Graphic Options" on Page 31):

1. NVIDIA Quadro K6000 PCI-E Graphics Adapter (730874-B21)
2. NVIDIA Tesla K40C 12GB Computational Accelerator (753960-B21)
3. NVIDIA GRID K2 Reverse Air Flow Dual GPU PCIe Graphics Accelerator (753958-B21)

**HPE ProLiant DL580 Gen8 Quickspecs**
https://h20195.www2.hpe.com/v2/getdocument.aspx?docname=c04199329

Since Nvidia RTX 2070 Super has never been tested with DL580 Gen8 Server hence we can't confirm about the compatibility of the same. Please approach the Card Vendor (NVIDIA) for checking the Hardware Compatibility List for this card. Please do ensure to have the latest System ROM prior to the usage of this card.


I work for HPE

Accept or Kudo

Wishgranter042
Advisor

Re: HP Proliant DL580 Gen8 and GPU's

Hi All 

So afterg a lot of reading and etc. i can confirm thaat GTX or RTX GPUs are locked out from dl580 gen8 servers. 
HP just use PCI-E Vendor IDs from the GPUs and when its not a QUADRO PCI-E IDs,  it just not work or direclty crash system  ( trying to identify card by a sw ) and card have permanent error code 43 - and it cant be solved.  so nice way to lock up using anything except QUADROs what are not so powerfull vs gaming cards in term of COMPUTE performance. 

JBDynamics
Occasional Visitor

Re: HP Proliant DL580 Gen8 and GPU's

The HPE Proliant DL580 G8 is an enterprise product, as such is meant to only be compatible with enterprise grade GPUs. I can personally say that the Tesla M80 works on the DL580 G7. I have 2x Tesla M80's with the IO expander on the true x16 double wide slots with 4x E7-4870 and 512GB RAM. The Tesla P100 works great on the DL580 Gen8, I have 3x Tesla P100's with 4x E7-4890V2 and 768GB RAM. The server runs smoothly in the data room at my office, I had to install 2x large window air conditioners along with central air to keep the room under 24ºC, but I have 9 months uptime right now. I have the machine virtualized with VMware and I use PCIe passthrough to Ubuntu 18.04 VMs for TensorFlow containers with CUDA compute.

You are correct that GTX cards are not compatible, but it is for good reason. There are multiple layers of redundancy and CRC error checking on Tesla and Quadro products. Gaming products do not have these features. In order to keep a stable enterprise server, it must have enterprise grade peripherals. There is no wonder that an HP Enterprise product crashes when you install consumer grade GPUs. Yes, HP locks out the cards by PCIe vendor and product ID, but that is insurance for the warranties they provide. If you burn up your server because of some over clocked, hot running, consumer GPUs, that should be your responsibility, but instead of putting that burden on the owners, they lockout everything they know to not be compatible.

6BQ5
Frequent Advisor

Re: HP Proliant DL580 Gen8 and GPU's

Just discovered this thread due to @JBDynamics reply floating it to the top. Interesting topic!

I'm impressed that people have installed multiple 2070 and better cards in their servers. At full power these cards can consume over 225W and create enough heat to warm a home. Installing three of these cards will easily take up over 650W of power! Add to that 2 x 130W power house CPUs, ~10W per hard drive, and who knows how much the motherboard and memory require ...

650W + 260W + 10W * 8 drives + 75W (??)  motherboard = 1090W

Holy smokes!! (literally) You're going to need HPE's beefiest power supply which I think is 1200W / each, I think? Remember, assume that one supply can fail putting the whole burden on the remaining supply.

Gamers plan their builds around the GPU and CPU and use online tools that add up component power consumption. I'm sure there's got to be one for HPE's servers

Side note : a humble Quadro P4000 card in comparison uses only 100W at full power.

I recenty installed a GeForce 1080 card into my ML350p Gen8 without any hitches or glitches. I also have 2 x 130W CPUs, a slew of little SFF hard drives, and enough memory to last me a few centuries. A good stress test is running SETI@Home or another distributed computing project with BOINC That will fire up all your GPU and CPU cores. The server has 2 x 750W supplies and they run fairly warm with the server at full throttle. I have been thinking of adding two more supplies to ease the burden on the first two.

I haven't heard of HPE locking out unsupported GPU cards but maybe that only applies to more newer servers or specific models?


Boris
PeterPT
New Member

Re: HP Proliant DL580 Gen8 and GPU's

Trying to make a K80 work on the DL580G7 with no luck...any insights?

Have the pcie expansion set up and running a k40c at the moment but need more horsepower for simulations and render.

4xpsus up and running, all green and no errors.

 

Thx!!!