DL380G7 - PCIe and memory bandwidth?

[Alex Geroulaitis]

Please forgive the rookie for n00b questions:

1. What's the total PCIe bus bandwidth of the DL380G7 server?  The 5520 chipset appears to support 36 lanes, and if I am not mistaken, the server supports up to 32.  If it's indeed 32 PCIe 2.0 lanes - does it mean 16GB/s of maximum bandwidth?  (The plan is to stick a couple of 10GbE quad-port cards in, so the bus bandwidth may become an issue.)
2. Memory - it appears from the QuickSpecs (and from HP Memory Configurator Tool) that the maximum memory speed of 1333MHz can only be achieved with 1 module per channel, and the maximum size of each module is 8GB, i.e. the maximum memory that can run at 1333MHz is 48GB... Does this sound right?  I could swear the Z800 supported 96GB at 1333MHz despite having fewer memory slots...

Thanks!

[danielrbowers]

1. What matters is what bandwidth is available to the PCIe slots you use.  If you put a card into a slot that has a "x8" bus width (See "Expansion slots" section of the DL380 G7 Quickspecs), then that card will have a max bandwidth of 32Gb/s.  (8 lanes * 4Gb per lane).  In theory, that means the PCI bus could be a bottleneck for a quad-port 10GbE card (32Gb/s available bandwidth versus 40Gb from a quad-port card).   If that concerns you, you could use multiple dual-port cards instead of one quad-port card.

2. You can put 2 DIMMs per channel in the DL380 G7 and still run at 1333MHz.  That would be 96GB with 8GB DIMMs, or 192GB with 16GB DIMMs.  Make sure you pick 1333MHz-capable DIMMs, and a 1333MHz-capable processor; not all DIMMs and processors can do 1333MHz..

[Alex Geroulaitis]

Daniel - thanks - neither is an acceptable answer for me. 1 - I asked a question about the total bandwidth. Not interested in individual slot bandwidth, thank you. 2 - plz check your answer - e.g. run an HP DDR3 Memory Configuration Tool - and see if you can make 16GB DDR3-1333 modules work in 1333MHz on DL380G7.

(Thank you for trying to be helpful though - that's appreciated.)

[Suman_1978]

Hope this helps you.

http://www.intel.com/content/www/us/en/chipsets/server-chipsets/server-chipset-5500.html
"Serial I/O technology provides a direct connection between the MCH chipset and PCI Express* component/adapters with bandwidth up to 48 GB/s on each PCI Express Gen2 x8 interface. PCI Express offers higher bandwidth, lower latency, and fewer I/O bottlenecks than PCI-X."

HP ProLiant DL380 G7 Server - Configuring Memory
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c02215414

Thanks

[Alex Geroulaitis]

---

http://www.intel.com/content/www/us/en/chipsets/server-chipsets/server-chipset-5500.html
"Serial I/O technology provides a direct connection between the MCH chipset and PCI Express* component/adapters with bandwidth up to 48 GB/s on each PCI Express Gen2 x8 interface.

---

This is just strange: PCIe 2.0 is 500MB/s per lane (one-way) or 1GB/s per lane full duplex.  So an 8-lane PCIe 2.0 socket supports up to 8GB/s full duplex (4GB/s one-way).  So where did that "48GB/s" come from?

---

HP ProLiant DL380 G7 Server - Configuring Memory
http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=c02215414

---

Thanks, I did see that page; from the look of it, 48GB RAM (6x 8GB) is the maximum capacity at 1333MHz, like I said in my original post.  I just wanted to confirm that.  Can you confirm that? :)

Thanks again!

[Suman_1978]

Well I cannot confirm that personally but I will go with the documentation provided by HP.