Microserver Gen10 Plus PCIe bifurcation support issue

[Penn_ch]

Thanks for your reply,  That helps a lot.

sad for It's not working...

[MJFH]

Hey, y'all, again!

Thanks for that Kudo, which motivated me to re-read the whole thread.

There's something i'd like to add; I'd like to disagree with @KevinSpringPM -- "would probably need some kind of riser to be useful, which wouldn't fit in the MicroServer".

There ARE PASSIVE bifurcation adaptor cards - even LOW PROFILE ones, which would fit just fine - trust me, i've just re-sold a complete low-profile kit including a card (with PCIe clock re-distributor ICs) , slimSAS cables and a quad-slot x16(4x4) socket board, the other month.

It's either a feature HP/HPE, stubbornly, will not support, via the 8th political layer (OSI model)… …or they're too proud to admit that their hardware designers didn't plan for it (…or have been told [edit: "not"] to plan for it) …i personally have no idea why there's actually no PCIe bifurcation.

I have yet to read about a successful bifurcation scenario, involving a MicroServer Gen 10 Plus.

However, if you can afford an active one, a card with a PCIe switch/bridge - easy-peasy - there are some options, which require a rather steep fee, though LOL … from Qnap and Synology - but that's not what we're talking about, in this thread. (iirc)

Does HPE even offer bifurcation accessories for this MicroServer? <- this answer should yield a final description of their intentions, imho.

EDIT: semantics + typo

(btw: happy new beer, y'all)

[mnkdk]

I can confirm the findings by @MJFH , that the passive 4 x NVMe cards only has one working slot when installed in the Microserver Gen10 Plus. I ended up buying the QNAP QM2-2P-384 which works as expected. The only downside is the price and that the card comes with a rather loud fan that kicks in from time to time - not because the SSDs get hot, but because the whole cards sits on top of the iLO BMC which is constantly 80°C. By the way, why is the iLO BMC consuming 8 watt of power when idle, when a Raspberry PI 2, which would probably be powerfull enough to run the iLO, uses only 1 watt?

I really hope that somebody creates a passive(and inexpensive) 2 x NVMe card that uses the 8x8x bifurcation, since it would make it an ideal ZFS NAS box: 1 SATA SSD for OS, 3 spinning SATA disks in raidz1, and the two NVMe SSDs as special devices for metadata and small files.

[Skyweb]

Hello
I bought this model:
Linkreal PCIe x16 4-Port NVMe M.2 SSD Adapter
LRNV9547L-4I
It is a low profile card with x4 NVMe M.2 ports : no problem to insert inside the server
And it is independant to the motherboard bifurcation
I have 4 SSD connected
All recognizes
Only one problem: it is hot ~86°C (07-BMC sensor)
Actually sell at ~240$

[NunoF]

Anybody had luck with 8x4x4x bifurcation?

The MB and the processors support this mode and I found a board that apparently also supports this mode properly to expose 3 nmve disks.

https://amzn.eu/d/gVBl14y

My use case would be 3 cache disks (2 of them in Mirror)

Regards,
Nuno

[MJFH]

Hi @NunoF!

The card you're talking about doesn't require properly working bifurcation, since it's got an asmedia ASM1184E PCIe packet switch. The advertisement is missleading, though - see asmedia specs for reference: https://www.asmedia.com.tw/product/556yQ9dSX7gP9Tuf/b7FyQBCxz2URbzg0 - you'll probably be able to run 4 NVMe SSDs, but you'll only have 1 single lane of PCIe Gen2 (that's the spec of the upstream port, which connects to the motherboard). That's a terrible deal, wich kinda negates the purpose of an NVMe cache.

The Startech pex8m2e2 uses a better switch chip - iirc Gen3 x8 towards upstream and 2x m.2 downstream - no mobo-bifurcation required.

[NunoF]

Hi @MJFH ,

Thank you for taking the time to look at this.. indeed the information on the ad is not coherent... What is described feature wise is something that would be possible with ASM2824 but not with ASM1184e. 

In either case, bifurcation in MB would not be needed since these are PCIe switches.

Tks

Nuno

[NunoF]

Hi @MJFH ,

I contacted the seller and he told the following:

Yes. PA41 uses ASM1184E. This chip does not has pcie bifurcation function, so your motherboard must has pcie bifurcation function.

         PA41 only use the clock signal of ASM1184E,  and do not use data signal of of ASM1184E. That is why PA41 can support gen4 features.  

I actually don't know if this is possible, I guess that if the MB/BIOS can support 8x4x4x then I would get 3 working NVMe at PCIe Gen3 speeds.

Probably the only way to know is to test it. 

[NunoF]

I am back with (bad) news.

I ordered the board and I could only get to recognize NVMe in the first slot.

What is odd is that I can no longer find the bifurcation option in the BIOS.. I am sure I had seen in there but it is no longer available (I updated to the latest BIOS last week).

Can someone with the last BIOS confirm that the PCIe bifurcation option has indded disappeared in the latest BIOS?

Thanks,

Nuno

[Liceo]

I know, it doesn't solve the issue but: I just received some of the brand new MicroServer Gen10 Plus v2 with the new Xeon E-2314 which supports up to 64GB of RAM and have PCIe 4.0 on board.

Bitfurcation still doesn't work, but i have a single 2TB Samsung NVMe 990 Pro mounted in a Delock Riser Card. The performance is great, i got up to 7100 MByte/s sequencial read an about 6000 MByte/s write. I'm running on Windows Server 2022.

I'm plan to test in Windows 2022 with Windows Storage Spaces to see if i get storage tiering works in order to use the NVMe SSD as a fast cache. Would be nice to increase the over all capacity whilst keeping the performance. I don't need "in-box" redundandency, as i always deploying at least two micro servers to achieve HA setup.

Let's see how it goes..

[stonktrader]

Can confirm, from the BIOS 2.60_07-14-2022 it explicitly stated that "Removed PCIe bifurcation option in the ROM Based Setup Utility (RBSU) because it is not supported through ROM on this platform".
I even performed the System Recovery Set to roll back bios to v2.52 but I still cannot get the bifurcation work.

[ETOPS-370]

Hello,

is the bifurcation limitation also afecting the HPE ProLiant MicroServer Gen10 Plus v2 (Xeon E-2314)? 
I can't see the bifurcation option in the BIOS.

I have mouted this PCIe Card: 
Delock PCI Express 4.0 x16 Card to 4 x internal NVMe M.2 Key M 110 mm - Bifurcation - Low Profile Form Factor (https://www.delock.com/produkt/90090/merkmale.html?f=s)

On the PCIe card I added 4 x NVMe M.2 SSDs, but one one SSD is reconized, the other 3 are not recognized. I tried to access in the BIOS the bifurcation menu, but that bifurcation menue seems to be gone

Thanks for any hints.

[ETOPS-370]

Hello everyone,

I got today the official confirmation from HPE support that the PCIe Bifurcation functionality has been completely removed from the Gen10 Plus Microserver. 

I have to say that I am very disappointed about the decision that HPE has taken. Removing the Bifurcation functionality from a server that has only 1 PCIe slot it basically killing the Unique Selling Proposition for that server. This Microserver is a great piece of hardware and unique in the market, but without Bifurcation it is basically useless for many usecases.

I will avoid HPE products in future.