topic Gen10 Array Controllers in mixed mode in ProLiant Servers (ML,DL,SL)

Gen10 Array Controllers in mixed mode

ralfgro — Tue, 14 Apr 2020 13:29:49 GMT

Hi,

for Ceph we want to use bunch of DL380Gen10 server with i816a array controller. 2 Disks in RAID 1 and the remaining disks in HBA mode for Ceph. I know that there have been issues with mixed mode and that it's still not supported for VMware vSAN (IIRC). According to https://h20195.www2.hpe.com/V2/GetDocument.aspx?docname=A00017196ENW mixed mode is supported with Gen10 controllers. But it also states: "Improved efficiency with mixed mode for Smart Array Controllers". What impact/downside does this mixed mode have? Is it just performance, if yes how much?

Someone recommended me to just use a seperate RAID 0 for each disk, but this seems to have an impact on disk replacement and needs a reboot to confirm the data loss (I could not check this).

Re: Gen10 Array Controllers in mixed mode

SanjeevGoyal — Thu, 16 Apr 2020 08:55:22 GMT

Hello,

I hope all is well.

I think you are using the P816i-a controller not i816a. If yes, follow the below documents for more clarification.

HPE Smart Array P816i-a Controller – Overview

https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00018964en_us

HPE Smart Array SR Gen10 User Guide

http://itdoc.hitachi.co.jp/manuals/ha8000v/hard/Gen10/SA/879006-002_en.pdf

VMware Compatibility Guide

https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsan&productid=41786

If you feel this was helpful please click the KUDOS! thumb below!

Regards,

Re: Gen10 Array Controllers in mixed mode

ralfgro — Thu, 16 Apr 2020 09:09:22 GMT

Hi,

thanks for your reply, I know these documents. I'm refering to https://kb.vmware.com/s/article/2129050

If the vSAN disks are in pass-through/JBOD mode, the non-vSAN disks mustalsobe in pass-through/JBOD mode

We are not using vSAN, we are using Ceph. and I know that p816i-a controller supports mixed mode. But I don't know the impact of mixed mode, using 2 disks in RAID 1 for OS and remaining Ceph disks in HBA mode. What are the drawbacks? If this configuration is not supported for vSAN, I can imagine that it might affect Ceph too.

Re: Gen10 Array Controllers in mixed mode

SanjeevGoyal — Mon, 20 Apr 2020 09:11:13 GMT

Hello,

SA P816i-a controller supports mixed-mode and you can use OS drive and configure the rest of the hard in VSAN.

But I would like to suggest configure OS hard drive with another SA controller and configure VSAN in other SA controllers to avoiding any issue.

If you feel this was helpful please click the KUDOS! thumb below!

Regards,

Re: Gen10 Array Controllers in mixed mode

ralfgro — Mon, 20 Apr 2020 09:19:56 GMT

This is about Ceph, not vSAN. I brought up vSAN only because there is a VMware KB article that states that mixed mode is not suported with vSAN.

You write that you do not suggest to use mixed mode to avoid issues. What issues? That's what this thread is all about. To get a feeling what issus might be introduced by mixed mode. Some technical information. But I could not find any information _why_ mixed mode is a problem?

Re: Gen10 Array Controllers in mixed mode

SanjeevGoyal — Wed, 22 Apr 2020 13:07:53 GMT

Hello,

No issue with the mixed mode.

I would suggest you follow the below documents for more clarification.

HPE Smart Array SR Gen10 User Guide

http://itdoc.hitachi.co.jp/manuals/ha8000v/hard/Gen10/SA/879006-004_en.pdf

Mixed mode (RAID and HBA simultaneously)
Any drive that is not a member of a logical drive or assigned as a spare is presented to the operating
system. This mode occurs by default without any user intervention and cannot be disabled. Logical drives
are also presented to the operating system.
Controllers that support mixed mode (P-class and E-class) can reduce the number of controllers in the
system and efficiently use drive bays within a backplane. For example, a solution that needs all the drives
presented as HBA (except a two-drive mirror for boot support) can be accomplished with a single
controller attached to a single backplane.

If you feel this was helpful please click the KUDOS! thumb below!

Regards,

Re: Gen10 Array Controllers in mixed mode

ralfgro — Wed, 22 Apr 2020 13:22:19 GMT

No issue with the mixed mode.

Okay, then I still do not understand why VMware does not support it for vSAN.

Do not mix the controller mode for vSAN and non-vSAN disks.
If the vSAN disks are in pass-through/JBOD mode, the non-vSAN disks mustalsobe in pass-through/JBOD mode.
If the vSAN disks are in RAID mode, the non-vSAN disks must also be in RAID mode.
Mixing the controller mode will mean that various disks will be handled in different ways by the storage controller. This introduces the possibility that issues affecting one configuration could also affect the other, with possible negative consequences for vSAN.

I guess we have to find out how mixed mode works with Ceph and if there will be issued as with vSAN.

Re: Gen10 Array Controllers in mixed mode

KevinSpringPM — Wed, 22 Apr 2020 15:38:49 GMT

yeah, this comes down to quirks with the software vendor's implementation. Could vSAN make their software work well with mixed mode? Maybe. But they'd have to make it work well with every vedndor's controllers which may not work as well in mixed mode as HPE's. This question is better asked to whoever you're working with on your Ceph delployment, whether that's Red Hat, Suse, or someone else. It will come down to what they're comfortable with.