Cannot configure SmartArray e208I-P SR GEN10 on HP MicroServer GEN10

[DanielL1]

Luck is what you need with this card indeed. And yes, as you've noticed, it upsets me, that luck is the only thing you and other HPE employees do wish us with this server + card combo.

As others and I wrote before: the card is not recognized in UEFI mode on some firmwares. So why the hell does anyone want to use SSA in UEFI mode?

The only firmware I know that's working in UEFI mode is 1.60. I've tested 1.65 which failed. I did not test any other versions as I'm running the card in a production system which I cannot use for testing every broken firmware.

For everyone who want this the combo to work in UEFI mode do the following:

1. Update server to firmware ZA10A360; I've used the linux bits with a CentOS live CD and restored BIOS defaults afterwards
2. Update the E208i-p SR Gen10 card to firmware 1.60; I've used the linux bits with a CentOS live CD for this as well (you can unpack the bits with "rpm2cpio firmware-smartarray-f7c07bdbbd-1.60-1.1.x86_64.rpm | cpio -id", then run ./setup within the folder) and restored BIOS defaults afterward again.
3. Use Service Pack for ProLiant (SPP) in legacy mode to configure the controller (I've used P09037_001_spp-2018.06.0-SPP2018060.2018_0618.64.iso). Caution: do never ever change anything in the Power Mode settings! If you do, the card will be dead. I don't know if it is dead in general but at least it is with the Gen10 Micro Server.
4. Switch to UEFI mode and install OS

If you have upgraded the card with a firmware that is not recognized in UEFI mode: check if it is there in legacy mode. If it is, downgrade the firmware in legacy mode with CentOS live CD. Just do the same as described under point 2.

General warning and recommendation: as SplatMan wrote: do not buy this server + card combo or return it if you still can. HPE support can not help you and you might spend hours to get this nightmare working!

[Edit: added caution to configuration ot the controller]

[Edit: added general warning and recommendation]

[SplatMan]

---

@Jimmy Vance wrote:

---

---

My comment displayed here needs to be in the proper context. Versions 3.25  and older of offline SSA will boot both BIOS and UEFI systems.. The later versions of SSA are designed only to work with UEFI based systems and are not created using the USB key utility. The latest versions are created by just copying the contents over to a FAT32 formated USB key.  Read the installation instructions on the download pages tab. 

All versions of the e208 firmware including 1.60 are avalable to download, click the revisions history tab to get to the older versions.  I currently don't have access to a Gen10 Microserver to look at, I do know they are different to work with than a standard ProLiant server

The SSA usitlity is just for creating and managing the array configurations, it doen't contain any drivers.

Thanks for the clarification. However: my comment still stands, especially as valuable to information to people who create their own USB boot configurations - regardless of OS and tools used.

The server will not - due to a lack of file system drivers in the EFI firmware - boot from USB keys that are NTFS formatted. This is a rare thing these days, where NTFS is the norm/standard for all things Microsoft, and it may confuse IT professionals who are used to create their own USB boot keys.

I am not sure it boots any Linux file systems natively either. The USB key supplied by HP with the server is FAT32 formatted even though it does not boot a Windows environment.

As such, the lack of proper/common file system drivers in the EFI firmware of the server is important information for IT professionals unfortunate enough to work with this server. They need to be aware of this so they can format USB keys correctly (ie. only with FAT/FAT32), or add the correct drivers to the EFI envrionment using the console, if that is required for some reason.

An EFI NTFS driver can be obtained here (among other places):

https://github.com/pbatard/efifs

It would sure be nice if HPE added an NTFS driver to future versions of the system firmware. Though, honestly, I am not holding by breath ...

[SplatMan]

---

@DanielL1 wrote:

For everyone who want this the combo to work in UEFI mode do the following:

1. [see original post above...]

That's just what this thread needed. Thanks!

To any HPE staff, here is a summary of issues you can take with you:

1.) The system firmware of the Gen10 MicroServer is generally very buggy, and it's obvious it has not been tested before release. It's also obvious nobody at HPE actually uses the boxes, because then you would have known what was wrong with them. This means your QA procedures for this product line is provably defunct - if it exists at all.

1.1) Latest system firmware does not work correctly with the SmartArray controller cards when those are also updated to latest firmware. When in UEFI mode the card is not initialized correctly and EFI drivers are not loaded. As a result, it is not possible to see neither the controller or disks attached.

1.2) Latest system firmware does not work correctly with the SmartArray controller cards when those are also updated to latest firmware. The controller will appear in the UEFI environment for configuration, but selecting ANY option in the controller menu will simply return the user to the first UEFI configuration screen, with no change and no explanation. As a result it is impossible to configure the controller in the UEFI menu system.

2.) The SmartArray controllers will only work in Legacy mode when updated to their latest firmware. In order to work with UEFI the card must be downgraded to version 1.60, which is contrary to official HP recommendations.

2.1) Note that the controller will be completely absent from the UEFI environment when latest software is used and system is not in legacy mode. It is therefore hard to identify what is wrong, and hard to correct the issue.

3.) Configuration software for the controller cards will apply changes immediately, rather than apply all changes collectively when all parameters and variables have been set. This makes it very hard to change configuration for the card if the system was booted from disks connected to the card. Changes to the running environment that affect the IO subsystem will immediately result in OS failure (for example BSOD in the case of windows).

4.) Configuring the SmartArray controller cards to use the "Balanced" power setting will apparently brick the card, at least in the Gen10 server. I have no desire to test if it's possible to unbrick it by mounting it in a non-HP server, or my local workstation. But if the Gen10 server is all you have for this card, then changing the Power settings for the controller will result in a defunct server, and only an RMA case will solve the issue.

5.) The Gen10 MicroServer EFI environment lacks common file system drivers. This means that standard boot media from Microsoft - arguably one of the biggest vendors of commecial software on the planet - does not work with this server. Customers must follow special procedures, or must make their own custom boot devices with this server. This is due to a lack of attention to detail, and a lack of QA procedures for the product line.

6.) The policy of locking firmwares and updates away behind some kind of "paywall" is extremely frustrating and hostile to your customers. It creates ZERO value for us. I don't know who though that "Entitlement Required" was a great concept to introduce for something as simple as firmware upgrades. But realize this: There is ONE SINGLE reason for customers to download firmware. And that is to FIX PROBLEMS THAT SHOULDN'T HAVE BEEN THERE IN THE FIRST PLACE, for products we already paid for. Asking me to pay multiple times for the same - defunct - product is not a good way to keep our relationshop healthy. No other vendor, bar Cisco, gives us that kind of crap.

7.) Amazingly, the download page for the Gen10 MicroServer does not work with Chrome, which is the most commonly used browser on the planet. This is once again something that is telling about HPE's lack of attention to detail and QA. It looks like some async call to populate the user page with possible downloads fails in Chrome. Copying the URL to another browser "fixes" the problem. And the most annoying thing is: this problem is probably only occuring thanks to the whole customer hostile "paywall" concept, because otherwise serving a list of downloadable files should really be a piece of cake...

I wish you all a happy new year.

[SplatMan]

*** UPDATE ***

(This post left here for honesty and clarity - please see UPDATE 2 post below)

On dec. 20. 2019 a bunch of new updates were issued to the Gen10 Micro Server. Sadly no updated System ROM (so that's still ZA10A360 as of this writing), but there was an updated firmware for the SmartArray controller to v 2.62.

I tested this update so you didn't have to. Results:

1.) The controller still can't be configured from the UEFI menu system. The existing error where the user is simply returned to the first UEFI menu screen is still present, making it impossible to configure any aspects of the arrays/disks without booting a full operating system.

2.) The controller still doesn't work when the system is set to UEFI mode. The only way to use the controller or any drives attached is to force Legacy mode on. It also doesn't work with CSM, even though one has to wonder why (the option ROM should be loaded with CSM enabled but it doesn't).

If you need UEFI mode and you applied the new version, you may have to downgrade to v. 1.60 which is the highest version people have working with UEFI mode. Be aware that several issues listed as fixed in later versions could be affecting you, so make sure you understand the consequences of running on that version. Also be aware that there is no guarantee that 1.60 will work in UEFI mode. My setup - for reasons unknown - doesn't.

Note that booting an OS from a USB Key while in UEFI mode also doesn't work. The controller won't be initialised, so it won't be visible to you.

If you already have version 1.60 on it, and you're booting in UEFI mode then DO NOT UPGRADE THE FIRMWARE. If you do, for example to v 2.62 as "recommended" by HPE, the system will stop working and you will be unable to boot untill you downgrade the controller firmware. If at all.

** SEE UPDATE 2 BELOW FOR MOST RECENT INFORMATION **

Thanks HP ... (no, not really).

[SplatMan]

** UPDATE 2 **

I was in fact able to configure the SmartArray controller to work with UEFI mode. It took a lot of trial and error, and since the server's POST process is really slow, it's something that requires patience.

To be clear: You can get the controller working in UEFI mode with latest firmware, but there are a lot of pitfalls.

Here are the things to consider during installation (use at your own risk):

1.) After updating server+controller to most recent firmware versions you still can't configure the arrays from within the UEFI menu system, even though this is clearly something that was intended. The menu system will just throw you back at page 1 of the menu system. So forget about configuring anything from here.

2.) The controller can not be configured in UEFI mode using HPE SSP software. It will start booting but eventually halt at a white screen. I am told some people have had luck configuring the controller using command line options through Linux, but don't expect to configure the controller while the server is in UEFI mode.

3.) If you install a full OS (for example Windows 10 or Linux) while the system is in Legacy mode then you may be able to configure it. If your Linux-fu is not strong, then I suggest using Windows 10 and the HPE SSA software package. To achieve this, attach a SATA device to the internal port and install your OS of choice on it. You will then have a permanent boot option in Legacy mode. From here you can configure the controller.

4.) If you experience problems configuring the controller, you may need to reset it's configuration completely. There is an option in the software to do that. In SSA it's a button called "Clear Configuration". Reboot when you're done. Note that this should not break your arrays, since the configurations of arrays is written directly to the disks. Having said that, as I noted above, I take no responsibility for your use of these tips/guides.

5.) With the system in legacy mode and the ability to boot, and having cleared the SmartArray controller config, configure the array(s) you need. If you plan to boot from any devices attached to the controller, make sure to enable them as boot devices in the controller config software. In SSA this is an option called "Set bootable Logical Drive/Volumes".

6.) When you're done with these steps, remove the legacy SATA device and set the server to UEFI mode. I suggest powering it off completely after reverting to UEFI mode.

7.) You may now use the SmartArray controller in UEFI mode.

As mentioned in other posts: DO NOT CHANGE THE POWER SETTINGS TO "BALANCED". Doing so may brick your controller and leave you unable to use it. If you already did this, contact HPE to start an RMA case.

I wish you the best of luck.

[Jimmy Vance]

Now that I'm back in the office, I was finally able to get ahold of a Gen10 Microserver.

You can indeed setup the RAID controller using UEFI mode with offline SSA and the e208i controller using the latest versions of the AMI ROM and Smart Array firmware.

The versions I've tested with are 

ROM ZA10A360 and firmware versions 1.65 and 2.62.

I can understand your frustrations, this process needs to be documented much better.

As mentioned elsewhere in this thread, from the AMI UEFI Setup screen, even though the controller is listed, you cannot access any of the configuration options from this menu.  I'll assume this is due to the fact this system uses an AMI ROM and not a HPE ProLiant ROM. Systems with ProLiant ROM you can access these options. I'll report this back to the Microserver team so they can address the issue.

To configure the controller, you need to use the Offline SSA utility.  Today I used both v3.30 and v4.15 of offline SSA

With the system in UEFI mode, boot from the USB key that has had the contents of the ISO image copied to it.

Offline SSA is a Linux based boot environment, unfortunately the kernel version in the tool has KMS issues with the AMD video in the Microserver Gen10,  If you were going to load a Linux OS with a GUI, you'd have the same issue. To load a Linux OS you will also need to add an option to the bootline. I was using 'nomodeset', but as I discovered with Offline SSA versions 3.40 and newer, the boot option 'modprobe.blacklist=amdgpu' has been added to address this.

Working combination

Microserver Gen10 with E208i in UEFI mode

System ROM ZA10A360
e208i Firmware v2.62
Offline SSA version 3.40 or later

Copy the contents of the SSA ISO image to a USB key to boot.
When booting form the USB Key select "USB Boot: Smart Storage Administrator" or "USB Boot: Smart Storage Administrator CLI"

You can then configure the RAID and controller settings

If your going to run in Legacy BIOS mode, you will need to set the system to UEFI mode to run the utility or custom build a bootable USB key to boot via MBR. you can use one of the older versions that support BIOS/UEFI boot as a guide on how to add syslinux boot options

If using legacy boot mode, you will need to Set Bootable Logical Drive/Volume (this isn't required for UEFI boot) under controller configuration

I did not try setting the controller to balanced power mode, as mentioned elsewhere in this thread, this causes issues. Once again, I'm assuming this is due to the AMI ROM not being fully compatible with ProLiant ROM features

Hope this helps

[mprenter]

e208i Firmware v2.62 no longer available by HPE