Firmware updates - time and threshold

[kmn3686]

Howdy, 

I am looking for some feedback regarding Firmware updates via OneView build 5.02. Have been utilizing OneView the past few years starting with 3.x, 4.x and we recenlty upgraded to 5.2

I was watching a demonstration video prior to upgrading to 5.0, one of hte whats new regarding OneView 5.x from HPE Product and Solutions now page. One of enhancements discussed was 5.0 considerably faster install times when deploying firmware / firmware updates. Sine we have upgraded to  5.0 we have not experienced any improvement regarding faster install times during firmware upgrades and or installs. No other variables have changed in our environment, utilizing the same hardware / networking, 10gb network switche and uplinks, however our firmware updates continue to average around 45 minutes when upgrading firmware from templet.

We use server profile templates and our process for updating consists of downloading the new firmware bundle, upload to Oneview, attach to template and with the server powered off choosing the update from template option.I was just looking for some feedback from others on average time they are experiencing.  Should I be expecting faster install times, any recommendations?

Another question I have for which I haven’t found a definitive answer...  the number of firmware updates you can have running simultaneously before performance is impacted. We have decent size environment and with average firmware update taking 45 minutes it can make for long change windows when trying to get our hosts updated, the more firmware updates we can run simultaneously the better.

Thanks in advance    

[ChrisLynch]

Faster deployments are a combination of a few different variables:

• The version of HPSUM within the SPP
• Gen10 (when iLO5 is at 1.10 or newer) versus Gen8 or Gen9
• Online versus offline

If you have the ability to, using iSUT can make the process go faster, as the components are staged on the host before installation.  Then, a reboot of the server is needed (called Activation).

OneView can queue up to 64 parallel operations for offline Firmware Only operations.  OneView can support up to 128 parallel online updates.

[MCSAP]

If OneView is supposedly the manager of all managers, orchestrator of HPE components, and single pane of glass...why must we use additional products for patching servers?  (ie. iSUT, HPSUM, SUT)

I can almost understand why it would be required for legacy generations 7,8, or 9...almost.  But, it seems like you are asking alot from the customers to maintain all these different components to manage/patch HPE servers.

When will HPE finally have a single product that does everything these three or four products do to maintain the HPE servers?

[ChrisLynch]

If OneView is supposedly the manager of all managers, orchestrator of HPE components, and single pane of glass...why must we use additional products for patching servers?  (ie. iSUT, HPSUM, SUT)

SUT and iSUT are really the same thing.  OneView has no access into the operating system.  So, for OneView to deploy components online, and same with HPSUM and Gen10, SUT is required.  It is the bridge between the iLO, OS and deployment tool (HPSUM or HPE OneView).  HPSUM is ultimately the engine used to deploy components, which is why Gen8 and Gen9 SPP ISO's require HPSUM to be present and it to be bootable.  HPE OneView orchestrates HPSUM to deploy the components to Gen10, while HPSUM is executed to inventory and deliver components to Gen8/9 systems.

I can almost understand why it would be required for legacy generations 7,8, or 9...almost.  But, it seems like you are asking alot from the customers to maintain all these different components to manage/patch HPE servers.

I'm not sure if I understand.  Once iSUT/SUT is installed within the OS, and IF you are performing an online installation (which you should be, otherwise why are you installing iSUT/SUT), it will automatically be updated to the current version present within the SPP/Baseline.   HPSUM is included with the new SPP we publish.  Yes, you could run HPSUM as an internal service for your team, but that is independant from HPE OneView.

When will HPE finally have a single product that does everything these three or four products do to maintain the HPE servers?

These are not 4 different products.  iSUT/SUT is a utility that can be installed within the OS, which typically we would recommend customers install during the OS installation/provisioning process.  It is not required to install firmware using the offline method.  It is only required when you wish to install driver, firmware and HPE utilities while the server is powered on, and running a supported operating system (RHEL, SLES, Windows or vSphere).

If the ask is "why can't HPE provide a way to ensure iSUT/SUT is present and install if missing", we already do check for iSUT presense, and installing it is not a trival process.  As I stated above, HPE OneView does not have access to the OS.  It does not inventory, nor have authorization access to the OS, which typically needs root or Administrator rights, and seperate communication path.  Which then has its own set of security challanges (like HPSIM did for those that have experience).  

I hope this helps.

[Sri8]

we are having similar, firmware updates are taking ~1Hr to complete. we have tried different options online vs offline, there is no difference in the time lines. Staging is not an option for production environments, as it might violate internal process , and might rise more cocerns if somwthing happens while staging a FW, for example if server crashes. These are common problems. we are looing for better approach and faster way to deploy firmware to servers including server OS patches.

[ChrisLynch]

Staging would be the recommended approach to reduce the overall installation time. Staging installs the firmware and/or software updates to the servers. Then if a reboot is needed, a reboot be performed. When that reboot his needed, the administrator can control that behavior with the SUT Operation Policy setting (AutoStageNoReboot). OneView would report the server update is pending a reboot, or Activation, of the firmware.

Offline is the longer process as the server needs to reboot into an automation environment (the SPP or Smart Update Boot Environment, aka SUBE) and then reboot one more time.

We do look at ways to improve the overall experience. And work with our partners as well.

[Sri8]

Hi Chris,

Thank you very much for quic response. Really appreicate it. The challenge we are having is

we run vmware hypervisor, we have specific maintainance windows to deploy vmware patches and HPE firmware , our procedures do not allow to touch even staging on production servers ( I belive most customers follow the same change control process to deploy anything). we have many HPE servers, and allowed to perform only during maintainance window, no changes are allowed outside change window ( even staging is called change). While critical production vms are running, I beleive no one want to make changes underline hardware with out putting host in maintainance mode.  

Here is the process we follow , place host in MM/assign FW base line in one view/ One view stage and deploy/REboot as needed is taking 1hr per server. with this approach it will take months to push fw.  Was there any aleternatives to reduce the window following this process. Thank you.

[ChrisLynch]

Customers should be using the HPE. OneView for VMware vCenter plug in, which provides the VLCM provider to perform these operations. With the VLCM integration, VMware controls the host going into maintenance mode, and the installation happens online. Once the OEM update is done, VLCM does the vSphere updates and reboots the cluster node. Otherwise, yes, a complex operation process (manual or scripted) would be needed as you stated.

[Sri8]

Thank you.  Yes we do use HP one view for vcenter plugin and deploy using VLCM. what ever the process we follow there is no change that takes time to deploy. Looks like no alternatives to reduce the window.

[pirx]

I can confirm the issues with timeouts. It's just not working for us in 80% of the cases. I looked into raising the timeout for tasks in vCenter - which is by default 60min and is not mentioned anywhere in the OV universe as far as I know, but we are still running into timouts. Also typical, server is patched, last step is compliance check that fails as - for whatever reason - OV does not have the same status of fw versions as the ILO has. If I do a manual compliance check after that, it's ok. From all the contacts I have in this business that are using HPE servers, none is using OV. Some of them tried and failed same way I did. Just with putting less effort in it.