HPE BL465c G7 Upgrade from 5.5 to 6.0 not going well

[seandrethegiant]

I am trying to get my first BL465c G7 (AMD 6100 series processor) with Emulex NC551i upgraded to vSphere 6.0 U2.  I have run the SPP disk from late 2016 as well as run what I believe to the be the latest emulex upgrade disk (10.5.160.0) as the later versions of the Emulex disk (Legacy_OneConnect-Flash-11.1.183.23) doesnt update and the documentation doesnt list the NC551i that we have on our blades.   

Our chassis firmware for flex 10 is at 4.40. 

When I run the upgrade (or attempt to install from ISO) It installs and then I get a NIC not found error (more specifically the NIC with MAC ADDRESS xxxxxxxxxxx not found). 

Can anyone give me additional info on what drivers/firmware/update im missing to get vSphere 6.0 U2 installed?

[bl465c-G7-NC551]

I am also experiencing a very similar challenge.

I have HPE BL465c G7 with Emulex NC551i cards, worked fine with ESXi 5.0 and 5.5 using the be2net driver VMware_bootbank_net-be2net_4.6.100.0v-1vmw.550.0.0.1331820.vib

As for firmware versions, this version works in ESXi 5.5: be2net  Version: 4.6.100.0v, whereas none of these versions I tried: elxnet  Version: 10.2.309.6v (at VMware's request)    ;   elxnet  Version: 10.7.220.6     ;     elxnet Version:  10.7.110.44 (last two are for VMware 6.0 U2)

I am trying to get ESXi 6 Update 2 installed and have customised HP's custom image to remove the EMU_bootbank_elxnet_11.1.145.0-1OEM.600.0.0.2768847.vib as this is known not to be compatible and replaced it in image builder with EMU_bootbank_elxnet_10.7.110.44-1OEM.600.0.0.2768847.vib to no avail.

REFs: HPE VMware recipe - see p15 NC551i ; Apr 2016 VMware recipe SPP 201604_20.2 ; HPE Advisory c05154182

I end up booting into ESXi 6, but without network: "no compatible network adapter found. please consult the product's hardware compatibility guide (HCG) for a list of supported adapters"

Anecdotally there was reference on a website to disabling the elxnet driver and reenabling be2net, unfortunately the be2net isn't in the vibs in the HPE custom image for 6.0 U2 Dec 2016.

FootNote 10 in the HPE VMware recipe advises to use the HPE custom image 6.0 U2 Apr 2016, unfortunately I can't find that image and end back with the Dec 2016 image when I follow any links for the April download.

I've had a call with VMware tech supp for the past week without resolution - can anyone please advise how to resolve ?

[Torsten.]

I'm really not an ESXI expert, but I know firmware and driver must fit together.

Your firmware version probably does not match the driver, see for example:

http://h20564.www2.hpe.com/hpsc/swd/public/detail?sp4ts.oid=5033633&swItemId=MTX_e8cf2643b58d4cae862f7630b4&swEnvOid=4183#tab6

http://h20564.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=5033633&swLangOid=8&swEnvOid=4183

http://www.vmware.com/resources/compatibility/detail.php?deviceCategory=io&productid=19068

...

[bl465c-G7-NC551]

Thanks for the speedy response, Torsten, I should have also stated that all firmware was patched to latest SPP 2016.10.0 (at time of writing) using HP SUM 7.6.0 before running VMware updates to 6

[Torsten.]

The first link says

The firmware for the following CNA products is now frozen at version 4.9.416.15

What firmware is currently installed?

[bl465c-G7-NC551]

Hi Torsten,

the HP SUM report lists the firmware version as "4.6.100.0v" , which coincidentally is exactly the same as the driver version, so I have no confidence at all in the firmware version for the NIC cards, I'll keep digging see if I can find a firmware version that makes sense ;-)

OK the be2net driver reports the following (running the current ESXi 5.5 software):-

ethtool -i vmnic0:-

driver: be2net
version: 4.6.100.0v
firmware-version: 2.102.517.703
bus-info: 0000:04:00.0

so looks like the firmware is delivered as part of the driver vib

[Torsten.]

Looks like the firmware on the NIX is too old, you probably need

http://h20564.www2.hpe.com/hpsc/swd/public/detail?sp4ts.oid=5033633&swItemId=MTX_e8cf2643b58d4cae862f7630b4&swEnvOid=4183#tab3

[bl465c-G7-NC551]

Hi Torsten,

unfortunately that firmware won't load:-

"./CP028194.vmexe
Starting Smart Component...
argv[0]=[./hpsetup]
Calling OEMFLASHER.DOFLASH()...
Calling oem_do_discovery_with_files
m_oDiscoveryHeader.m_sDiscoveryFile=[/tmp/EMULEX_FC_HBA_DISC.xml]
m_sFirmwareBinDir=[./Flash/]
Segmentation fault
"

I wonder if it's for fibre channel cards only, although the driver title says "HPE Firmware Flash for Emulex Fibre Channel Host Bus and Converged Network Adapters for VMware vSphere 5.5 and 6.0" the Type says "Firmware - Storage Fibre Channel and Fibre Channel Over Ethernet"

I did look for network adapter firmware the other day and drew a blank, thanks for your recommendation though.

ta,

[Torsten.]

>> Segmentation fault

Something went wrong during file transfer I guess. Did you ftp the file?

However, here are the instructions again:

http://h20564.www2.hpe.com/hpsc/swd/public/detail?sp4ts.oid=5033633&swItemId=MTX_e8cf2643b58d4cae862f7630b4&swEnvOid=4183#tab3

The file contains a few different firmware files, but at least this applies to your problem:

http://h20564.www2.hpe.com/hpsc/swd/public/detail?sp4ts.oid=5033633&swItemId=MTX_e8cf2643b58d4cae862f7630b4&swEnvOid=4183#tab2

...

Updated CNA (BE2) firmware 4.9.416.15

...

The device itself has firmware, the firmware containing in the driver must fit this version, otherwise it won't work.

Take care to NOT update from a version 2.x or 3.x directly to 10.x, make sure you update to 4.x version!

[bl465c-G7-NC551]

Hi again Torsten,

I've repeated the download, checked the md5 sum again, uploaded to ESXi host and checked md5sum again, and still segfaulted:-

"md5sum CP028194.zip
6f72730d9ad278220f6a70a4adb1b5b5  CP028194.zip
/tmp/scratch/var/tmp # unzip CP028194.zip
Archive:  CP028194.zip
  inflating: CP028194.vmexe
  inflating: CP028194.vmfile
  inflating: CP028194.xml
  inflating: README.txt
/tmp/scratch/var/tmp # chmod +x CP028194.vmexe
/tmp/scratch/var/tmp # ./CP028194.vmexe
Starting Smart Component...
argv[0]=[./hpsetup]
Calling OEMFLASHER.DOFLASH()...
Calling oem_do_discovery_with_files
m_oDiscoveryHeader.m_sDiscoveryFile=[/tmp/EMULEX_FC_HBA_DISC.xml]
m_sFirmwareBinDir=[./Flash/]
Segmentation fault"

thanks for your advise so far, has the original poster not got a solution yet - it's gone all quiet there ?

I am expecting a webex with VMware this afternoon and will report how we get on in due course.

[Torsten.]

I would try to boot SPP in interactive mode and check if this update is selected.

option 2 would be to run the *.scexe file (need to get the exact name) manually directly in the SPP environment (per CTRL ALT d b x shell).

[bl465c-G7-NC551]

Hi Torsten,

VMware tech support are investigating offline, I'll see if they come up with a solution.

I did try the SPP offline automated firmware upgrades as well as the interactive firmware upgrades but the nic card firmware wasn't identified as upgradeable.

This is probably because there is no scexe file for the nic firmware upgrade, not going by what I've looked for anyway.

I am a little puzzled over the firmware versions though - is "CNA (BE2) firmware 4.9.416.2 (frozen)" newer than "Updated CNA (BE2) firmware 4.9.416.15" ?

On the basis 15 is higher numerically than 2 I would assume the former is the later, but on the other hand 2 could be interpreted as a higher revision in some cases (as in .20 vs .15 with a dropped zero suffix) and the 'frozen' would indicate the last/latest release, though the 15 is from a later version date than the .2

It looks like the .15 *.scexe file is "CP024662.scexe", though this isn't listed in the updates in 871790_001_spp-2016.10.0-SPP2016100.2016_1015.191.iso, so I'll download it and see what happens.

[bl465c-G7-NC551]

OK I tried to apply the .15 firmware via the ILO interface whilst booted into SPP and entered the console window but no luck :-

"smartstart:/mnt/cdrom/hp/swpackages # sh CP024662.scexe --direct

Error locating dependencies(flash.so, libz or libxml12)"

I also tried without the --direct switch and obtained the same result.

The PATH contains "/opt/hp/hp-phoenix/bin:/opt/hp/hp-phoenix/sbin::/opt/hp/hp-phoenix/srv/www/bin:/root/compaq:/sbin:/usr/sbin:/bin:/usr/bin"

[Torsten.]

During boot you should see a line similar to this. What version is shown?

[bl465c-G7-NC551]

Hi Torsten and OP,

I can see why you're an acclaimed contributor, it appears (early days yet) that updating firmware did the trick.

We have no connectivity with these versions of network driver firmware installed: 2.102.517.703 & 2.102.517.7 (sic)

The firmware utility that updated the NIC firmware to a working version (4.9.416.15) is available at :

http://h20566.www2.hpe.com/hpsc/swd/public/readIndex?sp4ts.oid=5033633&swLangOid=8&swEnvOid=54

I hope that helps someone else stuck in the same predicament (OP?).

thanks,

[neotic]

This is really annoying, but I've figured out a way to update the hosts

NC551i is running with the BE2-firmware, the latest firmware is 4.9.416.15 (it's in Legacy_OneConnect-Flash-10.7.110.38.iso) and HPE ESXi 6.0 iso's feature an unsupported driver for NC551i

Create an updated ISO with a supported driver (VMW-ESX-6.0.0-elxnet-10.7.110.44-offline_bundle-4014430)
I've used ESXi-customizer powercli script to inject the above driver into the HPE ESXi offline bundle (VMware-ESXi-6.0.0-Update3-5050593-HPE-600.9.7.0.17-Feb2017-depot.zip)
With that .ISO you can add it to VUM (vSphere Update Manager) and deploy it on your hosts

Don't forget to remove a conflicting driver before issuing the remediation
esxcli software vib remove -n net-mst

Installation succeeds!

But after reboot, you need to load the correct driver for NICs as a newer (unsupported) one is installed

# List installed driver
esxcli software vib list | grep elxnet
elxnet 11.1.145.0-1OEM.550.0.0.1331820

# Remove that driver
esxcli software vib remove -n elxnet

# These hosts can't access the shared storage since PowerPath isn't installed so you have to mount a folder through iLO
# (if you can access shared storage just put the driver there and install with esxcli software vib install -v /vmfs/volumes/<datastore>/<driver.vib> 
# Stop usbarbitrator service so you can access the folder from ESXi
/etc/init.d/usbarbitrator stop

# Attach folder through iLO
Virtual Drives -> Folder

# Load the driver for the folder
vmkload_mod iso9660

# Mount the folder on the ESXi host
/sbin/vsish -e set /vmkModules/iso9660/mount mpx.vmhba35:C0:T0:L0

# Install the driver
esxcli software vib install -v /vmfs/volumes/mpx.vmhba35:C0:T0:L0/elxnet-10.7.110.44-1OEM.600.0.0.2768847.x86_64.vib

 Unmount the folder, reboot and after that the NICs should work! 

[bl465c-G7-NC551]

Hi neotic,

we are experiencing occasional loss of network connectivity for the upgraded Gen 7's ( we didn't use your method as we'd already found our own method )

can you please confirm whether you have stability of the ESXi hosts upgraded using your method ?

thanks,

[bl465c-G7-NC551]

I downrevved the ESXi nic driver from 10.7.220.6 to 10.7..110.44 and used the nic driver firmware version 4.9.416.15 and this now works reliably