HPE Storage Tech Insiders
Showing results for 
Search instead for 
Did you mean: 

Nimble Fibre Channel: Nimble FC Boot from SAN


Over the last few days, we have been looking at the basic provisioning of data volumes to Windows, ESX and Linux.  We will now turn our attention to SAN Booting.  Booting a host from a SAN is popular as it it makes the host stateless, in that there is no implicit relationship between the server platform and it's boot disk (assuming the hardware and therefore the drivers are the same!).  Another good use case (especially for our lab environment) allows us to repurpose hosts quickly by just changing their associated boot devices, therefore a node one moment maybe a ESX server and by simply changing it's boot device may quickily reprovisined to be a Linux or Windows Server.

Boot from SAN leverages the capability of the Fibre Channel Host Bus Adaptors to pass a specific disks to the BIOS as a boot device.  In order to do this you are required to configure the Host Bus Adaptor first to Enable Boot from SAN and then to inform the card which devices it should pass to the BIOS and in which order.


As the disk is available via multiple paths, each path will be tried in turn and the first path to successfully provide I/O will be booted.  If the path is unavailable (either the path is down or the path is a Standby path) then the next path in the list will be selected and tried until a successful path is found (or all the paths are exhausted and no bootable device exists!).  The first step in booting from SAN is first to create your volume and write an OS to it.  I'm not going to cover that in this blog but essentially you would provision your volume in the Nimble OS (similar to the previous guides).  You would then boot your node using your regular bootable media (CD, DVD, USB) and during the install you would direct the install at the SAN attached device rather than an internal SCSI volume on the host.

Once the OS is installed, you would now need to configure the HBA's to select the volume to Boot.  The GUI's are slightly different for Qlogic and Emulex but the process is almost identical!  

Firstly boot the host.  During it's boot cycle press CTRL+E (Emulex) or CTRL+Q (Qlogic) to allow you to configure the HBAs:

(Left: Qlogic   Right: Emulex)


Next you need to select each adaptor in turn, in order to configure that adaptor:


Enable the HBA to Boot from SAN:


and then tell the HBA about the Targets and the Devices that it is able to Boot from and the order:


Note: In the above I am telling the Emulex cards to boot from LUN ID 0 which is why there is a couple of more screenshots.

You will also note in the above example that each Nimble Array Target port is listed along with it's associated WWPN. This effectively gives us the order in which the targets will be tried.  HBA1 in this instance is seeing Target Ports 1,3,5,7.  HBA2 when we configure that will see Target ports 2,4,6,8.  So mapping my Target WWPN's to the Arrays ports should allow us to define an efficient discovery order:


Ideally you should balance the selects across the A and B controllers as at boot time you will not know which controller is Active and which is Standby and therefore which paths are active/standby state.  So round-robining the choice should find the active path in the shortest time.

Once the device selections are configured.  Press ESC to get out of the menu, Remember to SAVE the configuration and don't forget to configure each port, if there are more than one port in the host!

All of the above steps can also be accomplished by the Host Bus Adaptor software, which is another good reason to install the software.

The final step is to configure the Boot device in the BIOS.  Boot your server and enter it's BIOS (usually pressing F2 during boot).  Then select the Boot Menu:


Select the Hard Disk Drives, if the HBA's have been configured correctly you should see the HBA Device as an option:


During the boot cycle it's always a good sign if you see your devices in BIOS when the adaptors initialise (as you know the Zoning is correct, the LUN Masking is correct and the HBA is configured to boot):


Finally the OS will boot!


The process of switching OS now is very simple (as the HBAs will alway boot to which ever device is configured to be LUN 0).  You can simply change which OS should be booted by bringing the correct volume online, adding the host to the Access Control List and associating the volume to LUN0:


The whole process from beginning to end is also recorded on the following video:

I'd highly recommend qualifying, testing and retesting with different levels of HBA firmware to ensure devices not only boot but they also boot when the first path is a standby path.  As sometimes bugs in the card adaptors specific firmware won't pass the next device in the list (even though they should).  Of course once the OS boots and the multipath drivers load then all paths, including the boot volume will be resilient to failures, you can see this from the previous Linux blog as the boot device (10GB volumes is fully protected with mpio as the host fully boots).

Tomorrow in the final series of the blog, we will look at Zoning.   As ever questions and comments are welcome below!

About the Author



We havea lot of issues to perform FC SAN boot for  ESXi 6.0.

There is no problem booting up ESXi 5.5 with 2 configured paths (Primary& Standby).

Environment:  ESXi 6 on Cisco UCS B200M4 blade server with SAN booting on Nimble storage (Cisco MDS 9148S as SAN fabric)

Assuming all configurations are properly setup, i can't seem to get it function on ESXi 6.0.

  1. If put in 2 paths to Nimble volume, reboot server will have core dump error and cannot find system log folder.
  2. If terminate 1 path (Using Nimble GUI which remove 1 HBA from initiator group). Reboot server with no issue.

  • Cisco UCS Manager and Firmware Version: 3.11e
  • Boot-loader version: v3.6.0
  • Kernel version: 5.0(3)N2(3.11e)
  • System Version: 5.0(3)N2(3.11e)
  • Nimble OS Version:
  • ESXi NCM Version: nimble-ncm-for-esx6-2.3.1-600006
  • ESXi Version: Vmware-ESXi-6.0.0-3620759-Custom-Cisco-
  • vCenter Version: VMware-vCenter-Server-Appliance-

Hi Nigel, are you able to show the screenshot of error when booting?