- Integrated Systems
- About Us
- Integrated Systems
- About Us
11-08-2011 01:14 PM
Customer wants to swap out VC 1/10F for VC FlexFabric modules....
Andew was looking to help a customer:
"We have a lot of configurations for Virtual connect Fibre Channel slots 3 / 4 and Virtual connect Flex-10 in slots 5 / 6. All of which would have to be manually rebuilt because the flex fabric module replacements in slots 1 /2 will likely change all macid, wwn, networking and other virtual connect specifics related to the server profiles.
With this upgrade many business infrastructure dependents (ad, dns, dhcp, dfs, nap, iis-farms, sql-farms, sps-farms, rds-farms, clusters, hv csv, sc-vmm, email, internet, tmg arrays, uag array, vpn, etc, etc, etc) will be down until the full reprogramming of virtual connect is back online."
The customer have a variety of blades G1, G5, G6 and G7.
They have already purchased the FlexFabric modules that they plan to replace the VC 1/10F modules. The customer is concerned about the impact during the upgrade to their configuration.
My suggestion to the customer is to use HP Services or buy another C7000 enclosure - install the FlexFabric modules in to the new C7000. Unfortunately the customers datacenter is not able to install another C7000 at this time.
The concern is that when the interconnect bays 1 and 2 are replaced with VC FlexFabric modules this will require the configuration to be rebuilt from scratch. The customer IT environment will be impacted during the down time.
Should the customer attempt to perform this upgrade - they have setup their initial enclosure and current environment.
Is this going to be a easy or hard upgrade?
How much downtime should be expected?
Should steps be done prior to the upgrade.
Could the following be done?
Update existing FW to 3.30 (Note - read the CA).
Back up the existing VC configuration.
Remove both 1:10F modules.
Install both FlexFabrics
Rebuild configuration for external connections check each profile for connectivity to external links.
Test each server for network and FC connectivity..
Considering the VC 1/10F modules are the Primary modules of the domain, upgrading them to VC FlexFabric modules will not be supported. You will be unable to swap modules one-by-one, or even restore a saved domain configuration from backup. You will need to recreate the entire VC domain from scratch.
Considerations from Alex:
In addition, going from 1/10 modules to FlexFabric modules will change the way your MAC addresses were allocated in the server profiles if you have G6 and G7 servers which are Flex-10 enabled. Now instead of having dual port 10GbE adapter as a LOM, you will have 8 FlexNICs on the LOM for the G6/G7 servers. This will cause profiles to have more network connections. One big caveat that they need to be *very* aware of is that mapping of the enet-connections in the profiles to vmnics will change drastically. If you are running VMware, they will either need to adjust their vSwitch configuration or completely re-do the profile to account for the round-robin enet connection allocations. With the exception of vmnic0 & vmnic1, all other vmnics will either point to an unlinked/unmapped or different downlink ports. As you can see, you are going to have to do a whole lot more manual work than just what VC requires.
Unfortunately, I don’t believe this upgrade can be simply accomplished by creating CLI scripts from VCM side.
Advice from Markus:
I'd like to point out some other experience I made.
Be careful with your profiles, you might need to rework them completely.
If you e.g. had G6/G7 hooked up to 1/10F modules before and used Flex10 Mezzanines a change of the 1/10F to Flex10/FF will dramatically change the assignment of your networks to the FlexNICs as you can use the onboard FlexNICs now too.
This brings some aspects that need to be considered:
a) Networks show up on different NICs now, so you might have to change vSwitch or teaming configurations
b) there might be more "inter VC" traffic going on, so your stacking links might become more utilized.
Not applicable to this case here but also something useful to keep in mind is that you potentially need a new/other driver if you add Mezzanines for additional interconnects. Also adding Mezzanines/Interconnects has a similar effect as changing 1/10F with Flex10, because of the way VCM assigns networks to NICs (max performance and max redundancy) so a & b above also apply if you add new VC modules.
Have you done an upgrade and had to rebuiild everything? Let us know how you did it.
11-09-2011 04:52 AM
Re: Customer wants to swap out VC 1/10F for VC FlexFabric modules....
We did something similar about 15 months ago, i.e. 1/10 VC Ethernet Modules to Flex10 Ethernet modules.
As already stated, the 1/10 VC configuration is NOT compatable with the Flex10 modules. (As far as I am aware, this has not been addressed yet by HP). From what I see, they modified the documentation so that this incompatability is noted.
It is possible to script the whole reconfiguration process, and ensure that all WWN and MAC's remain the same, however it is not trivial. Bury your head in the VC CLI Scripting manual (the scripting is not difficult, however it takes considerable thought in order to get the required final product),
If you talk to your HP Services person, it is possible that HP can provide you with a script (derived from your existing configuration). Be prepared to pay if necessary.
In our case, We generated the scripts ourselves, however we had the advantage of a test enclosure to play with, and still got bitten (slightly). The basic reason why the old config will not work on the new modules is that the numbering of the downlink ports is reversed on the new modules, i.e. instead of 1-16, it is 16-1)
Anyway, good luck