BladeSystem - General
1748185 Members
3913 Online
108759 Solutions
New Discussion юеВ

Re: Advice on setting up new blade systems

 
Mike O.
Regular Advisor

Advice on setting up new blade systems

We recently purchased two C7000 blade systems to use with VMWare to migrate approximately 130 older Proliant systems. We're working with a consultant on the virtualization migration, but we need to do the basic hardware setup ourselves.

Along with the hardware, we also purchased some HP training for the Blade system, the Insight software, and the Virtual Connect interconnects. However, due to a whole series of issues, the training purchase was delayed, and the result is that we have the hardware, and need to get it going, but won't be attending any of the training for at least a month.

We've been assembling the hardware and getting it ready, and I'm looking for some suggestions on the next steps.

Here's the status:

We have two C7000 chassis. Each chassis has four Virtual Connect Ethernet interconnects, two VC Fiber interconnects, and two onboard admin cards. We have 5 BL465G5 blades for each chassis, each with two AMD quad core CPU, 32G of RAM, and an Ethernet and Fiber Channel mezzanine card.

All of the chassis hardware has been assembled, but I haven't powered it up yet. None of the Ethernet or Fiber Channel virtual connect modules have been connected to our production network or SAN infrastructure.

We're also setting up a dedicated DL360 to run the HP Insight software. The network ports from the onboard admin cards (two from each chassis) and one of the NIC's on the DL360 are all connected to each other on a private network.

The DL360 has Windows 2003 installed, but I haven't installed any of the HP management software yet.

I'm not sure what I should do at this point. I want to get the hardware all powered up and configured, and update any firmware (if necessary).

What I was thinking I would do next is power up the blade chassis (with no blades) and use the built in LCD to set the IP of the onboard admin cards. Then install the software on the DL360 and let it find the OA cards. Then use the installed software to configure the chassis and interconnect modules. After the chassis is configured, install the blades and install the O/S on them.

Does this sound like the correct sequence?
On the HP software, what modules should I install? It looks like the disk contains a lot more things than what we'll need.

I'm working through the HP documentation, and searching the net, but any suggestions would be appreciated.

Mike O.
17 REPLIES 17
Adrian Clint
Honored Contributor

Re: Advice on setting up new blade systems

Mike,

Setup the OA IP on the front.
Connect up the OA's to the network.
Login to the OA's and run thru the first time wizard and make sure you setup the EBIPA for all the blades and interconnects.
Download the Blade Firmware 1.41 Update CD. (On the drivers page)
Connect to each iLO on the blades and using virtual media mount the CD or an ISO and let each blade boot up and then let it update the firmware.
After they are all done.
Download the 2.32 OA firmware and upgrade that on the OAs.
Download the Virtual Connect firmware 1.34 and the firmware update utility. Update the firmware using the CLI.
Connect up any stacking cables between the left hand VCs and between the right hand VCs.
For the DL360.
Download the Microsoft WAIK DVD
Install the ICE DVD and let it run the ICE advisor program. Get all the pre-reqs it wants done.
And that lot should take you at least a day. So post where you are at when finished.
I am doing exactly the same today.
The Brit
Honored Contributor

Re: Advice on setting up new blade systems

Mike,
first thing to think about is where to put your Mezz cards and VC Modules. (As you will learn, this is critical)

You should put your 4-Port NIC card on MEZZ 2, since this is the only slot that can map all 4 NIC Ports. In this case the ports map to IC modules 5, 6, 7 and 8, so this is where you must put your 4 ENet VC modules.

Put your dual port FC HBA on Mezz 1, this maps to IC 3 and IC 4, so that is where your FC VC modules should go.

Now remember that ALL of your servers must have the Mezz cards configured the same since the connections from the servers to the IC modules is Hardwired and cannot be modified.

Hope this helps

Dave
Mike O.
Regular Advisor

Re: Advice on setting up new blade systems


We not using 4 port mezzanine cards, we're using a 2 port Ethernet card. The first two NIC's on the motherboard will be interconnect bay 1 and 2, the 2 port NIC on Mezzanine card 1 will be interconnect 3 and 4, and the two port Fiber Channel (on mezz 2) will go to interconnect 5 & 6 (the VC FC modules). Interconnect 7 and 8 are empty.

The Brit
Honored Contributor

Re: Advice on setting up new blade systems

Sorry about that Mike,
I was the one who got confused. I was thinking that the 4 eNet modules were additional to IC bays 1 & 2.

The reason I suggested Mezz2 for your NIC, (2-port or 4-port) is that you mentioned that you are going to use this for VMWare.

I'm not a VMWare guy, however I work closely with a guy who is, and his constant comment is that there aren't enough NIC's for what he needs.

The point to consider is that Mezz1, by definition, will never map more than 2 ports, whereas Mezz 2 has the capability of mapping 4 ports. The current family of FC cards do not include any 4-port cards that I am aware of. Therefore putting the FC card on Mezz1 loses you nothing, whereas having the Ethernet card on Mezz2 allows you the ability to upgrade to a 4-port card in the future, without having to disturb your FC setup, all you will have to do is add 2 extra ENet modules in 7 and 8.

(this discussion only applies to halfheight blades, however)

Anyway, just a thought

Dave.
Adrian Clint
Honored Contributor

Re: Advice on setting up new blade systems

Mike,

still .... you want to connect the SAN modules to interconnect bays 3 & 4 (HBA in lowest mezzanine) and the 2nd pair of network interconnects in interconnects 5 & 6.
This will allow you to expand to 6 NICs on a half height server without moving the SAN interconnects.

Best practice... best for 99% of installations.
Mike O.
Regular Advisor

Re: Advice on setting up new blade systems

Thanks, that's a good idea. I didn't realize that you could access interconnect 7&8 with any half height server, I thought you needed full height blades to access those ports.

When I first saw your comment, I didn't understand it, but now I realize that if I put a quad port mezzanine card in connetor two, I can access interconnect port 7 & 8.

Steven Clementi
Honored Contributor

Re: Advice on setting up new blade systems

Mike:

Are you planning on being able to vMotion any of your VM's?

If so, you may want to look at getting the 4port mezz card now or at least plan for it in the future.

Your vMotion network should be separate from your console and public data networks. You would be able to do that now, but end up with no redundancy for both networks.

Now, you can use your console/management network for vMotion as well, but Best Practice calls for them being separate.

Just an idea to think about before you move into your project full force.


Steven
Steven Clementi
HP Master ASE, Storage, Servers, and Clustering
MCSE (NT 4.0, W2K, W2K3)
VCP (ESX2, Vi3, vSphere4, vSphere5, vSphere 6.x)
RHCE
NPP3 (Nutanix Platform Professional)
Mike O.
Regular Advisor

Re: Advice on setting up new blade systems

Yes, we're going to be using VMotion, unfortunately we're pretty much locked on the hardware at this time (very little additional funding).

We're working with a consultant on the VMWare installation and configuration; they're the ones that recommended the current NIC config (two on board and two port mezz card). From what I understand, they're going to be using 2 NIC for production, and 2 for VMWare. I'm not sure if the idea is one each for VMotion and one for VM management, or set the two NIC's up as a redundant pair and run both VM functions through them.

Initially we only have 10 blades (5 in each chassis), and we're looking at about 130 virtual machines total, so it probably won't be a major performance hit at this time. By the time we need to expand with additional blades we can look at upgrading the existing two port cards (I did move the interconnects so we can take advantage of the 4 port modules).
Mike O.
Regular Advisor

Re: Advice on setting up new blade systems

Now I'm having a problem (or just missing something) on the connection for the OA.

Our network group set up a private subnet on the switches in the 10.5.100.x subnet.

I set up the IP for each OA module using static IP in the 10.5.100.x subnet. I then set the 2nd NIC on the DL360 to the same 10.5.100.x subnet. I was able to ping each OA module and get <4ms response.

I then opened up IE and connected to one of the OA modules. I got the "certificate error" message, then told it to connect anyway. At that point, IE waited a while then timed out. Doing a PING on the OA address either timed out or returned response times in the thousands of ms.

I dug into the documentation some more and found the section on "temporarily" connecting a PC to the chassis uplink port on the 169.254.x.x. subnet. I went ahead and did a direct connection from the chassis to the 2nd NIC on the DL360, set it to the 169.254.1.254 address, and was able to access the system using IE to go to the OA service address the OA credentials.

What am I missing on the connections? If I want to use the DL360 server to manage the chassis and the devices in it, do I connect to the port(s) on the OA, or through the uplink port?

I really wish our purchasing group had got the training set up first...

Mike O.