BladeSystem - General
cancel
Showing results for 
Search instead for 
Did you mean: 

C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

SOLVED
Go to solution
Schwager
Advisor

C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Hi,

we own a C7000 with one Flex-10 10Gb in Connection-Bay 1. Also, we have some B460C G5.

Is it possible to

- DISABLE the LOM-Nic's (embedded 1GBit-NIC's) from the B460CG5
- and add a mezzanine card NC532m in M1 or M2 (Dual Port Flex-10 10GbE Multifunction BL)
- and connect it to interconnectionbay 1 (Flex-10 10Gb ) ??

The problem - i would like to use the 10GBit from my switch with the "old" G5 Blades - but we cannot buy a new 10GBit-Switch for interconnection bay nr. 3...

So, is there a way to manage this setup ?


regards
Danny
12 REPLIES
rick jones
Honored Contributor
Solution

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

The backplane in the c7000 chassis is passive. The mezz slots will not go to Bay 1. I forget off the top of my head which mezz slot goes to which I/O bay, but you could, in theory, since you have no need for 10G downlinks to LOMs, put say a 1/10 VC Enet module in Bay 1, and then shift the Flex-10 module to the bay corresponding to the mezzanine slot you pick.

But you will need to buy something.

IIRC, if you want to drive both ports of the NC532m, you will need two VC Flex-10 modules, or two Procurve 6120's. The VC Flex-10 does Flex-10 but does not "switch" between the uplinks, the ProCurve 6120's don't do VC/Flex-10 but will switch among the uplink ports.
there is no rest for the wicked yet the virtuous have no pillows
The Brit
Honored Contributor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Danny,

To back up Rick's response above, the simple answer to your basic question, "can Mezz 1 map to Bay 1?", is NO!!

The ports on ANY type of card placed in Mezz 1 on a half-height blade, in a c7000 enclosure, will (and can only) map to interconnect bays 3 and 4.

Mezz Port 2 will accept either a dual or quad card, in which case, ports 1 & 2 from Mezz2 map to IC bays 5 & 6, and ports 3 & 4 from Mezz 2 will map to IC bays 7 & 8.

These mappings are HARD, and are not negotiable.

As Rick pointed out, you could move your Flex10 from Bay 1 to Bay 3, and place a 1/10Gb VC ENet module in Bay 1. (You can probably pick up refurg 1/10Gb modules for <$1500 these days). If you do this, then you will be able to configure flexNICs on the Mezz 1 card.

You should also note (although you are probably aware of this), with only one Flex10 module (in either Bay 1 or Bay 3) you will only get one (1) mezz port active.

Also, the swap of VC modules is going to require all of your blades shutdown while it takes place.

HTH

Dave.
Schwager
Advisor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Hi Dave,
hi Rick,

thanks for your answers. We will insert the 10GBit-Switch to Bay3 AND put in some NC532m to mezzanie Slot M1.

So, we can reuse our 6x 460cG5 using the max 4 "phy" NICs (supported by NC532m) - sure - not redundante.

This config (Bay1+2: empty, Bay3: 10GBit-Switch, Bay5+6: FC-Switch) is not "supported", but i think, it sould work (http://h18004.www1.hp.com/products/quickspecs/13127_div/13127_div.html)


Thx very much
best regards
Danny

The Brit
Honored Contributor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Danny,
This WILL NOT work, at least not with VC.

There MUST be a VC ENet module of some flavour in (at least) Bay 1, (and preferably also in Bay 2). The reason is that Virtual Connect Manager (VCM) ONLY runs from Bay 1, (or Bay 2 if Bay 1 fails). VCM cannot run from any other bay.

Dave.
Schwager
Advisor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Hi Dave,

thanks for this info !

So, is the following possible:

Bay1: 10GBit-IP-Switch
Bay2: empty

Bay3: GbE2-IP-Switch
Bay4: empty

Bay5+6: VC-FC Modul (409513-b21)


regards
Danny
Schwager
Advisor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Added detailed module name -
is the following possible:

Bay1: Flex-10 10GB VC Ethernet Modul
Bay2: empty

Bay3: BLc GbE2 Ethernet Switch
Bay4: empty

Bay5+6: VC-FC Modul (409513-b21)
The Brit
Honored Contributor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Danny,
Putting the Flex10 in Bay 1 will work, and will allow you to configure FlexNICs on any blade which has 10Gb LOM NICs.

There is another thing you will need to consider however, and that is connectivity to the external network (i.e. UpLinks). If your external network is 10Gb (lucky you) then you can connect using 10Gb CX4 cables, or the 10Gb SFP+ ports. However, if your external network is 1Gb or less, then you will have to purchase the 1Gb SFP Options kit (one for each uplink port you want to use). It could work out cheaper to get a refurb 1/10Gb VC ENet module (~$1200-$1500) and stack it with your Flex10. You can then run your uplinks from there at 1G (or combine them for upto 8Gb) for no additional cost.

The "BLc GbE2 Ethernet Switch" is not a VC module and therefore cannot be combined with the Flex10. I am not really familiar with using non-VC modules in these enclosures, so I cant comment on the usefulness of having it in the enclosure.

Anyway, with Flex10 in Bay 1, and VC FC modules in 5 & 6, you will be able to use VC profiles on your blades.

I guess that the significant point is that non-VC modules, such as the GbE2, are not detected by the VCManager, and therefore VCM knows nothing about what blade NIC ports are connected to the GbE2 (if any) or about any connections from the GbE2, to the main external network.

It you understand that and are comfortable with the limitations, then you are good to go.

Dave.
rick jones
Honored Contributor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Do do "Virutal Connect" things, and by extention then "Flex-10" things (subdividing a 10 Gigabit port into up to four slower "flex-nics") there must be a VC module of some sort in Bay 1. That is the only place (well, that and Bay 2 I believe) where the Virtual Connect manager software will run. Without a VC module in one of those two bays, there is no way to configure VC modules in the other bays.

If you do not have need of VC/Flex-10 functionality, and all you want is basic 10 Gigabit connectivity to the Blades, I would suggest using the ProCurve 6120 Blade Switch.

If you don't really need any 10G switch capability in the chassis, and are content running cables/transceivers for each 10 Gigabit port in your blade(s) there is also a 10 Gigabit Ethernet "pass through" module.
there is no rest for the wicked yet the virtuous have no pillows
rick jones
Honored Contributor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

The NC532m will indeed talk through a GbE2c (in the proper bay :) - at least that is what I have been told, I've not done it myself. However, given the chip in the NC532m is the same chip as the NC532i which is the LOM on at least some G6 blades, and the LOM on those G6 Blades can indeed speak through a GbE2c I'm quite confident it will actually work.

However, it will only speak at 1 Gbit/s and will have no Flex-10 functionality available.
there is no rest for the wicked yet the virtuous have no pillows
Schwager
Advisor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

So,

to make the setup more "supported" i think i should follow Dave suggestion:

Bay1: 1/10GBit VC-Eth Switch (renew, refund)
Bay2: empty

Bay3: 10GBit VC-Eth-Switch
Bay4: empty

Bay5: VC FC-Switch
Bay6: VC FC-Switch

a) there's a VC on Bay1, so i can connect all for VC-Switches together to the management.

b) i can plugin some NC532m (@M1 == Bay3) to my old 460cB5 and can use the 10GBit splitting them to 3 Nics (1GBbit/1GBit/8GBit)

c) i can plugin some NC532m (@M1 == Bay3) to my new 460cG6, so i can also use the 10GBit splitting

d) i have not to buy up to 8 SFP (RG45, copper) for connecting the c7000 to external eth-network (Hallo Dave ;-)

e) I can later (if money returns ;-) buy the redundant modules for Bay2+4 and configure the both vc-eth switches for HA/Failover.

Do you confirm a) to e) ? I think, you guys tell me the right way.

Yeh ... so, our nbd-daemon@solaris@b460G6 can supply the kvm@linux@460cG5 on 8GBit network - and you can try linux online on opensource-online.org faster than now (-:

One more question: Do i have to interconnect the both vc-eth modules (CX4-cable) together ? Like shown in http://h18004.www1.hp.com/products/quickspecs/13127_div/13127_div.html ? The VC-FC module must not be interconnected to anything, or ?

Another question: Could i move the the VC-FC-modules to Bay1+2 and the VC-10GBit-Eth module to Bay3 - so there's a a VC module (ok, it's FC not eth) in Bay 1. May this setup could make an additional 1/10 VC-Eth module not necessary ... this is yust for my understanding - refer to Q4 in http://h18000.www1.hp.com/products/blades/components/ethernet/10-10gb-f/questionsanswers.html


Danny

The Brit
Honored Contributor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

OK Danny,

Based on the configuration you gave in the previous post:

A. Yes! run a 0.5m CX4 cable between IC bays 1 & 3. (This is a MUST, the CX4 cable provides the path from your M1 ports (mapped to Bay 3) and your external uplinks (from Bay 1 to External Network).

B & C. Both Correct.

D. SFP's not required if you run uplinks from Bay 1 (1/10Gb has 8x1Gb ports which can be combined into single or multiple Uplink Sets - (Sum of Uplinksets <= 8Gb)). If you want additional uplinks in future, from Flex10 (in Bay 3) you will need to purchase the 1Gb SFP Option.

E. Correct.

Q1. See answer A above.

Q2. VC FC Modules CANNOT be in bays 1 or 2. Bay 1 and 2 can only be populated with a VC Ethernet Module in order for VCM to run.

Dave.
Schwager
Advisor

Re: C7000, disable LOM-Nic's, remapping Mezzanine to BL460cG5 and Flex-10 10Gb

Hi Dave, hi Rick,

thanks very much for your help. Today, we ordered 1xVC-Eth10GBit and 2xVC-Eth-1GBit (redundant bay1+2, all renew) followed your suggestions.

I'm curious about configure the VC-Stuff - never did it before (-:

Thx
Danny