- Community Home
- >
- Servers and Operating Systems
- >
- HPE BladeSystem
- >
- BladeSystem - General
- >
- Re: Can't get 10Gig networking working with BL860c...
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-23-2019 06:25 AM
10-23-2019 06:25 AM
I have recently purchased two BL860c i2 blades and a BL860c i1 blade in a c3000 chassis. These are Itanium and are running HPUX 11i v3. The two BL860c i2 blades have an embeded 10Gig network card that shows 16 NICs. The c3000 has a network passthru module that provides 4 ports to each blade. I have a 1gig network module in Mezz2 and that goes to a 2nd passthru module.
I am setting this system up for the first time and I'm running through the hardware makeing sure that everything works. The SAN works, the 1gig network works. But when I configure the 10Gig networking, It does not work.
The first 10Gig port is on lan0. It shows UP when I plug it into a switch, and down when I remove the connection. So I have confirmed that I am in the right port. When I have blade1 and blade2 10Gig connections plugged into a flat Nexus switch, they can't ping each other. I then plugged both of them into a Cisco 6509 just incase they didn't like the nexus switch. It still didn't work. I have two Solaris system with 10Gig NICs that I used as a control group. I plugged them into the same switches. They can ping each other, but not the BL860c i2 systems.
I then took a small fc cable and connected the two BL860c i2 servers to each other without a switch between them. This worked, they could ping each other. I then connected a Solaris server directly to one of the BL860c i2 servers. No ping.
So I have these two BL860c servers that can only ping when they are conencted directly to each other, but fail when connected to any switch or directly to other servers. I am hoping that there is some 10G setting that I do not know about that I need to use to get this working. I just used defaults and smh to configure the ports.
# ioscan -funC lan
Class I H/W Path Driver S/W State H/W Type Description
========================================================================
lan 0 0/0/0/3/0/0/0 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 1 0/0/0/3/0/0/1 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 2 0/0/0/3/0/0/2 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 3 0/0/0/3/0/0/3 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 4 0/0/0/3/0/0/4 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 5 0/0/0/3/0/0/5 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 6 0/0/0/3/0/0/6 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 7 0/0/0/3/0/0/7 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 8 0/0/0/4/0/0/0 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 9 0/0/0/4/0/0/1 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 10 0/0/0/4/0/0/2 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 11 0/0/0/4/0/0/3 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 12 0/0/0/4/0/0/4 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 13 0/0/0/4/0/0/5 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 14 0/0/0/4/0/0/6 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 15 0/0/0/4/0/0/7 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in FLEX-10
lan 16 0/0/0/5/0/0/0/2/0/0/0 iether CLAIMED INTERFACE HP 447881-001 PCIe 4-port 1000Mb/s Mezzanine Adapter
lan 17 0/0/0/5/0/0/0/2/0/0/1 iether CLAIMED INTERFACE HP 447881-001 PCIe 4-port 1000Mb/s Mezzanine Adapter
lan 18 0/0/0/5/0/0/0/4/0/0/0 iether CLAIMED INTERFACE HP 447881-001 PCIe 4-port 1000Mb/s Mezzanine Adapter
lan 19 0/0/0/5/0/0/0/4/0/0/1 iether CLAIMED INTERFACE HP 447881-001 PCIe 4-port 1000Mb/s Mezzanine Adapter
# lanscan
Hardware Station Crd Hdw Net-Interface NM MAC HP-DLPI DLPI
Path Address In# State NamePPA ID Type Support Mjr#
0/0/0/3/0/0/0 0x643150008850 0 UP lan0 snap0 1 ETHER Yes 119
0/0/0/3/0/0/1 0x643150008854 1 UP lan1 snap1 2 ETHER Yes 119
0/0/0/3/0/0/2 0x643150008851 2 UP lan2 snap2 3 ETHER Yes 119
0/0/0/3/0/0/3 0x643150008855 3 UP lan3 snap3 4 ETHER Yes 119
0/0/0/3/0/0/4 0x643150008852 4 UP lan4 snap4 5 ETHER Yes 119
0/0/0/3/0/0/5 0x643150008856 5 UP lan5 snap5 6 ETHER Yes 119
0/0/0/3/0/0/6 0x643150008853 6 UP lan6 snap6 7 ETHER Yes 119
0/0/0/3/0/0/7 0x643150008857 7 UP lan7 snap7 8 ETHER Yes 119
0/0/0/4/0/0/0 0x643150008858 8 UP lan8 snap8 9 ETHER Yes 119
0/0/0/4/0/0/1 0x64315000885C 9 UP lan9 snap9 10 ETHER Yes 119
0/0/0/4/0/0/2 0x643150008859 10 UP lan10 snap10 11 ETHER Yes 119
0/0/0/4/0/0/3 0x64315000885D 11 UP lan11 snap11 12 ETHER Yes 119
0/0/0/4/0/0/4 0x64315000885A 12 UP lan12 snap12 13 ETHER Yes 119
0/0/0/4/0/0/5 0x64315000885E 13 UP lan13 snap13 14 ETHER Yes 119
0/0/0/4/0/0/6 0x64315000885B 14 UP lan14 snap14 15 ETHER Yes 119
0/0/0/4/0/0/7 0x64315000885F 15 UP lan15 snap15 16 ETHER Yes 119
0/0/0/5/0/0/0/2/0/0/0 0x0025B3B44464 16 UP lan16 snap16 17 ETHER Yes 119
0/0/0/5/0/0/0/2/0/0/1 0x0025B3B44465 17 UP lan17 snap17 18 ETHER Yes 119
0/0/0/5/0/0/0/4/0/0/0 0x0025B3B44466 18 UP lan18 snap18 19 ETHER Yes 119
0/0/0/5/0/0/0/4/0/0/1 0x0025B3B44467 19 UP lan19 snap19 20 ETHER Yes 119
LinkAgg0 0x000000000000 900 DOWN lan900 snap900 22 ETHER Yes 119
LinkAgg1 0x000000000000 901 DOWN lan901 snap901 23 ETHER Yes 119
LinkAgg2 0x000000000000 902 DOWN lan902 snap902 24 ETHER Yes 119
LinkAgg3 0x000000000000 903 DOWN lan903 snap903 25 ETHER Yes 119
LinkAgg4 0x000000000000 904 DOWN lan904 snap904 26 ETHER Yes 119
# smh
SMH-> Networking and Communications-> Network Interfaces Configuration-> Network Interface Cards
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Interface Name Subsystem Hardware Path Interface State Interface Type IPv4 Address IPv6 Address
-------------------------------------------------------------------------------------------------------------------------------------------------------------
lan0 iexgbe 0/0/0/3/0/0/0 up 10GBASE-KR 192.168.5.26 Not Configured
lan1 iexgbe 0/0/0/3/0/0/1 up 10GBASE-KR 192.168.4.26 Not Configured
lan2 iexgbe 0/0/0/3/0/0/2 down 10GBASE-KR Not Configured Not Configured
lan3 iexgbe 0/0/0/3/0/0/3 down 10GBASE-KR Not Configured Not Configured
lan4 iexgbe 0/0/0/3/0/0/4 down 10GBASE-KR Not Configured Not Configured
lan5 iexgbe 0/0/0/3/0/0/5 up 10GBASE-KR Not Configured Not Configured
lan6 iexgbe 0/0/0/3/0/0/6 down 10GBASE-KR Not Configured Not Configured
lan7 iexgbe 0/0/0/3/0/0/7 down 10GBASE-KR Not Configured Not Configured
lan8 iexgbe 0/0/0/4/0/0/0 up 10GBASE-KR Not Configured Not Configured
lan9 iexgbe 0/0/0/4/0/0/1 up 10GBASE-KR Not Configured Not Configured
lan10 iexgbe 0/0/0/4/0/0/2 down 10GBASE-KR Not Configured Not Configured
lan11 iexgbe 0/0/0/4/0/0/3 down 10GBASE-KR Not Configured Not Configured
lan12 iexgbe 0/0/0/4/0/0/4 up 10GBASE-KR Not Configured Not Configured
lan13 iexgbe 0/0/0/4/0/0/5 down 10GBASE-KR Not Configured Not Configured
lan14 iexgbe 0/0/0/4/0/0/6 down 10GBASE-KR Not Configured Not Configured
lan15 iexgbe 0/0/0/4/0/0/7 down 10GBASE-KR Not Configured Not Configured
lan16 iether 0/0/0/5/0/0/0/2/0/0/0 up 1000Mb/s 192.168.16.180 Not Configured
lan17 iether 0/0/0/5/0/0/0/2/0/0/1 down 1000Mb/s Not Configured Not Configured
lan18 iether 0/0/0/5/0/0/0/4/0/0/0 down 1000Mb/s Not Configured Not Configured
lan19 iether 0/0/0/5/0/0/0/4/0/0/1 down 1000Mb/s Not Configured Not Configured
-------------------------------------------------------------------------------------------------------------------------------------------------------------
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-23-2019 02:48 PM
10-23-2019 02:48 PM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
I had the network guys check the 6509 Cisco switch and they do not see MAC addresses for my HP systems, but they do for the Solaris systems. So that tells me I have a layer two issue. Hmmm..
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-23-2019 08:42 PM
10-23-2019 08:42 PM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
> [...] 192.168.5.26 [...]
> [...] 192.168.4.26 [...]
> [...] 192.168.16.180 [...]
What's the subnet mask for these interfaces? Around here (/24),
each would be on its own subnet.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2019 12:08 AM - edited 10-24-2019 12:08 AM
10-24-2019 12:08 AM - edited 10-24-2019 12:08 AM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
Maybe the blade was left in virtual connect mode, but you don't have VC.
Worth to try a "clear vc mode", see
Hope this helps!
Regards
Torsten.
__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.
__________________________________________________
No support by private messages. Please ask the forum!
If you feel this was helpful please click the KUDOS! thumb below!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2019 06:36 AM
10-24-2019 06:36 AM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
192.168.5.x/24
192.168.4.x/24
192.168.16.x/22
The first two are just "test" networks that I'm using to verify 10Gig NIC functionality. The 192.168.16.x/22 network is a lab network on the 1gig interface. That NIC is working just fine. Just the 10gigs that are giving me issues.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2019 06:56 AM
10-24-2019 06:56 AM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
Thank you for the reply. I went into the OA and I found that the VC mode is set to disabled. So I am guessing that this isn't it.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2019 07:08 AM - edited 10-24-2019 07:09 AM
10-24-2019 07:08 AM - edited 10-24-2019 07:09 AM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
A single blade in non-VC mode looks like this (without mezzanine):
# ioscan -Clan
H/W Path Class Description
=================================================================
0/0/0/3/0/0/0 lan HP PCIe 2-p 10GbE Built-in
0/0/0/3/0/0/1 lan HP PCIe 2-p 10GbE Built-in
0/0/0/4/0/0/0 lan HP PCIe 2-p 10GbE Built-in
0/0/0/4/0/0/1 lan HP PCIe 2-p 10GbE Built-in
So I think the VC mode is still active for this blade.
I have seen the same issue before, now trying to remember how we solved this ...
Hope this helps!
Regards
Torsten.
__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.
__________________________________________________
No support by private messages. Please ask the forum!
If you feel this was helpful please click the KUDOS! thumb below!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2019 07:19 AM
10-24-2019 07:19 AM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
Try this.
Do a ssh session to the OA, then connect to the MP/ILO of the blade (connect server <bay#>)
switch to CM menu.
Do
CM:hpiLO-> CILON -iloKeepLan
go back to OA.
the do
OA> reset server <bay number>
boot up and check "ioscan -fnClan" again.
Hope this helps!
Regards
Torsten.
__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.
__________________________________________________
No support by private messages. Please ask the forum!
If you feel this was helpful please click the KUDOS! thumb below!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2019 12:39 PM
10-24-2019 12:39 PM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
I reset the ILO memory. I hate to say it, but no change.
[hp26ilo] CM:hpiLO-> CILON -iloKeepLan
CILON -iloKeepLan
WARNING: This is not an HP supported command and it is intended for internal
use only. Execution of this command is not recommended and can lead to
unpredictable results.
The following command options are available:
"CILON -ilo" - Clears MP NVRAM and reboots MP.
"CILON -iloKeepLan" - Clears MP NVRAM, except for MP LAN settings,
and reboots MP.
"CILON -sfw" - Clears SFW NVRAM and reboots MP.
"CILON -all default" - Clears both MP and SFW NVRAM and reboots MP.
This command will clear MP NVRAM and reboot MP.
All MP parameters will be reset to their default values.
MP LAN Settings will not be cleared.
Are you REALLY sure you want to continue? (Y/[N]) y
y
-> MP NVRAM has been cleared.
MP is now being reset...
-> Command successful.
[hp26ilo] CM:hpiLO->
Checked the c3000
hp25> show server list
Bay iLO Name iLO IP Address Status Power UID Partner
--- ----------------------------- --------------- -------- ------- --- -------
1 hp26ilo 192.168.16.177 OK Off Off
2 hp27ilo 192.168.16.178 OK On Off
3 hp28ilo 192.168.16.179 OK On Off
4 [Absent]
5 [Subsumed]
6 [Subsumed]
7 [Subsumed]
8 [Absent]
Totals: 3 server blades installed, 2 powered on.
hp25> reset server 1
WARNING: Resetting the server trips its E-Fuse. This causes all power to be momentarily removed from the server. This command should only be used when physical access to the server is unavailable, and the server must be removed and
reinserted.
Any disk operations on direct attached storage devices will be affected. I/O
will be interrupted on any direct attached I/O devices.
Entering anything other than 'YES' will result in the command not executing.
Do you want to continue ? yes
Successfully reset the E-Fuse for device bay 1.
hp25> show server list
Bay iLO Name iLO IP Address Status Power UID Partner
--- ----------------------------- --------------- -------- ------- --- -------
1 [Unknown] N/A OK Unknown ?
2 hp27ilo 192.168.16.178 OK On Off
3 hp28ilo 192.168.16.179 OK On Off
4 [Absent]
5 [Absent]
6 [Subsumed]
7 [Subsumed]
8 [Absent]
Totals: 3 server blades installed, 2 powered on.
hp25>
<Eventually the server came back up>
hp25> show server list
Bay iLO Name iLO IP Address Status Power UID Partner
--- ----------------------------- --------------- -------- ------- --- -------
1 hp26ilo 192.168.16.177 OK Off Off
2 hp27ilo 192.168.16.178 OK On Off
3 hp28ilo 192.168.16.179 OK On Off
4 [Absent]
5 [Subsumed]
6 [Subsumed]
7 [Subsumed]
8 [Absent]
Totals: 3 server blades installed, 2 powered on.
hp25>
It did wipe out my ILO settings. I had to rebuild the userid and when I booted, it didn't see my local hard disks. I had to go through the efi menu to boot. This system was installed without the passthrough modules installed. I'm going to reinstall the OS and see if that helps.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-24-2019 06:43 PM
10-24-2019 06:43 PM
Re: Can't get 10Gig networking working with BL860c i2 blade in a c3000 chassis
Hello
Since there is no virtual connect in the ensclosure, please ensure there is no setting/vLANs configured in the upstream switches,. Also when you mentioned no ping., how switch saw these Server NICs , Does the switch saw the MAC of these NICs and Speed patches with the NIC(Auto0Negotiation ON) ? By default for IEXGBE the Speed is set to Auto_Negotiation ON (which is a read-only attribute) .
I am an HPE employee