BL860c with HP-UX and VC-Enet cannot ping gateway

[Horia Barbul_2]

Hello.

I encounter the following situation:

- C7000 enclosure with 12 BL460 G1 servers in various slots and one BL860c in slot 5 (with slot 13 "covered" and slots 6 and 14 empty)- alone in VC Domain.
Interconnect Modules:
- 2 X 1/10 VC-Enet Module in IM bays 1 and 2 ( FW 2.01)
- 2 X 4GB SAN Switch in IM bays 5 and 6


All BL460 have Windows 2003 Server installed and are production servers. Some are connected to the "main" network called DataCenter ( with uplinks on enc0:1:1 and enc0:2:1) and some to a separate HR-Net( uplink on enc0:1:3). The MAC Address Type is VC-Defined. Some of the BL460 have access to vdisks on an EVA8000 some not.

The issue is with the network connectivity of the BL860c. I just installed HP-UX B.11.23 on it ( June 2008, boot-from-SAN) and apparently all went well, BUT I cannot ping the gateway I set up for lan0. However I can ping bothways with the BL460 servers in the same enclosure and - of course - the BL460 can ping the gateway. Also - the network guys tell me that packets from "my" MAC (00-17-A4-77-00-10) get to the core switches. Some relevant outputs:

From VCM CLI:

CNE_Profile_Blade5 enc0:5 server BL860c OK

CNE_Profile_Blade5 1 DataCenter UseBIOS 00-17-A4-77-00-10 1Gb OK
CNE_Profile_Blade5 2 UseBIOS 00-17-A4-77-00-12 1Gb OK
CNE_Profile_Blade5 3 UseBIOS 00-17-A4-77-00-64 1Gb OK
CNE_Profile_Blade5 4 UseBIOS 00-17-A4-77-00-66 1Gb OK

===========================================================================
Name Status Shared VLAN ID Native Private Preferred
Uplink Set VLAN Speed
===========================================================================
DataCenter OK -- -- -- -- Disabled Disabled Auto
HR-Net OK -- -- -- -- Disabled Disabled Auto

========================================================================
Port Server I/O Adapter Type ID Profile
Module
========================================================================
1 enc0:5 1 Embedded Ethernet enc0:1:d13 CNE_Profile_Blade5
2 enc0:5 2 Embedded Ethernet enc0:2:d13 -- --
3 enc0:5 1 Embedded Ethernet enc0:1:d5 -- --
4 enc0:5 2 Embedded Ethernet enc0:2:d5 -- --

From HP-UX

# lanscan
Hardware Station Crd Hdw Net-Interface NM MAC HP-DLPI DLPI
Path Address In# State NamePPA ID Type Support Mjr#
0/2/2/0 0x00226436902A 2 UP lan2 snap2 1 ETHER Yes 119
0/1/1/0 0x0017A4770010 0 UP lan0 snap0 2 ETHER Yes 119
0/1/1/1 0x0017A4770012 1 UP lan1 snap1 3 ETHER Yes 119
0/2/2/1 0x00226436902B 3 UP lan3 snap3 4 ETHER Yes 119
#
# ioscan -funC lan
Class I H/W Path Driver S/W State H/W Type Description
===================================================================
lan 0 0/1/1/0 igelan CLAIMED INTERFACE HP PCI-X 1000Mbps Dual-port Built-in
lan 1 0/1/1/1 igelan CLAIMED INTERFACE HP PCI-X 1000Mbps Dual-port Built-in
lan 2 0/2/2/0 igelan CLAIMED INTERFACE HP PCI-X 1000Mbps Dual-port Built-in
lan 3 0/2/2/1 igelan CLAIMED INTERFACE HP PCI-X 1000Mbps Dual-port Built-in
#
# ifconfig lan0
lan0: flags=1843
inet 172.16.1.70 netmask ffffff00 broadcast 172.16.1.255

And now some totally puzzling outputs ( I would expect the <<#lanadmin -x 2>> to show the link up ( lan0 has NM ID 2) and
<<#lanadmin -x 4>> fails to attach to lan3) :

# lanadmin -x 1
The link is down. The speed and other link state information are undefined. The configuration specified for this card is:
Speed = 1000 Full-Duplex.
Autonegotiation = On.
#
# lanadmin -x 2
The link is down. The speed and other link state information are undefined. The configuration specified for this card is:
Speed = 1000 Full-Duplex.
Autonegotiation = On.
#
# lanadmin -x 3
The link is down. The speed and other link state information are undefined. The configuration specified for this card is:
Speed = 1000 Full-Duplex.
Autonegotiation = On.
#
# lanadmin -x 4
Unable to attach to interface 4
#

I would highly appreciate any troubleshooting ideeas.

Thank you

[Michael Leu]

What does lanadmin -x 0 say? (lanadmin need the PPA number and not the NM ID)

[Benoy Daniel]

Did you use any VLAN?. Make sure the VLAN is configured in your uplink.

[Horia Barbul_2]

Hello.

Michael, thank you for pointing the NM ID vs PPA error. I assume that if I'll get a "link up" at <> command most probably everything is OK on the HP-UX side. However I found an advisory stating that with LOMs on older BL860c under B.11.23 an issue may exist with lanadmin reporting a "link up" when in fact the link is down. I'll see if this is my case.

Benoy ,thank you for your suggestion. VLAN are used and - obviously - their configuration might be a reason for my issue, but they are not under "my control". I allready inquired with the Network Administrators and they stated that they have port based VLANs, hence all blades added in the enclosure connected to the DataCenter VC network should work fine. Anyway, if everything looks well on HP-UX side and I can ping all and every other blade in the enclosure, I guess I'll have to further look in the VLAN configuration.

Something else - I remeber that there is a Virtual Connect flag that ensures that the status of the uplink port is mirrored towards the OS in the lan status, but I cannot find it. I don't know either if it is a global setting or a profile/slot specific flag.

Thank you
Horia

[Horia Barbul_2]

I went again through the info I collected from the system and I found the following entries in /etc/rc.config.d/netconf:

------------------------------------
INTERFACE_NAME[0]="lan0"
IP_ADDRESS[0]="172.16.1.70"
SUBNET_MASK[0]="255.255.255.0"
BROADCAST_ADDRESS[0]=""
INTERFACE_STATE[0]=""
DHCP_ENABLE[0]=0
INTERFACE_MODULES[0]=""
-------------------------------------

ROUTE_DESTINATION[0]="default"
ROUTE_MASK[0]=""
ROUTE_GATEWAY[0]="172.16.1.1"
ROUTE_COUNT[0]="1"
ROUTE_ARGS[0]=""
-------------------------------------
I do remember answering with "1" at the hops number question during the installation process withot giving it much of a thought (:-0).

Now - ROUTE_COUNT is described as:
#
# ROUTE_COUNT:An integer that indicates whether the gateway is a
#remote interface (one) or the local interface (zero)
#or loopback interface (e.g., 127.*).
#
And , in my case , since the gateway is "local", it should be 0. I'll change it to see what happens. Any thoughts ?

[Horia Barbul_2]

"And , in my case , since the gateway is "local", it should be 0. I'll change it to see what happens"

Wrong. "Local" in this case means that the server is the gateway. ROUTE_COUNT[0] should be 1.

[Horia Barbul_2]

As expected lan0 is up under hp-ux:

# lanscan
Hardware Station Crd Hdw Net-Interface NM MAC HP-DLPI DLPI
Path Address In# State NamePPA ID Type Support Mjr#
0/2/2/0 0x00226436902A 2 UP lan2 snap2 1 ETHER Yes 119
0/1/1/0 0x0017A4770010 0 UP lan0 snap0 2 ETHER Yes 119
0/1/1/1 0x0017A4770012 1 UP lan1 snap1 3 ETHER Yes 119
0/2/2/1 0x00226436902B 3 UP lan3 snap3 4 ETHER Yes 119
#
# ifconfig lan0
lan0: flags=1843
inet 172.16.1.70 netmask ffffff00 broadcast 172.16.1.255

# lanadmin -x 0
Speed = 1000 Full-Duplex
Autonegociation = On.
#

[Horia Barbul_2]

If someone is interested - the issue was solved by assigning the profile network ports to a newly created VC network ( with two uplinks on the two VC-ENET modules).

[Horia Barbul_2]

See my last message.