Switch receiving packets but not transmitting any

[guyHalestorm]

Hey everyone!  I've been Googling for hours trying to figure this one out and I'm not coming up with anything.

I've got a ProCurve 3500yl-48 with an X2 module (J8346A 10GbE-SR, official HP) connected to a HP server with the HP Mellanox card and a FTLX8571D3BCL-H2 HP 10GbE-SR SFP.  I'm getting a link but no network connectivity.  Switch is flat with no configuration except for an IP address.  The switch diagnostics are showing that I'm receiving packets but the switch isn't transmitting any.  I have tried different fiber with no change.  I even flipped the polarity on both ends of a patch cable to see if it would error the other way, but it's consistent - it sees packets from the server but isn't sending any.  Anyone have an idea if there's some config option I'm missing, or do I just have a bad X2 module, or...?

coresw# show interfaces transceiver a4 detail

Transceiver in A4
   Interface Index    : 52
   Type               : 10GbE-SR
   Model              : J8436A
   Connector Type     : SC
   Wavelength         : 850nm
   Transfer Distance  : 300m (MM generic)
   Diagnostic Support : DOM
   Serial Number      : DE818YS07Q

 Status
   Temperature : 44.156C
   Voltage     : 0.0000V
   Tx Bias     : 7.840mA
   Tx Power    : 0.6945mW, -1.583dBm
   Rx Power    : 0.5742mW, -2.409dBm

  Time Stamp    : Sat Jan 14 01:59:57 2017

Any help would be appreciated.  Thanks!

[parnassus]

Hello, apart from detailing the status of:

• Which model of Mellanox NIC (Adapter Card) are you using? as example: is it a single/dual 10G port Mellanox ConnectX-3 EN or what else exactly?
• Device Driver (of Mellanox NIC) for the specific used host OS on your Server, if not yet updated.
• Device Firmware (of Mellanox NIC), if any and if not yet updated.
• Is the HP ProCurve 3500 Switch Firmware (Is your Switch exactly a J9472A or what?) updated?

Have you already verified about a possible physical port duplex mismatch (in case of a non working/not set auto-negotiation on one end or on both ends)?

What are involved Server NIC port settings actually?

Does a simple ping (from server to switch and from switch to server) work flawlessly?

Doesn't a file transfer (from server to switch and from switch to server) work?

If you (ex)change the X2 slot into which the HP X131 10G X2 SC SR Transceiver (J8436A) was installed, does the issue follow the X2 slot or not?

[guyHalestorm]

Thanks for the reply man!

• HP-branded Mellanox ConnectX-2 EN (part 671798-001)
• Device driver is the latest HP driver sourced from here: http://h20566.www2.hpe.com/hpsc/swd/public/detail?sp4ts.oid=3884088&swItemId=MTX_4d7b4f9f795b49d6ac926d4a4e&swEnvOid=4168
• OS is Windows 2012 R2, however I first tried it with Ubuntu 16.04 LTS and had the exact same problem.  I decided to try Windows so I could test the official HP drivers.
• Device firmware is HP customized and cannot be updated with the firmware provided on Mellanox's website.  I've found HP firmware "tools" but no firmware update available.  Mellanox firmware won't take because it's an HP part.
• Switch is precisely a J8693A ProCurve 3500yl-48G-PWR with the J8649A yl module installed.  Running the latest firmware K.16.02.0014.
• I cannot even find a setting in the switch or the NIC to change the duplex setting.  On the switch side the only option you can change is flow control, which I've tried both enabled and disabled on both switch and server sides.  Mode on the switch is Auto, if you change it to any of the other options it says "Invalid value."  Server size doesn't have an option to change port mode.  I've uploaded a copy of the system snapshot report from Mellanox's utility that provides a lot of detail: https://1drv.ms/u/s!AiesdJ69JpdSgpM4Mid8Q0wLOOa_sA
• Ping does not work, either from server to switch or switch to server.  DHCP does not grab an IP, and if I set a static IP I still can't ping.
• File transfers obviously do not work.
• I have used the X2 in both slots A1 and A4 and have had the same issue.

I also realize I didn't share my switch config, so:

coresw# show config

Startup configuration: 15

; J8693A Configuration Editor; Created on release #K.16.02.0014
; Ver #0f:01.81.30.02.34.59.2c.6b.ff.f7.fc.7f.ff.3f.ef:33
hostname "coresw"
module 1 type j86yya
module 2 type j86xxa
module 3 type j8694a
no stack
ip default-gateway 192.168.50.1
interface A4
   flow-control
   exit
snmp-server community "public" unrestricted
snmp-server contact "Halestorm"
vlan 1
   name "DEFAULT_VLAN"
   untagged 1-48,A1-A4
   ip address 192.168.50.30 255.255.254.0
   exit
spanning-tree
device-profile name "default-ap-profile"
   cos 0
   exit
password manager

(I've tried both spanning tree enabled and disabled, it was just currently enabled.)

Other devices connected to the switch via copper work fine, btw, including this server.  

[parnassus]

Sorry , 10G always uses Full Duplex mode.

Provided that the HP 10GbE 2-port X2 / 2-port CX4 yl Module (J8694A) installed into the HPE 3500yl 48G PWR Intelligent Edge Switch (J8693A) correctly supports the HP X131 10G X2 SC SR Transceiver (J8436A) and the issue looks not bound to a specific J8694A port the culprit could be probably found on Server side.

Switch side:

1. Does the operating ambient temperature exceeded 40°C (104°F)? that's because X2 Transceivers should work with an ambient temperature below 40°C, see HP ProCurve Switch yl Module Installation Guide here on page 18 (OK, the X2 Transceiver measured temperature - about 44°C as reported - shouldn't represent the ambient one but just its own internal working temperature - X2 has a larger operating temperature range - so you should be basically OK, I think).
2. What is the Switch and Module LED(s) status?
3. Are there Switch/Modules LED(s) status patterns you are able to recognize on troubleshooting scenarios listed on the above manual (see page 14)?
4. Is the Fiber Optic cable used to interconnect both transceivers properly set (Tx on Rx and Rx on Tx: so crossed)?

Server side:

1. I noticed you have an HP ProLiant DL180 Gen6: neither Microsoft Windows Server 2012 R2 nor Ubuntu Linux 16.04 LTS are (and were) supported by that model (for that specific 6th generation), so a question: what's about testing a supported OS first...like, as example, CentOS 6.x (take the latest)?
2. HP ProLiant DL180 Gen6 System ROM looks the pretty old 2010.05.19 (A), latest released was the 2013.07.01 (A) release...so update to that version first.
3. NOTICE that a System ROM 2012.10.01 (A) fixed an issue regarding Mellanox NIC "Resolved an issue where the Mellanox 10Gb PCIe Single Port NIC would intermittently fail to train to the correct speed." so, to be really consistent with your tests, I recommend you to upgrade to latest available System ROM for your HP ProLiant DL180 Gen6 and use first a supported OS, then - eventually - try with Microsoft Windows Server 2012 R2.