Superdome 2 Cabling

[Nosaj]

Hello All,

I'm trying to connect four IOXs to a Superdome and IOX boxes 11 and 12 error out everytime I power on my partition of five i4 blades. 

I'm able to power the partition up with two IOX boxes, but as soon as I add a third it bangs out, indicts the IOX box, and the SEL shows fabric link failures. Usually IOX 11 has all four links failed and IOX 12 has three. 

The following is how I have everything cabled. 

IOX 9 cabled to XFM port 1 of all four XFM modules. Bay 1 ethernet plugged into GPSM 1 port 1 and bay 2 plugged into GPSM 2 port 1

IOX 10 cabled to XFM port 2 of all four XFM modules. Bay 1 ethernet plugged into GPSM 1 port 2 and bay 2 plugged into GPSM 2 port 2

IOX 11 cabled to XFM port 3 of all four XFM modules. Bay 1 ethernet plugged into GPSM 1 port 3 and bay 2 plugged into GPSM 2 port 3

IOX 12 cabled to XFM port 4 of all four XFM modules. Bay 1 ethernet plugged into GPSM 1 port 4 and bay 2 plugged into GPSM 2 port 4

I have been through two Domes, ten IOX boxes, several XFM modules, and lots of cables and the issue with IOXs 11 and 12 stick. 

Thanks for any advice in advance!

Jason

[jsm6]

Hi Jason,

The cabling to IOX and XFM looks correct.

IOX 9  - Port 1 of all XFMs
IOX 10 - Port 2 of all XFMs
IOX 11 - Port 3 of all XFMs
IOX 12 - Port 4 of all XFMs

1.How many enclosures (domes) are there?
2.Have you set the BIOX ID correctly on IOX's (no conflict between IOX's)?
3.Are you using the correct lengthy cables (99"-2m50) for IOX 11 and 12?
4.What exactly is the indictment for these IOX's (link failure or any other msg)
5.After reseating the failed links (both ends) and acquitting it,try to run "test fabric" (with npar powered off) again. Check if it still treports any cabling errors?
6.Did you try to replace the failed J-link with new cable (atleast one link) and see if error goes off for that link?

I would need atleast a "show all" and parstatus output  (capture in putty) from system to understand the setup.

Thanks

[rose45scott]

Hello Jason,

That sounds like a persistent and frustrating issue you're battling with! It's definitely unusual to have the problem follow across so much hardware. Given that you've swapped out so many components, let's think about some potential areas we haven't explicitly covered yet.

Here are a few thoughts and troubleshooting steps that come to mind:

1. Configuration and Firmware:

XFM Configuration: Have you verified the configuration of the XFM modules themselves? While the cabling seems straightforward, it might be worth checking if there are any specific configuration settings on the XFMs that could be influencing how they interact with IOXs 11 and 12. Perhaps there's a subtle setting related to the ports or the number of connected devices.
Firmware Consistency: Could there be a firmware mismatch between the Superdome, the XFM modules, and the IOX boxes? Even if the hardware is the same model, different firmware revisions can sometimes lead to unexpected incompatibilities. It might be worth checking the support matrix for your Superdome and ensuring all components are running compatible firmware versions.
Partition Configuration: Double-check the partition configuration for your five i4 blades. Is there anything specific in the partition profile that might be limiting or misconfiguring the resources allocated to the IOXs?
2. Backplane and Interconnects:

Midplane/Backplane Issues: While less likely given you've used two Domes, could there be an intermittent issue with the midplane or backplane of the Superdome in the slots where the XFM modules connected to IOXs 11 and 12 reside? This is a long shot, but since the problem seems tied to specific "logical" IOX numbers regardless of the physical box, it might be worth considering.
XFM Module Slots: Have you tried swapping the physical XFM modules around in their slots to see if the issue follows a particular slot? This could help rule out a problem with a specific XFM slot in the Superdome.
3. Power and Environment:

Power Stability: While it seems unlikely to affect specific IOXs, have you ruled out any power stability issues? Are all power supplies for the Superdome and the IOX boxes functioning correctly and providing stable power?
Environmental Factors: Are there any unusual environmental conditions (temperature, humidity, etc.) that might be affecting the IOX boxes in certain locations or logical assignments?
4. Diagnostic Tools and Logs:

Detailed SEL Analysis: You mentioned the SEL shows fabric link failures. Can you provide more detail about the specific error messages? Are there any other events logged around the time of the failures that might offer clues?
Superdome Diagnostic Tools: Have you utilized any specific diagnostic tools available on the Superdome to test the interconnects and the health of the XFM modules and IO slots?
IOX Diagnostics: Are there any specific diagnostic procedures or logs available on the IOX boxes themselves that you can access?
5. Cabling (Re-Verification):    

Cable Integrity: While you've swapped cables, it might be worth meticulously re-verifying the continuity and integrity of the cables specifically connected to IOXs 11 and 12. Even a seemingly good cable can have a subtle internal fault.
Connector Seating: Ensure the cables are fully and correctly seated in the ports on both the XFMs and the IOX boxes.
Given the persistence across hardware, it almost feels like there might be a subtle configuration quirk or an environmental factor at play.

[jsm6]

Hi,

I understand you have opened a support case with HPE Superdome team and working with them.

Please continue working with HPE support folks. We would go ahead and archive this community post.

Thanks

[Nosaj]

Yes, thanks for the help everyone!

We finally opened one yesterday and it looks to be that different cabling is used for IOXs 11 and 12 like you mentioned could be the case. 

We have ordered the cables so I hope that is it. Strange that longer cables is the answer, but fingers crossed. 

[support_s]

Hello,

Let us know if you were able to resolve the issue.

If you are satisfied with the answers then kindly click the "Accept As Solution" button for the most helpful response so that it is beneficial to all community members.

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".