Point to Point Network with 1000Base-SX LAN Card, HowTo?

[MaSa_3]

Hello All,
I am building a two node SG Cluster,
I want to use a dedicated Point to Point heartbeat LAN with no active components.
Both nodes have a 1000Base-SX LAN card, which I want to use for that purpose.

I connected them with a Crossover Fibre Cable, On both Cards the "green LED" indicates a Link.
But I did not manage to get it working landamin shows always "Operation Status (value)=down"

Maybe it is not possible at all to link to Cards in that way!?
Dose anybody know how to make it work?

Thanks in Advance !


Ioscan:
lan 1 0/3/1/0/6/0 igelan CLAIMED INTERFACE HP A9782-60002 PCI/P
CI-X 1000Base-SX FC/GigE Combo Adapter


LAN Admin:
PPA Number = 1
Description = lan1 HP PCI-X 1000Base-SX Release B.11.11.25
Type (value) = ethernet-csmacd(6)
MTU Size = 1500
Speed = 1000000000
Station Address = 0x15605eb906
Administration Status (value) = up(1)
Operation Status (value) = down(2)

[Duncan Edmonstone]

I've never really used fibre GbE very much, so I can't comment on whether this will or won't work...

What I would suggest though is that you take auto-negotiation out of the equation (as I can see this failing when a switch isn't involved) by attempting to set both cards to a specific hard speed.

Try using:

lanadmin -X 100FD 1

To set the card (PPA=1) to 100 full duplex. Make sure you do both cards.

And see how that works.

You are aware of the issues with using a cross-over cable with Serviceguard?

http://www12.itrc.hp.com/service/cki/search.do?prevQueryString=&mode=id&searchString=emr_na-c00933836-4

[Duncan Edmonstone]

hmmm link didn't work - try this one:

http://www12.itrc.hp.com/service/cki/search.do?prevQueryString=emr_na-c00933836-4&mode=id&searchString=KBRC00009382&submit1=Search+%C2%BB&docType=Advisories&docType=EngineerNotes&dateSearchType=lastXdays&dateRange=all&startDateYear=1999&startDateMonth=0&startDateDay=1&endDateYear=2008&endDateMonth=3&endDateDay=21&resPerPage=20&sortCrit=rank

[Steve Lewis]

Yes it will work, I have done it myself for normal network operations.

I suggest you check your cable for the required twist and reverse the fibre cables in one end if you dont have it.

However I dont think its a good idea for Serviceguard because if a node gets a link down on that interface it won't know if its the local end that has the problem or the other node.

Therefore you should really use a switch inbetween, which keeps a link up to the local node if the other end goes down. You can buy special 1000-base-SX transcievers for most switches these days.

[Steve Lewis]

Bad spelling SteveL: I before E except after C ...

or is it a transponder ?

or a transducer ?

[MaSa_3]

I am aware of the MC/SG issues, I have got several Clusters running with Crossover Links with Copper Crossover Cables, but the Clusters
should be prepared to be migrated to Campus clusters without changing the architecture to much, and that should be done
with a minimum support and costs from the networking infrastructure department.
That is the background of my attempt.

So lets keep out Serviceguard and just focus on the networking issue.

Autonegotioation is already OFF.
Changing the Link Speed is not possible on 1000Base-SX Card, they have Fixed Link speeds.

@Steve You have already did that ?
Have you some more details?
Thanks for the Tip with twisted cable. - I did that already!
If I use a standard cable, the green Link LED went on out.


xxxx(root): / > lanadmin -X 100FD 1
ERROR: This option is applicable to 1000Base-T only.
xxxxx(root): / > lanadmin -x speed 1
The link is down. The speed and other link state information
are undefined. The configuration specified for this card is:
Speed = 1000 Full-Duplex.
Autonegotiation = Off.

[Steve Lewis · Accepted solution]

Hi again,
I did nothing special to get it going. I just configured the IPs at each end on a completely different subnet, plugged the crossover cable in and did a linkloop, then a ping.
Then I got clever and set the MTU to 9000 because my link was an interconnect between NPARs in an old Superdome.
You sound like you know what to do. Maybe your fibre cable is broken. Maybe you have the wrong type of fibre cable (optical quality). I doubt that is the case because 1000-base-SX isn't as fragile as 10GBE or 4GB FC but its worth checking. Put the cable back to normal and test it in another server, or from server to switch.
If you are going from building to building then make sure that you have the correct cable at the other end.
If ping does not work but linkloop does work, then check your IPs and routing to ensure that it isn't sending from the wrong interface.
Get linkloop going first. Then worry about Serviceguard. I assume that Serviceguard will be supported for heartbeats over 1000-base-SX but I am not sure. You need to get it pinging and linklooping first as I am sure you know.

[rick jones]

Gigabit Ethernet cards do not require cross-over. Either for fibre or copper. The cards figure it all out on their own.

Fibre does not want twist :)

Gigabit "always" wants autoneg.

If you have a switch and connect the cards to it, do you get an operational status of up?

If you do not have a switch with which to try the experiment, you might try getting your hands on a loopback hood and see if you can get each card opstat up in turn.

Another consideration would be to try to run some diags on the cards - either online via stm or offline diags if you can bring the boxes down.

And there is the usual song and dance of being up on the latest patches. :)

[MaSa_3]

Steve was right !
Nothing special was has to be done to get it working, just the twisted cable in the correct ports and it is working fine.

My fault was that I got confused with the hardware paths, the systems got two of those cards and I treated one set of interface cards with ioscan and
landiag, while the cables are plugged in the othe interface card showing the Linke state with the green LED.
So it was my blindness that causes that trouble ....

sorry for that an many thanks to all for the help!

[MaSa_3]

see commments above.