maximized transfer speed Gigabit Ethernet

[yunardi]

Dear Experts,

I have litle problem with my connection.

I have 3 Server using Gigabit Ethernet Cards (1000Base-T) : 1 RP8400, and 2 Partition of HP Superdome. They are using same version of OS (HP-UX 11.11i). File to transfer is 10MB to 100MB in root directory (int. disk)

Between RP8400 and 1st partition of superdome, I'm using direct cable connection (cross cable). Both cards have been set to 1000-Full Duplex-Autoneg=on. Transfer speed using this connection is about 17MBps, but I think its should be much better why...

Between 1st partition and 2nd partition (hard partition), I'm using LAN Connection with Catalyst Network Switch, of course the data transfer between this partition will be accross the network. But its amazing because transfer speed is between 60MBps to 105MBps.

It's strange, why transfer speed on direct connection slower than via LAN... the file size, card setting, cable are same ?

Can you help me, because our HP Support cant give any solution, they said may be I'm using bad cable for RP8400-superdome connection, so I switch the cable and the result is still same.

Thanks in advance.

Regards,
EKO

[Sudeesh]

Hi,
I think you need to check the connection speed (between rp8400 and superdome)of the lan cards using lanadmin. There is a chance autoneg is on and these cards get connected at a lesser speed.

I would suggest to turn autoneg off and forcefully set both cards at 1000Full duplex.

Give it a try.

Best of luck

Sudeesh

[yunardi]

Dear Sudeesh,

I have been trying to set autoneg to off and force it to 1000-FULL (using SAM) but always get the messages:
"Speed value cannot be set to 1000, when autoneg is off"
I think Gigabit Ethernet only support AutoNeg for 1000.

The connection from 1st and 2nd partition of superdome is set autoneg too, and it gives diferent result.

Thanks,

[Sudeesh · Accepted solution]

have a look at this document...this may help you

http://docs.hp.com/en/J5683-90010/ch05s01.html#d0e3471


Sudeesh

[yunardi]

Hi Sudeesh,

Refer to your link tou give, there are some parameter can be set by lanadmin or SAM.

I have been set it to 1000/FULL/Autoneg=ON, but as you know the result not relly good; so I set the MTU Size from 1500 to 9000 and its give better result.

The transfer speed (ftp) is range between 30MBps (put) to 105MBps(get).

Btw why while using ftp put speed is always slower than get.

Thank very much

EKO

[MAUCCI_2]

Hi,

could you compare get and put when
1-ftp client is your rp8400 / server is SD
2-ftp client is your SD / server is rp8400
?

If get#1 == put#2 then it is consistant.
If not, there might be a unix process priority issue on either host.

Leave auto neg to on, and check through lanadmin -g mibstats #lanid
that you get 1000FD (on both sides).
Do it a few times to check that your lan does not oscillate between 1000FD and 100FD or 1000FD and 1000HD or things like this.
I used to see plenty of auto neg pbs with 100BT cards that led to such behaviors, either with cross over cables or with some old hubs/switches.

Since 1000FD can not be set without autoneg on, and if you fall into an autoneg pb between those GBe cards (what model is it BTW?), you may need a GBe switch in between.

When you talk about 17, 60 and 105.
Are you sure these are MBps (Mega Bytes per sec), or are they Mbps (Mega bits per sec)?




++cyrille

[yunardi]

Hi Cyrille,

Thanks for your advice.
The result is :
1. get1:
ftp> get testftp
10485760 bytes received in 0.12 seconds (84072.25 Kbytes/s)
2. put2
ftp> put testftp
10485760 bytes sent in 0.22 seconds (47548.29 Kbytes/s)

So get1 > put2, is this normal ?

The result of lanadmin is :
1. on SD :
# lanadmin -g mibstats 3
PPA Number = 3
Description = lan3 HP PCI 1000Base-T Release B.11.11.15
Type (value) = ethernet-csmacd(6)
MTU Size = 9000
Speed = 1000000000

2. on RP8400
# lanadmin -g mibstats 2
PPA Number = 2
Description = lan2 HP PCI 1000Base-T Release B.11.11.09
Type (value) = ethernet-csmacd(6)
MTU Size = 9000
Speed = 1000000000

I try it few times and get the same result, it allways on 1000FDX

Thanks,
EKO

[MAUCCI_2]

EKO,

what I meant by comparing get and put on both hosts is the following:
let's say rp8400 is machine A and SD partition 1 is machine B

Log onto B:
ftp A
1-get bigFile1
2-put bigFile1
quit

Log onto A:
ftp B
3-get bigFile1
4-put bigFile1
quit

Logically, get from B to A should be the same as put from A to B, right?

Also, when dealing with GBe, please transfer big files, like at least 100 MB files and do the tests 2 or 3 times at least.
Can you please re-run the test, and send us the results for 1, 2, 3, 4.

You copied lanadmin outputs
for lan3 on SD and lan2 on rp8400.

Are you sure these are the right lans that are part of your ftp xfer?

Thanks
Cyrille

[yunardi]

Hi Cyrille,

I have ftp 100Mb files between 2 servers (put and get), but the result is different :

1. On SD as client :
ftp> put ftptest
104857600 bytes sent in 4.87 seconds (21014.71 Kbytes/s)
ftp> get ftptest
104857600 bytes received in 4.65 seconds (22016.02 Kbytes/s)
ftp>

2. On RP8400 as client :
ftp> put ftptest
104857600 bytes sent in 2.45 seconds (41837.46 Kbytes/s)
ftp> get ftptest
104857600 bytes received in 2.08 seconds (49217.33 Kbytes/s)

And I have make sure about Lan ID for lanadmin. Isn't is the number in "Crd In#" column when we using lanscan.

[MAUCCI_2]

EKO,

have you run the ftp tests several times?
Do the results vary?

Is there any activity on either platform other than your ftp tests?

In any case the source of your pb seems to depend on which host is ftp cli and which one is ftp srv...

I'll investigate this.

Regards
Cyrille

[harry d brown jr]

Did you set the MTU to 9000 on both servers?

The ftp->put is probably being slowed down by the speed of the disk drive. Using ftp to measure through put is a bad idea. Use netperf: http://www.netperf.org/

I have found that Jumbo frames (MTU=9000) works great when:
â ¢ The Socket size is set to 128K
â ¢ The Message size is set to 32K
â ¢ the lan is used only for data transfers

The difference between Jumbo frames (mtu=9000) and MTU=1500 is 6 to 1, meaning you will also reduce cpu overhead. BUT I would NOT use Jumbo frames for normal network traffic. It should ONLY be used on lan's that send or receive large amounts of data.

I have proven than three servers sending to one server, using jumbo frames, can sustain a rate of 96 MegaBytes a second! I guess the next step is 10gbit cards?

Also, a utility called ttcp is useful also to do "benchmarking": http://hpux.cs.utah.edu/hppd/hpux/Networking/Admin/nttcp-1.47/

live free or die
harry d

[Alzhy]

EKO,

Try this to test your inter-server connectivity - whatever the connection. This will totally negate effect of diskspeed and cache (Filesystem).

In your FTP session:


ftp> put "|dd if=/dev/zero bs=32k count=1000" /dev/null

Try both ends and post your results.

1000BT links are always AUTO-NEG and if I rememeber cannot be set to 1000FDX. You should get a pure network throughput of between 60 to 100 Mbytes/sec.

[Florian Heigl (new acc)]

For Your information - when 1000BT hosts are connected in a cross-over setup, autoneg really has to be enabled, because they not only negotiate link speed, but also a master/slave handshake happens. If that doesn't happen, there be dragons. ;)

[yunardi]

Great...
Thanks for all information, I wonder if our HP Support in my country like you; because I have facing this problem for almost 1 year.

For information, we are using SD and RP8400 for Data Warehouse, and we are transfering large amount (Gigabyte) of data everyday.

I'll try your suggestion.

Thanks,

[MAUCCI_2]

Hi EKO,

in addition to all the tests listed above, it would really be great that you try to connect them through a GBe switch and see if the symptom is still there.

In any case, please keep up posted about your findings, because it is still interesting that we know the root cause of the pb...

++Cyrille

[Jay Kidambi]

Any more data on this? Let me try a few more suggestions:

Have you tried with standard size MTU, i.e., 1500 bytes? Also in one of your earlier e-mails, I noticed that the SD-SD ftp time was shown to be longer than the SD-rp8400 number. Was that a typo? On the rp8400, which LAN interface are you using? An "ioscan -fkC lan" output would be useful. In the ioscan output , please identify the lan interface you are using.

Jay