Performance of A4924 1000Base-SX on a K?

[Don Mallory]

We have a K that's being used as a development server and there is a large amount of data that needs to be transferred to it in a short amount of time on an irregular basis.

We're looking at GigE as an option to reduce the downtime for the transfer.

There's a number of forum articles mentioning that you will not get anything close to line speed of 1000Mb on a K, but nobody says exactly how much less, only that it's faster than 100Mb.

The question is, how much faster than 100Mb, or how much less than 1000Mb can this card be driven on a K?

Thanks,
Don

[Steven E. Protter]

You kind of have to test to get those figures.

I've seen stuff written by Bill Hassell and others noting that the K class bus can't handle the high transfer rate and fully take advantage of the band width.

You will see an improvement over a 100 BaseT card, assuming your network switch architecture and settings support that.

SEP

[Zinky]

From experience:

100BT -- 8 to 11 Megabytes per second.

1000SX/BT -- 30 to 60 Megabytes per second.

[Don Mallory]

Understood, but is there a percentage, even with a reasonable error factor? Are we talking 10% slower, 20%, 50%?

I guess another way to look at it would be, what is the speed of the K-class backplane? If we know that the backplane runs at, say 500Mb, we can assume that the most we can get out the card is 500Mb, keeping in mind that this will still need to take into account time for accessing disk, etc., so likely half that, or 250Mb. This is a 2.5X speed increase over what we have now, and may be reasonable.

Based on this line of thought, if it's only a 250Mb backplane, then there is little to no benefit for the cost.

Any ideas?

[rick jones]

One is lucky to get 50 MB/s out of a single HSC slot. Consider that an uppoer bound. Then factor-in the slower (by today's standards) CPUs in the K and figure that 300 to 400 Mbit/s is probably in the ballpark of what you may get.

Since you say there is a large amount of data, that implies there will be disc access, so factor that into consideration as well - the transfer will not sustain a throughput higher than what the disc(s) can provide. FWD SCSI on the K is something like 20 MB/s right?

[Ted Buis]

You performance may also depend on what else is going on with some of the other I/O slots.
The first expansion board for HSC slots that is added to the K-class (one immediately to the right of the core I/O when looking from the back) can have 2 or 4 slots but there is only one HSC bus for all slots, so that bandwidth is shared. The second HSC expansion board for the K5x0 servers only, has four slots spread across two HSC buses, so if you are pulling from disk over FC and sending it out over GigE, then I would recommend to put each on a separate HSC bus. You should be able to tell some of this from the paths from an ioscan.

[Chris Prescott]

I've been using two gig-SX cards on a K570 for over a couple of years now for a NAS storage network. It was definitely an upgrade from the FWD scsi that we were using before that. One thing I'd recommend is that you try and configure Jumbo frames for the GigE card since this will increase throughput and reduce overhead since there will be larger data payload for each frame. Some gigE switches don't support the 9000MTU so check out what your switch is capable before configuring.

[Bill Hassell]

Rick covered the performance issues...most are related to the underlying K-series architecture and not the card itself. The GigE card will just be shuffling along waiting for the K-box driver and backplane to deliver data. It will be a little bit faster than 100BaseT but nowhere near 5x or more.

Believe it or not, you may find the fastest way to transfer massive amounts of data between two machines is a tape drive. Modern drives like a DDS4 or DLT 8000 transfer data (in LAN bits/sec) at rates more than 2000Mb/sec, way faster than GigE, especially on a K-box. True, the transfer rate is for one direction so restoring the data takes another pass, reducing the overall data rate to about half. Note that the overhead in popping the tape in and out should be insignificant for massive (dozens of Gb) data transfers.

[Steve Lewis]

If 20Mbytes per second is the i/o limit, then why don't you just use two 100Mbit cards instead and copy half the data down each interface?

Another (wild) option if you are using HVD SCSI and the servers are close together is to avoid using the network altogether.
Connect up the SCSI bus as per serviceguard with a controller on each machine; vgexport/vgimport the volume group data; unmount and mount the filesystems exclusively one server at a time; then you can copy it disk to disk or maybe even use the data in situ.

[Don Mallory]

Thanks for all your help everyone.

The disk on this box is actually on a VA7410, so it's already fiber attached.

We have done the whole create a LUN, copy the data, vgexport/vgimport, copy the data, and it lends itself too much to administrator error.

The biggest difficulty is that one of the filesystems, although only 18GB has 347000 files and this is a bit of an issue in most copy methods. Other filesystems have as few as 300 files, but some files are as large as 4-5GB.

This particular server doesn't have a huge window for downtime removing the options of cabling and uncabling disks or tape solutions, and as always, money is a very large object here, so even the purchase of a simple DDS4 drive becomes a challenge.

Considering that a data copy now takes 12+ hours, even if we get 2 to 2.5 times the speed out, it reduces the time down to ~6+ hours.

What is involved in adding a second 100Mb card and running bonded ethernet?

Don

[rick jones · Accepted solution]

In HP-UX land, bonding is called "trunking" and is accomplished with the add-on Auto Port Aggregation (APA) product. You can then trunk multiple NICs together to get higher aggregate throughput.

HOWEVER.

It will not in and of itself speed-up the performance of a single TCP connection. The traffic of individual "flows" (eg TCP connections) always go over the same NIC.

So, if your transfer mechanism involves a single TCP connection, you would need to make it two or more (preferably more) to get things spread-out across the trunk.

If things are really that time-critical, you might look into getting a newer system. The new rp34XX's might fit the bill nicely, or perhaps an rx[124]XXX. Those will be systems where you can get higher FC speeds and have a decent (if not guaranteed) shot of hitting GbE link rate, and ifyou do have a multi-connection transfer, going more than GbE with APA.

[Bill Hassell]

You'll need the APA (Auto Port Agggregation) plus a switch that supports the protocol. Unfortunately, APA cost some $$$, so you may have to justify the amount with the cost of downtime periods. Note that the total throughput depends on the backbone speed so if the switch goes to a 100BaseT network, you'll see no improvement. With APA, you could aggregate several LAN ports but as mentioned with GigE, the driver overhead along with the slower K-boxes may not double or triple throughput as expected.

[Don Mallory]

Thanks for all your help everyone.

Don