Max throughput of 1GB / 2GB fibre

[Gordon_3]

Hi all,

Can anyone here tell me what's the spec. for 1GB / 2GB fibre on max throughput in terms of

1. No of I/O it support
2. MB per sec

Actually I have no idea on which is limiting factor, coz they are related indeed, I was told by EMC that normally 1GB is 1500 per sec, is that true? Coz I use XP in our site, and my Performance Advisor indicate that it has 2800 per CHIP port I/O .... I cannot match this 2 figure, pls help , thx.

Gordon

[Leif Halvarsson_2 · Accepted solution]

Hi,
Number of I/O (IOPS=I/O per second) depends on the size of each I/O, this is perhaps more related to the disk system (fast controllers can perform 100 000 IOPS or more).

Transfer rate (MB/S), the specification is 1 or 2GBit/s (slightly more then 100 or 200GB/s. You should note that FibreChannel is full duplex, in theory you can have this speed in both direction simoultaneous. This should mean 200 or 400 MByte/s.

[Leif Halvarsson_2]

Sorry,

Of course not 100 or 200GB, it should be 100 or 200MB (Megabyte).

[Gordon_3]

HI Leif,

Much thx for your information, indeed, I support your point that actually point 1 & 2 is related coz it depends of block size ( but too bad that seems I have no tools to see which block size it's using... so no idea about this part ) I have one docs from HP that it claims the fibre should reach around 90MB for max throughput, which is quite close to yr statement. Anyway thx for yr sharing.

Gordon

[Vincent Fleming]

Gordon,

There is a big difference between bandwidth (what the spec says it should be able to do) and actual throughput (observed throughput).

I've seen 1Gb Fibre Channel hit 94MB/s. 2Gb is good for just shy of twice that - about 180MB/s (maximum). BUT, it depends heavily on the disk array and host you are using.

For IOPS, it depends even more heavily on the host and array; I've seen 1Gb ports doing over 5500 IOPS sustained.

Perhaps EMC is limited to about 1500 IOPS per port (too bad for them, huh? :-o ), but as you can see from your XP, the limiting factor is not the Fibre Channel, but the disk array or host. The XP1024 has been tested at > 15,000 IOPS per port, on all 32 ports simulatneously. (all cache hits, of course)

So, typically, your actual throughput will not be constrained by the Fibre Channel itself.

I hope this helps,

Vince

[Gordon_3]

Hi Vince,

Many thx for your expert advise, so if the medium ( fibre ) is not a limiting factor for, for monitoring purpose, how can I know I already hit the bottlenect in both HBA, XP port, etc. I ask just for capacity planning, and also query from boss ^_^, pls help enlighten. thx thx.

Gordon

[Vincent Fleming]

Gordon,

It can be difficult to determine where a bottleneck lies in a FC network.

Monitoring tools are essential. For example, you need to monitor your ISLs (links between switches) - typically this is done via a web interface on the switches themselves, but there's other software available. Look for high utilization rates on the ISLs; this would indicate a bottleneck there.

On your XP, use Performance Advisor to determine if the ports CPUs are overloaded (port CPU utilization). Balance your load as best you can. Also look for array group hotspots, and overall utilization of the busses and ACP pairs.

OV SAM (OpenView Storage Area Manager) is a really good tool for monitoring your network and XP.

Basically, if you can rule out the network and the XP as the source of the bottleneck, the only other place it can be is on the host.

Often, I find customers with performance problems that are caused by their system's configuration - for example, they put their entire database on a single array group and are doing table scans because they have inappropriate indices on the database. This will cause a performance problem that cannot be solved by tweaking the array or network.

Don't forget that your array's performance is (in the end) gated by the number of drives you're hitting. Use host-based striping over LDEVs that are on different array groups for the best array performance. Each array group is good for somewhere between 400 and 600 IOPS, depending on drive model. If you need to hit 6000 IOPS for your application, be sure to stripe over at least 10 array groups (assign the host LDEVs from at least 10 different array groups).

Good luck,

Vince