Large disk performance

[Tonny Sejr Kromann]

Hi
Is there anything in 11.0 that creates a bottleneck when 3 "disks" with a size of ~150GB is put in one vg?
I had to increase PE-size to 8MB end thereby got about 65000 PE's. But perfoamnce is very poor, about 7MB/s. The "disks" are each 6 x 36GB in raid5, and I would expect a much larger throughput. I've been told that there must be a bottleneck in hpux. I've seen one single disk perform 23MB/s.
Anyone?

--
Best regards
Tonny

[Servio Cabrera]

Could be several thing, but start using GlancePlus, if you dont have it, you should install it.

Make sure you have the latest Patch Bundle, for HW and for General Releases. (They include performace patches)

With Glance try to double click on the process that is using PHy I/O, and see the stop reason.

[Stefan Farrelly]

Having 150GB of disk in one Volume group is not a problem. The question is how big are your logical volumes ? And are your logical volumes striped over all the available disks ?

So, whats the performance test time differences between;

[Physical disk - or Lun in in a Raid5 group]
dd if=/dev/rdsk/cxxxxx of=/dev/null bs=1024k count=50
and
[Logical volume]
dd if=/dev/vgXX/rlvolYY of=/dev/null bs=1024k count=50

I would expect these to be very similar. How did you measure your 7MB/s ?

Certainly increasing the PE size from 4 to 8 may degrade your performance a little, depending on your application. What is your app ?

[Vincenzo Restuccia]

Try man iostat.

[Tonny Sejr Kromann]

The vg is ~3 x 150GB
All my lvols are 2062MB and striped across the three raidgroups. Also I have tried to create one lv which was not striped, with similar perfomance. I've trie stripesize of 64k and 1024k.
The readtest was performed with:

time dd if=xx of=/dev/null bs=2048 (1k - 16M) and each time it ended up in quite the same time.

--
Regards
Tonny

[Stefan Farrelly]

Sorry, Im still confused. If you do the tests right now (when the system is hopefully not busy) on the raw disk and the raw lvol what are the exact results (just use a blocksize of 1024k)

[Servio Cabrera]

While U are doing the "dd" test, try in another window "sar -d 2 100" and tell us the blks/sec, shuld be higher than 20000.

Have you checked the cabling, If you run SCSI, check the lenght and propoer termination.

[Carlos Fernandez Riera]

What hardware are involved?

[Roberto Gallis]

Hi,
are u using alternate links?
if yes, check they are properly set.

[Tonny Sejr Kromann]

If I run it again

sar says:
13:25:26 c9t0d0 25.37 0.50 1019 4076 5.00 0.24
c9t0d1 29.85 0.50 1143 4573 4.98 0.25
c9t0d2 28.86 0.50 1080 4320 5.01 0.25

It is 1 Compaq HBA connecting via fiber to a Compaq SAN switch.
It read the 2062MB in 5:20.1 = 6.4MB/s which is absolutely disappointing.
I hope one of you have a good suggestion.
Thanks

--
Regards
Tonny

[Servio Cabrera]

First advice, DO NOT run COMPAQ Storage! ;)

If you have any utility like "fcmsutil", try to see the link statistics for the FC link.

Probably you have a intermitent link failures, like packets lost, syncronization fails, etc.

Send us the stats.

[Stefan Farrelly]

If that 6MB /s is using dd on the raw disk device then youve got a hardware problem. Nothing to do with HP-UX.

[Tonny Sejr Kromann]

Hi
Thanks for all your help.
This has not been the brightest day.
dd must of course read the raw volume (rlv).
My readperformance then reaches 50MB/s which I can accept since it's raid5.
Thanks again.

--
Tonny

[Carlos Fernandez Riera · Accepted solution]

excuse me:

I know that no one have helped you directly, instead there are some people trying helping...

Perhaps is it time to assings points???...

[Tonny Sejr Kromann]

Now I've assigned some point and hope I'm excused then :)
Btw. The XP256 disk-array with the slow 36GB disks can apparently only deliver 17-18MB/s.
Our Compaq array can deliver 50MB/s but is then not as secure as the XP256.
Thanks all.

[Carlos Fernandez Riera]

Points rejected: [N/A] is the correct for me.

17/18 Mb/s on XP's ..??? Maybe the connetion is from XP256 to HUB-FC. If you are using a hub all trafic share bandwise, and the performance decrease.

[Tonny Sejr Kromann]

Sure, I did'nt mean to give you 10pts. Kernel panic ...
Yes the systems are connected through hubs and each server has 2 ctrl's, each to a hub.
But the other systems does not put much load on the xp. I'd say max 20% when things are wild. But I'm planning on making a measurement with the 2 other databases shutdown.