topic Re: Large disk performance in Operating System - HP-UX

Large disk performance

Tonny Sejr Kromann — Mon, 30 Apr 2001 08:57:34 GMT

Hi
Is there anything in 11.0 that creates a bottleneck when 3 "disks" with a size of ~150GB is put in one vg?
I had to increase PE-size to 8MB end thereby got about 65000 PE's. But perfoamnce is very poor, about 7MB/s. The "disks" are each 6 x 36GB in raid5, and I would expect a much larger throughput. I've been told that there must be a bottleneck in hpux. I've seen one single disk perform 23MB/s.
Anyone?

--
Best regards
Tonny

Re: Large disk performance

Servio Cabrera — Mon, 30 Apr 2001 09:17:55 GMT

Could be several thing, but start using GlancePlus, if you dont have it, you should install it.

Make sure you have the latest Patch Bundle, for HW and for General Releases. (They include performace patches)

With Glance try to double click on the process that is using PHy I/O, and see the stop reason.

Re: Large disk performance

Stefan Farrelly — Mon, 30 Apr 2001 09:20:04 GMT

Having 150GB of disk in one Volume group is not a problem. The question is how big are your logical volumes ? And are your logical volumes striped over all the available disks ?

So, whats the performance test time differences between;

[Physical disk - or Lun in in a Raid5 group]
dd if=/dev/rdsk/cxxxxx of=/dev/null bs=1024k count=50
and
[Logical volume]
dd if=/dev/vgXX/rlvolYY of=/dev/null bs=1024k count=50

I would expect these to be very similar. How did you measure your 7MB/s ?

Certainly increasing the PE size from 4 to 8 may degrade your performance a little, depending on your application. What is your app ?

Re: Large disk performance

Vincenzo Restuccia — Mon, 30 Apr 2001 09:22:31 GMT

Try man iostat.

Re: Large disk performance

Tonny Sejr Kromann — Mon, 30 Apr 2001 09:32:30 GMT

The vg is ~3 x 150GB
All my lvols are 2062MB and striped across the three raidgroups. Also I have tried to create one lv which was not striped, with similar perfomance. I've trie stripesize of 64k and 1024k.
The readtest was performed with:

time dd if=xx of=/dev/null bs=2048 (1k - 16M) and each time it ended up in quite the same time.

--
Regards
Tonny

Re: Large disk performance

Stefan Farrelly — Mon, 30 Apr 2001 09:40:57 GMT

Sorry, Im still confused. If you do the tests right now (when the system is hopefully not busy) on the raw disk and the raw lvol what are the exact results (just use a blocksize of 1024k)

Re: Large disk performance

Servio Cabrera — Mon, 30 Apr 2001 09:41:15 GMT

While U are doing the "dd" test, try in another window "sar -d 2 100" and tell us the blks/sec, shuld be higher than 20000.

Have you checked the cabling, If you run SCSI, check the lenght and propoer termination.

Re: Large disk performance

Carlos Fernandez Riera — Mon, 30 Apr 2001 09:56:08 GMT

What hardware are involved?

Re: Large disk performance

Roberto Gallis — Mon, 30 Apr 2001 10:03:35 GMT

Hi,
are u using alternate links?
if yes, check they are properly set.

Re: Large disk performance

Tonny Sejr Kromann — Mon, 30 Apr 2001 10:33:33 GMT

If I run it again

sar says:
13:25:26 c9t0d0 25.37 0.50 1019 4076 5.00 0.24
c9t0d1 29.85 0.50 1143 4573 4.98 0.25
c9t0d2 28.86 0.50 1080 4320 5.01 0.25

It is 1 Compaq HBA connecting via fiber to a Compaq SAN switch.
It read the 2062MB in 5:20.1 = 6.4MB/s which is absolutely disappointing.
I hope one of you have a good suggestion.
Thanks

--
Regards
Tonny

Re: Large disk performance

Servio Cabrera — Mon, 30 Apr 2001 10:42:46 GMT

First advice, DO NOT run COMPAQ Storage! ;)

If you have any utility like "fcmsutil", try to see the link statistics for the FC link.

Probably you have a intermitent link failures, like packets lost, syncronization fails, etc.

Send us the stats.

Re: Large disk performance

Stefan Farrelly — Mon, 30 Apr 2001 10:48:06 GMT

If that 6MB /s is using dd on the raw disk device then youve got a hardware problem. Nothing to do with HP-UX.

Re: Large disk performance

Tonny Sejr Kromann — Mon, 30 Apr 2001 13:02:18 GMT

Hi
Thanks for all your help.
This has not been the brightest day.
dd must of course read the raw volume (rlv).
My readperformance then reaches 50MB/s which I can accept since it's raid5.
Thanks again.

--
Tonny

Re: Large disk performance

Carlos Fernandez Riera — Mon, 30 Apr 2001 13:11:18 GMT

excuse me:

I know that no one have helped you directly, instead there are some people trying helping...

Perhaps is it time to assings points???...

Re: Large disk performance

Tonny Sejr Kromann — Wed, 09 May 2001 09:16:00 GMT

Now I've assigned some point and hope I'm excused then :)
Btw. The XP256 disk-array with the slow 36GB disks can apparently only deliver 17-18MB/s.
Our Compaq array can deliver 50MB/s but is then not as secure as the XP256.
Thanks all.

Re: Large disk performance

Carlos Fernandez Riera — Thu, 10 May 2001 09:39:40 GMT

Points rejected: [N/A] is the correct for me.

17/18 Mb/s on XP's ..??? Maybe the connetion is from XP256 to HUB-FC. If you are using a hub all trafic share bandwise, and the performance decrease.

Re: Large disk performance

Tonny Sejr Kromann — Mon, 14 May 2001 07:40:09 GMT

Sure, I did'nt mean to give you 10pts. Kernel panic ...
Yes the systems are connected through hubs and each server has 2 ctrl's, each to a hub.
But the other systems does not put much load on the xp. I'd say max 20% when things are wild. But I'm planning on making a measurement with the 2 other databases shutdown.