Tunning problems...

[Jonathan Caplette]

Hi guys,
I have an easy one for you... I have a K370 box with an AutoRAID 12H runnig HP-UX 10.20, and I think that I have a little I/O problem.
This is what's happening, the machine is slowed down when I take the backup... I tried to monitor the processe and there CPU usage with top, but the CPU is not used much more that 30%... What I want to do is to see what the disk are telling me, can I?? Do you know a way to see what's the I/O load on my disks??? I don't what to use SAM...

I really don't think that the speed problems is coming from my disks, but from the network... The Telecomm people here are saying to me that the problems is coming form my box, I don't think so... I just want to proof them that it's not my box...

regards...
Jonathan

[Deshpande Prashant · Accepted solution]

Hi
Try #sar -d 1 4 to check disk io.
The higher numbers on avque tells slow disks.

If you have glance use it to see if any specific filesystem is loaded with read or write, instead of all file systems equally loaded.

Thanks.
Prashant Deshpande.

[A. Clay Stephenson]

Hi Jonathan:

If you have Glance that is the perfect tool. sar -b, sar -d, and iostat also give you data.

It is perfectly normal for backups to impose
large loads on the I/O channels especially if you are running Veritas, Omniback, or fbackup.

You can also be very I/O bound if your AutoRAID is configured so that most traffic is going through controller X and Y is only there as an alternate.

[Sachin Soni_1]

Hi,

Use #sar -d
look for %busy.

u will also get some idea , from sar- u
it tells abut cpu ,
look for %wio field.
it will go up , if there is high disk i/o.

Regards,
Sachin

[Sanjay_6]

Hi Jonathan,

Am posting a reply i posted for another answer just a few minutes back.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,1150,0x4284dfe5920fd5118fef0090279cd0f9,00.html

http://us-support.external.hp.com/cki/bin/doc.pl/sid=236a449608be7b73d4/screen=ckiDisplayDocument?docId=200000049436468

http://us-support.external.hp.com/cki/bin/doc.pl/sid=236a449608be7b73d4/screen=ckiDisplayDocument?docId=200000050018417

Hope thius helps.

Thanks

[A. Clay Stephenson]

Hi Again:

Now that I think about it, Glance is the answer. If you don't have it; install the 30-day Trial Version on the Application CD's. Glance will help you nail the problem whether it's disk, network, CPU, or Memory.

[Bill Hassell]

I would assume that any machine that is performing a backup while running other processes is going to run slower. A backup represents a very unusual load on the disks, that is, continuous directory and filesystem reads and should be considered to be quite normal.

Glance will show you that disk I/O is maxed out on specific channels, and the channels will change as other disks are being saved. And Glance will report that the backup program is the largest disk user, all quite normal.

[Jonathan Caplette]

thanks guys....

I will see what I can do with those utility... I got another problem on the same box... I've to copy a database... Last week the database was taking 11GB on my hard drives and the copy was done in an hour... And now, the DBA's have expand there DB to 13GB instead of 11GB and since then, the copy of the same database but with a new size, take aprrox. 2h 45min.... That's make the count to 1h 45min, to copy 2GB...

Could the utility u suggest to me help me to diagnose this, or could it be something else....

Regards
Jonathan

[Deshpande Prashant]

Hi
Sar and glance will certainly help you.
How are you really copying the database and to what kind of tape drives? It may be worth looking load on those drives.

Prashant.

[Jonathan Caplette]

I'm closing the DB, then copying it into a tmpdir, and open back the DB, cause I can't take online backup with ARCServe 2000 for NT... What I mean, is that I have a centralized NT server dedicated to the backups, the media is DLT tapes.

[Deshpande Prashant]

Hi
So you are copying 13GB data from one set of file system to another filesystem & this copy is your problem?

Are these file systems on different scsi/fiber channels? Certainly look at disk queue using sar for both set of disks and file systems.

Any messages loged in syslog during copy due to disk problem?

Thanks.
Prashant.

[Jonathan Caplette]

Copying the DB is a "new" part of my problem...

The filesystems are not on the same disk, but all my disks are in an AutoRAID 12H, so SCSI device.

Look at the attachment to see what I got in my syslog.log file... And tell if this told you something.... This is what I got frequently in my syslog file....

Thanks!

[Sridhar Bhaskarla]

Your syslog doesn't really tell anything. That is related to tooltalk server for X windows stuff.

When you are running your backup, try running

sar -d 2 100

Keep a look at %busy column. If it is around 100% most of the time, take a note of the correpsonding device and traslate it to the LUN on 12H. Also keep track of avque column. If it is anything other than 0.5, there is a queue built for that device. Simultaneously, check the avwait column. It shouldn't be too high and shouldn't be increasing over the time.
If a disk is constantly 100% busy with a lot of queuing, you may want to distribute the logical volumes on it across different disks.

While doing so, run sar 1 100 and check %wio. If it is more than 15, there is a disk bottleneck on the system. What are the buffercache settings on your system?. Do a kmtune |grep dbc and check the values. YOu don't want to keep a too high value for dbc_max_pct.

Try to get 'sarcheck' evaluation copy. If you like it, you can purchase it. It gives nice analysis reports.

-Sri

[Steve Sauve]

A couple things to consider.
Does anything else kick off after the first hour of copying the DB? (perhaps it runs into part of the backup thereby creating a ton of IO)
Where did the extra 2 gig come from. When the DB was extend was it onto the same disk or is that coming from slower/more bottlenecked disk?

You mention that you copy the DB, are you using DB tools to export the database or are you just doing a file level copy?

Steve

[Jonathan Caplette]

Hi Steve,

That's what I was thinking about... but I've tested the copy at a time that there's no activity on the machine, and it take approx. the same time...
I'm gonna keep you in touch...

thanks all...