topic Re: I/O problem in Operating System - HP-UX

I/O problem

Venky_1 — Fri, 31 Jan 2003 10:34:24 GMT

Hi ,

There is a serious I/O problem on one of our rp5470 model ( with 2x750 MHz CPU, 4GB memory and HP-UX 11i using VA7100 storage via brocade switch being shared by another 2 hosts ).

During Volume Testing, conversion process runs into two steps, first step is the extraction process from tables into flat files and second process uploads it into database. Extraction takes 10 hrs and upload takes 25 hrs.

Same setup/activity, when performed on HP-UX 11.00 ( with 4 x 440 MHz CPU, 4GB memory connected to FC-60 storage through FC interface.) takes 1 Hr for extraction and 12 Hrs for upload. There is an exponential difference in the performance.

Both the servers have latest OS patches. During the activity I/O is observed with the FS where oracle resides and there is no I/O with any other FS as per glance reports and other tools.

Oracle is properly tuned as the above tests on other server shows. CPU and Memory utilization is normal.

Any comments/suggestions.

Thanks
Venky.

Re: I/O problem

Ian Dennison_1 — Fri, 31 Jan 2003 10:45:55 GMT

Two questions,...

1. How often is the system waiting for IO when processing on the VA? (sar -u)

2. How is the System portion of CPU behaving? (sar -u or glance)

First impressions would be that the System may be spending time waiting for IO, or that it may be rolling in and out Processes from memory (did you reduce the number of Oracle processes for the machine with less CPUs?)

Share and Enjoy! Ian

Re: I/O problem

Stefan Farrelly — Fri, 31 Jan 2003 10:49:07 GMT

If your storage is shared on the VA7100 then this is going to affect i/o bandwidth. Can you retry the test without the other 2 hosts sharing the VA7100 just to eliminate this is not the cause (ie. shutdown the other 2 hosts sharing it).

I think that would be the first thing to try. If that doesnt fix the problem then there must be some serious difference between your 2 HP servers or the db setup. You will need to get into eact details, ora .ini config files, db layout/size, sar stats, glance stats, etc. in order to try to track it down.

Re: I/O problem

Eugeny Brychkov — Fri, 31 Jan 2003 11:24:41 GMT

Venky,
who configured VA7100 in the SAN? Are you sure it was done properly?
Gather: 'supportshow' outputs from each brocade, 'armdsp -a' output for VA, zip them with winzip and attach to your next reply
Eugeny

Re: I/O problem

Steven E. Protter — Fri, 31 Jan 2003 21:56:50 GMT

You'll need to see throughput on that SAN. How big is the data involved?

Is glance showing bottlenecks during the process. That can point to the real cause of the problem.

There are options concerning block sizes and such that can have a huge impact on oracle performance. It likes big block sizes.

Do you have adequte resources in that database for memory, rollback space and such?

Are archive logs turned off? They should be for a huge operation like that.

I'd like to see kmtune output to see what the kernel looks like and the init.ora file.

I'm a sabbath obervant Jew so I won't be able to look at anything until Saturday night.

P

Re: I/O problem

Steven E. Protter — Fri, 31 Jan 2003 22:14:25 GMT

There are a couple of nice Oracle data collection scripts in this thread that might let you monitor oracle functionality during the transfer.

Take a look, they might help.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x4177ef70e827d711abdc0090277a778c,00.html

I'm real proud of that thread and what I've used has been high quality.

P