IO issues after install of June 2004 11i Bundle

[Bernie Vande Griend]

Got an interesting one here. We have 2 systems were we installed the complete bundle from June2004 (hardware, apps core, etc) and today they are having IO issues with both. Both issues are with the filesystem that houses the temporary files for the application. The number of writes is doubled than what we see on systems without the patches. On one system it is the boot drive, on the other it is a SAN drive. We've pretty much ruled everything out except the patches. Does anyone have an idea what patch may have caused this issue or heard of anything like this? I would suspect a vxfsd patch or buffer patch. Any help is greatly appreciated.

[RAC_1]

Out of the patches that you have installed, how many of them are related to i/o performance?? Check those and check if any superceded patches are there or not.

Also, are you sure you have not anything else, kernel tuning etc??

Anil

[Steven E. Protter]

Can you clarify to me what you mean by i/o issues?

High wait times, errors? what?

Perhaps a little data collection would be good.

Patches can cause lots of problems but I've got the patch sets in on 3 servers and have not had the same kind of issues.

SEP

[Bernie Vande Griend]

OK, here's some more information on this continuing saga. Nothing else was changed except for the June 2004 bundles for hardware, base, and apps on both systems. Previous patch level for base was June 2003.
Identifying performance related patches is a good idea, but where do you start with this number of patches. That is why I was hoping to narrow it down.

As for details, we are really just seeing more IO traffic and just to the location the application uses for temporary files. I did a little more research using measureware and now clearly see the source of the extra traffic. Before patches, there was mostly just writes going to this filesystem. After patches, the write rate is about the same but we now have a significant read rate as well, equal or greater than the write rate. So now I just need to find out what patch(es) maybe causing an extra read for a write process.
Again, any help is greatly appreciated, I'm working with HP support as well.

[RAC_1]

Was there any changes in the way this temporary area is mounted?? What FS?? Was there any chnage in buffer cache settings?? What you use static, dynamic buffer cache. Also look at the patches that adresses "buffer cache"

Also one thing to check is, was there any chnage to application??

Anil

[Bernie Vande Griend]

No change in application nor in how filesytem is mounted.

Here is the filesystem on box #1:
/dev/vg01/lvol2 /work1 vxfs rw,suid,largefiles,delaylog,datainlog 0 2

box #2:
/dev/vg02/lvol7 /tmp vxfs delaylog 0 2

Buffer cache is dynamic with 5% min 10% max.
Good idea to check those related patches.

[Bernie Vande Griend]

HP Support could not help us find a related patch either and buffer related patches were not found. Hp believes it was more our application being triggered by something it should not have, but I guess that really doesn't help us much. We ended up using Ignite to bring both systems back to their previous patch levels. However, we're still going to have to find the trigger if we ever want to patch them again. We're hoping we can use a debugger with our app to find where the extra reads are coming from. If anyone else has an idea let us know.