VDump/ADVFS Hang issues

[EAMatt]

We have installed a GS1280 M16 and have been having a number of issues with backups. The environment is configured as follows:

GS1280 M16
Tru64 5.1b-4 (PK6)
IBM DS4800 SAN with SVC 4.3
HP MSL6060 tape library with 4 x LTO4 drives, fibre connected through the SAN
4 HBA's, two dedicated to SAN storage and two dedicated to tape storage
Oracle 9.2.0.8

The issue that I have is that when I perform a backup with the database online we get and ADVFS hang. When the database is shutdown totally and no applications are accessing the server the backups run without error.

The issue is that the backup runs well then for no known reason one of the vdumps stalls. The Vdump task starts using 99.9% of resources on the processor it is running on and you cannot access the volume that the VDump task is backing up. Other vdumps that are backing up other volumes remain unaffected and complete normally. It is not a consistent tape drive that is having the issue and the tape drive that it happens most regularly on has been replaced. It appears to happen on the data volumes that are most heavily accessed by Oracle.

We use vdump to perform the backups. We shut the database down, produce a cloneset using ADVFS tools, bring the database and applications back online and then use VDump to put this to tape. This same tape library and backup method worked well on our ES80. Our ES80 was running a lower patch level (PK5) and an earlier Oracle version and all hardware was SCSI connected with the exception of the tape library.

I have a theory about some known issues with Oracle Direct IO and Tru64 that were resolved in the latest patch kit, but would like to hear if any one has had similar issues and how they were resolved or if they have any ideas as to what might be causing this problem.

So, to pre-empt some obvious answers...Oracle cannot be upgraded to the latest versions due to compatability issues with our application that runs on the server. The system has had the latest firmware, drivers and patch kits applied.

I plan on using more modern methods to backup our system in the very near future, but need to get this issue resolved first.