Strange Phenomenon

[Faizer]

My configuration : 3 servers connecetd through a SAN to a va7410.Each server has its own dds4 tape units.
Restored my Oracle 11.0.3 application to server 1
Server 1: Action 1:
Immediately I did a SAM backup [cold ]of the mount points of the system. size 54gb Time taken 1 hour and 45 minutes using Ultrium
Action 2:
No user activity done. scheduled backup [cold ] of /data2 and /Oracle : size 24.5 gb. on a dds4 Tape it took 1 hour.

Server 2

The history of backing up my mount point /ora9i.
First time after installation of Application [ 9idatabase]. size 13.5 gb. Backup time approx. 30 minutes. using DDS4.
Second time after activities on the application. size 15.3 GB. Backup time approx.2 hours and 20 minutes using dds4
Third Time :after activities on the application. size 15.4 gb . Backup time approx 2 hours and 58 minutes using dds4.

On Server #3
Application installed is Oracle ERP 11.5.8 on the mount point /oracle1 This consists of 2 directories namely oracle and applmgr.

Oracle directory consists of data files and database binaries: size approximately 23gb
Applmgr consists of application binaries size approximately 21gb
Backup of oracle takes approx 2 hours using dds4
Backup of applmgr takes approx. 9hours using dds4
Can anyone let me know why the backup time is increasing with increase of activity.
best regards
Faizer Jameel

[Steven E. Protter]

Our backup and job run times on our SAN typically get longer in the evening. It is because there are a lot of backups and batch jobs in the evening. The busiest time for our san is 10 p.m. through 2 a.m. local time.

Its a simple throughput issue, and has no relationship to whether the database is up or down.

SEP

[Tim D Fulford]

I may or may not help here... we had "issues" with backing up from VA (dont get me wrong VA's are good storage devices, it is just that they are configured in a certain way).

1 - VA7410, stripes the data across ALL disks within a redundancy group. Thus by backing up one LV you will be hitting all the disks.

2 - From 1 the VA will be serving the other IOs in a similar manner, so increased activity willl mean slower more IO to each of the disks.

3 - I know Ultrium can backup faster than VA can present the data. a Ultrium 215 will do 15MB/s writing but with compression this could be anything up to the buss speed, say 90MB/s (bus dependant). I do not know what dds4 does but I think it will be similarly fast.

regards

Tim

[RolandH]

Hi Faizer,

the internal SAM backup use fbackup for backing up your files. If you now backup your files in a time your application is in use an files are locked, then fbackup tries 5 times to backup the file. This could be one reason for your increased time.

What is your backup device what you use?
fbackup is not supported with rewind magnetic devices like /dev/rmt/0mn. Use /dev/rmt/0m instead.

See also the fbackup man page !!


HTH
Roland

[blal]

Hi

One samll input for you,

There may be some application log directory .Oracle Application server writes a number of files in this directory .

Eventhough the file size of these files are very small, total number of files generated is very large.

Pls check this, if any such log file exists in your file system u can exclude that directory from backup.

Regards
baiju.

[Stuart Urquhart]

Bit of a wild shot. Does the backup time revert to short after a server reboot ? If so, you might want to look at patch PHKL_29049 :-

When large number of I/O requests are sent to the device and if this number exceeds the queue length limit of the device, the device returns QUEUE FULL status. To avoid frequent QUEUE FULL status messages from the device, the
driver will lower its own queue length so that the number of I/O requests sent to the device is reduced. The driver will increase the queue length when a certain number of I/O's have successfully completed at the current queue length. If the driver gets QUEUE FULL status when operating in queue length of 1 it will switch to untagged mode (no queuing). The driver will not return to tagged mode once this happens (even when the I/O load is reduced later). This results in slow I/O to the device and large avwait and avque values reported by sar(1M).

[Michael Steele_2]

Fragmented file system data. When first loaded file system data is mostly contiguous but over time it will fragment.

fsadm -F vxfs -d -D -e -E /file_system