Disk Backup runs slow.

[Uday_S_Ankolekar]

Here is the scenario,

Oracle 8i running on N-Class (HP-UX 11). around 300 Data files

We run HotBackup every night.
The perl script does following things.
Take each data file and copy into a different dirctory and then compress it and then go to next datafile ... This usually runs for 5 hours but this duration increases everyday (Today it is running for 15 hrs.!) But if we bounce the box it will run for 5 hours again and grows slowly day by day.
I want to know why this process keeps on increasing and why reboot fixes it.

Thanks

Uday

[Helen French]

Hi Uday:

Just some thoughts: Do you have any error message/warning in syslog.log ? Anything regrading resource overflow ? Did you check the resource usage with glance ? Any kernel tuning required ? Any patches ? Anything wrong with the script ( some counter increase ?)

HTH,
Shiju

[harry d brown jr]

Uday,

Memory leak? Check your swap area before starting and then afterwards. Hopefully you have glance, if so "watch the process".

live free or die
harry

[A. Clay Stephenson]

Without much data to work with, this sounds like a classic memory leak problem. Have you noticed the Oracle background processes growing with time? I would start checking memory (and shared memory) with Glance.

[S.K. Chan]

More like "process leaking memory" but do not rule out "kernel leaking memory" too. The system or user processes memory leak can be quickly determine. In glance/perfview, you'll see PROC_MEM_VIRT increasing over time. You will need need to look at process-level data too and that's in PROC_MEM_RES and APP_MEM_RES. Bottom line you got a lot of investigating to do here.

[Justo Exposito]

Hi Uday,

Perhaps memory buffers, I reduce my memory buffers cache (dbc_pct) and the backups gone very slow then I increase it and gone fast.
Do you change this kernel param?

Regards,

Justo.

[Uday_S_Ankolekar]

I collect almost all information during this backup process but I'm not able to make out what is causing this.

I've almost all recommended patches on this box.
Attached here is glimpses of system status during backup.

[S.K. Chan]

Those are current snapshot and from a glance it looks fine. What you need to do is run some detection process over time to determine if there is any leaking ..
2 ways ..
=========
1) Use glance ..
# cd /opt/perf/examples/adviser
# glance -adviser_only -syntax proc_mem_leak -j300 > output.file
===> Run glance in adviser mode every 5 minutes with the use of the syntax in "proc_mem_leak".

2) This is from previous thread (thanks JRF)
# UNIX95= ps -e -o "user,vsz,pid,ppid,args" | sort -rnk2 | more
===> Look for growth in "vsz" over time.

[Thomas Schler_1]

Uday:

What about zombie processes? (I have a user who creates zombie processes with his perl scripts.) Maybe your backup scripts also leaves zombies that need resources for nothing.

Create a new account from where you only run the backup scripts, e.g. uid=dbbup, gid=dba. After finishing backups, have a look if there are some processes of this 'database backup user' still on the system (should not).

[Uday_S_Ankolekar]

Chan,

My output also have the famous UNIX95 output for top ten process sorting on vsz.

I would like to run glance in advisory mode. ( Throw some for light on this .Pls.)

Thomas,

There are no Zombies. and once back up finished system runs normally. Users complain slow responce during backup ( This could be due to compress command)

Justo,

dbc_max_pct is 8 and min is 5.