topic Re: VMS Image backup in Operating System - OpenVMS

VMS Image backup

Jorge Cocomess — Fri, 24 Feb 2006 16:46:39 GMT

Greetings,

I am currently running an image backup on local SCSI drives on my VMS server. As for image backup, does it slow down the backup if you come across files that currently opened, even if you if the qualifiers /ignore=(label, inter)?? As I watched the backup log, I noticed the backup procedure slow down when it came across certain log files.

Should I skip log files altogether? If so, what's the qualifier to this?

Thank you in advance.

J

Re: VMS Image backup

Arch_Muthiah — Fri, 24 Feb 2006 17:32:30 GMT

Jorge,

Image backups should be performed with no interactive users on the system because of open file considerations.

Archunan

Re: VMS Image backup

Arch_Muthiah — Fri, 24 Feb 2006 17:34:38 GMT

Jorge,

Pls go thru this link which talks about "VAX/VMS 7.2 BACKUP/IMAGE behavior when a file is open"

http://groups.google.com/group/comp.os.vms/browse_frm/thread/75a3301e76ac309f/cf7b2d574f2c67ec?lnk=st&q=VMS+image+backup+open+log+slow&rnum=1&hl=en#cf7b2d574f2c67ec

Archunan

Re: VMS Image backup

Phillip Thayer — Fri, 24 Feb 2006 17:49:17 GMT

Jorge, I don't think that you can use any type of input file qualifiers on an Image backup so I don't think ypu can exclude any files from the image backup.

When you said you are looking at the log files, are you running a batch job and doing a type/tail on the log file or are you doing the image backup interactively?

Also, one other question, What is the configuration of the disk drives that you are backing up. Are they shadowed on the system (i.e. DSAnnn:). If you are backing up shadow sets then all you need to do is remove one member of the shadow set and backup it then place if back into the shadow set. You create a vulnerability to disk drive failure for a period of time that the shadow set is reduced for the backup but you will not have to stop any applications or users to get a clean backup.

Phil

Re: VMS Image backup

David Jones_21 — Fri, 24 Feb 2006 18:07:41 GMT

Continuously open log files are often the most fragmented files on a system due to the way they slowly allocate space. Backing up fragmented files can be a lot slower than backing up more contiguous files (more head movement and window turns). For those type of log files it is best if the application can set a large extend size.

Re: VMS Image backup

Arch_Muthiah — Fri, 24 Feb 2006 19:01:45 GMT

Jorge,

You can improve the backup performance by properly setting the process quotas for the process from which backups will be made.

WSQUOTA 16384
WSEXTENT Greater than or equal to WSQUOTA
PGFLQUOTA 32768
FILLM 128
DIOLM 4096
ASTLM 4096
BIOLM 128
BYTLM 65536
ENQLM 256

Our system mngr manula suggest above values, so make sure you have these approximate values (not exact). Ans as I said earlier, the best way to do the image backup is without any intercative users.

Also as you have SCSI cont, that automatically will do some
degree of optimization even if the files are open during the Backup operation in the way of orderily selecting the directory files and the reading the files from the optimised way.

Archunan

Re: VMS Image backup

Karl Rohwedder — Fri, 24 Feb 2006 19:18:46 GMT

As far as I remember, the correct quotas for a backup account are at least a little volatile over the years. At sometime the DIOLM couldn't be high enough and later on the saying was, that around 150 is enough.

See the current manual at:
http://h71000.www7.hp.com/doc/82FINAL/aa-pv5mj-tk/aa-pv5mj-tk.HTMl

Here is a snippet of our procedure which creates the backup related accounts (sorry I have only the dieted version available):

$WsMax=f$gets("wsmax")
$Virtual=F$Getsyi("VirtualPagecnt")
$ChannelCnt=F$Getsyi("ChannelCnt")
$wsquota=WsMax/2
$If (WsQuota.gt.32768) Then $ WsQuota=32768
$wsextent=WsMax
$Pgflquota=100000
$If (Pgflquota.lt.WsQuota+25000) Then $ Pgflquota=WsQuota+25000
$Fillm=128
$If (Fillm.gt.ChannelCnt-20) Then $ Fillm=Channelcnt - 20
$Diolm=150
$Astlm=1000
$Biolm=1000
$Bytlm=(512*Fillm)+(6*Diolm)+10000
$If (Bytlm.le.100000) Then $ Bytlm=100000
$Enqlm=1000

regards Kalle

Re: VMS Image backup

Allan Bowman — Sat, 25 Feb 2006 01:34:54 GMT

If you have some specific log files that are always open and get very large (and therefore fragmented), you may want to set those specific files to NOBACKUP (at creation time - you won't be able to do it while the files are open).

$SET FILE/NOBACKUP filename

This will make the IMAGE backup copy the file header, but not the data. If you need to back up the contents of the log file, you could possibly do it separately later and use IGNORE=NOBACKUP in order to backup the data.

Allan in Atlanta

Re: VMS Image backup

Jan van den Ende — Sat, 25 Feb 2006 04:36:31 GMT

David wrote:

For those type of log files it is best if the application can set a large extend size.

If fragmentation of growing files _IS_ an issue (which it usually is if there are growing files) you shoud modify the disk extend size.
NO need to modify any application, and can be done on the fly. Effective for any files created thereafter.

$ SET VOLUME /EXTEND=<512, or for bigger drives 1024 or 2048>

When the file is closed, any too large allocation is truncated back (as it is now),
so not much waste of diskspace.

hth

Proost.

Have one on me.

jpe

Re: VMS Image backup

Robert_Boyd — Sat, 25 Feb 2006 10:25:24 GMT

Jorge,

I'm curious as to what you mean by "the backup procedure slows down?"

Are you backing up to tape and you notice that the amount of tape motion decreases? Or are you noticing that the rate at which file names are being logged is decreased?

The reason I ask is that when BACKUP hits a larger file, the rate of logging will decrease since it takes longer to copy the contents of a big file than a small one. However, the rate of transfer of data to the output device usually doesn't drop significantly unless there is a very bad file fragmentation, or competition for I/O or compute resources on the system that would affect the performance of the process running BACKUP.

So, could you please explain what you mean about the procedure slowing down?

Thanks,

Robert

Re: VMS Image backup

Allan Bowman — Sat, 25 Feb 2006 14:30:08 GMT

The procedure can also appear to slow down when it hits a directory with thousands of files in it, or any time it is backing up a lot of tiny files (short .com files or empty data files).

Allan in Atlanta

Re: VMS Image backup

Wim Van den Wyngaert — Mon, 27 Feb 2006 03:53:55 GMT

Log files can be VERY fragmented. Backup has to read each part of it and the file can be scattered all over the disk if you have bad luck. So it can slow down your backup.

Wim

Re: VMS Image backup

comarow — Mon, 27 Feb 2006 11:04:41 GMT

I'd like to point out that the quota requirements have changed since the original recommendations.

Especially on SAN devices you need to reduce the DIOLM way down.

However, there has been considerable testing and the new recommendations are
here.

http://h71000.www7.hp.com/doc/82FINAL/aa-pv5mj-tk/00/01/117-con.html

Setting Software Parameters for Efficient Backups

Best of luck. I think the folks pointing out that backing up open files are probably badly fragmented is what is slowing you down, as opposed to just open files.

At some point you need to do an image backup - restore with the files closed.

Re: VMS Image backup

Jorge Cocomess — Fri, 03 Mar 2006 17:01:57 GMT

Sorry, I have been quite busy these days and not able to reply or make comments.

I appreciate all your comments and pointers. What I did for now is that I set log files to /NOBACKUP and off I go again.

Thanks everyone for your help on this backup issue.

J