1747992 Members
5355 Online
108756 Solutions
New Discussion юеВ

Re: Compressing files

 
Aziz Zouagui
Frequent Advisor

Compressing files

Greetings,

I have and oracle database 8.1.7.3 that generates a lot of archive logs, each one is 200 Mb. I have croned a job every 15 minutes that checks the directory and if it finds any new files, it will compress them.

Now, here is the question:

If the Oracle Archiver process happens to start archiving a file but has not yet completed writing the entire content of the file to disk, if my croned compress job kicks in, will it be able to compress that file even if Oracle has not finished with it yet ? I am a little paranoid, because if the system lets the compress to go through, then I may have corrupted the archive logs.

In other words, does HP-UX prevent other processes from accessing the file ? Bear in mind that compress reads the file and writes to a new file with *.Z and removes the original one.

Any thoughts would be helpful.

Thank you.-
15 REPLIES 15
A. Clay Stephenson
Acclaimed Contributor

Re: Compressing files

The simple answer is no; that is up to you.
If it ain't broke, I can fix that.

Re: Compressing files

You can of course use the fuser command to see if oracle stil has the file open. Once oracle has finished writing out the file it doesn't open it again so you can safely compress it.

HTH

Duncan

I am an HPE Employee
Accept or Kudo
A. Clay Stephenson
Acclaimed Contributor

Re: Compressing files

One thing that you could do is determine if a file has been modified within some period of time. If not then it is safe to compress it. Here is one method:

FNAME="myfile"
SECONDS=600
fileage.pl -m -s ${SECONDS} ${FNAME}
STAT=$?
if [ ${STAT} -eq 0 ]
then
echo "File: ${FNAME} has not been modified in the last ${SECONDS} seconds; it is safe to compress."
else
echo "File ${FNAME} has been modified; not safe to compress."
fi

I've attached fileage.pl; invoke fileage.pl without arguments for full usage.


If it ain't broke, I can fix that.
John Carr_2
Honored Contributor

Re: Compressing files

Hi

I have just read the man page for compress and it says this:

Access Control Lists
compress retains a file's access control list when compressing and expanding data.

I think this is saying it basically locks the file could one of the guru's comment on this.

cheers
John.
A. Clay Stephenson
Acclaimed Contributor

Re: Compressing files

Sorry, that dog won't hunt. ACL's are a set of extended permisssions. All that is saying is that both the compressed and uncompressed forms of the file retain the original's ACL's (if supported by the filesystem and provided that the ACL's have been set for the file in question.)
If it ain't broke, I can fix that.
harry d brown jr
Honored Contributor

Re: Compressing files

John,

If you are compressing FILEA and some process is still writing to it, and the compress completes, then the original file is DELETED, meaning the process still writing to FILEA won't be writing to squat (actually never-never-land).

live free or die
harry
Live Free or Die
Aziz Zouagui
Frequent Advisor

Re: Compressing files

Hey guys,

I appreciate all the input I am getting.
I like the fuser suggestion, I will have the job check the fuser before compressing the file, if there is a process list next to the file, then either sleep for a while and try again or abort to the next file and come back to the busy one later.

Please if there is a better suggestion, do not hesitate to update the forum.

Thank you for the feedback.
Jeff Schussele
Honored Contributor

Re: Compressing files

Hi Aziz,

I think the question boils down to...Why are you compressing them?
If it's disk space...Then as Clay puts it "That's dog's hunting way too much!".
In that case you really need to extend the VG to hold proper log count/length/size.
If it's to backup & save tape space then I'd recommend you mirror the VG & lvsplit the mirror out prior to compressing & archiving then lvmerge it back in. If redundancy is a major concern & you're nervous about having just one copy for *any* time period then keep 2 mirror copies & you'll always have at least 2 copies on-line.
If you just want to "roll" the logs, I believe Oracle handles that as well. I think you can mv the file to a relevant, time-stamped filename prior to compressing it & Oracle will start a new log...mind you I'm not an Oracle expert but I'm sure one will correct me if I'm mistaken.

Rgds,
Jeff
PERSEVERANCE -- Remember, whatever does not kill you only makes you stronger!
John Carr_2
Honored Contributor

Re: Compressing files

Hi

as A.Clay and Harry make clear the file is not locked presumably even if you fuser before start of compress another process can effectively attach to file during compress and think it is writing to it.

I would be tempted to rename the file before compression commences to ensure this cannot happen.

As Jeff points out the real solution is to pick another alternative.

cheers
John.