data archive

[maliaka]

I need to keep 6 months of data and archive the 7th month everytime the data grows older than 6 months.
10 points for the best solution!

[Dennis Handly]

Are you talking about a file, a few files, a directory tree, a filesystem, database or a whole system?
How much data, in Mb? And archive to what?

[maliaka]

Bunch of directories and files under one parent directory. I'm using a tool to do the archive but I have to enter the month so I just need help with the logic to know that the data are older than 6 months.

[James R. Ferguson]

Hi:

This will find files older than 180-days and copy them to another directory. Directory heirarchies are duplicated as necessary:

# cd /srcpath
# find . -type f -mtime +180|cpio -pudlmv /dstpath
# find . -type f -mtime +180 -exec rm {} +

The second 'find' removes the files that met the criteria for copy.

Regards!

...JRF...

[Dennis Handly]

>I just need help with the logic to know that the data is older than 6 months.

On a file by file basis?
JRF's find command can list just the filenames:
find . -type f -mtime +180

Note: That is any file not modified in 180 days.

[maliaka]

Sorry guys but I'm having trouble explaining what I need so let me try again.
Let's say the timestamp of the parent directory has Dec., the logic will automatically run the archive tool and pass june as the month to archive.
I'm just having difficulty creating that logic.

[Dennis Handly · Accepted solution]

>Let's say the timestamp of the parent directory has Dec., the logic will automatically run the archive tool and pass june as the month to archive.

It's clear now. :-)

Basically you can use fancy smancy calendar scripts like Clay's caljd.sh:
http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1157244

Or you can use a simple shell case to look up the month string and "subtract" 6 months:
mon=$(ll -d $directory | awk '{print $6}')
case $mon in
Jan) mon6="July";;
Feb) mon6="August";;
...
Dec) mon6="June";;
esac

archive_tool $mon6

[James R. Ferguson]

Hi (again):

Here are two rigorous variations of my original theme:

# cat .arch1.sh
#!/usr/bin/sh
typeset SRCDIR=/tmp/dummydir1 # change
typeset DSTDIR=/tmp/dummydir2 # change
typeset MYLIST=/tmp/mylist.$$
trap 'rm ${MYLIST}' EXIT

cd ${SRCDIR} || exit 1
find . -type f -mtime +180 -print > ${MYLIST}
cat ${MYLIST} | cpio -pudvm ${DSTDIR}
rm $(<${MYLIST}) 2> /dev/null
exit 0

# cat .arch2.sh
#!/usr/bin/sh
typeset SRCDIR=/tmp/dummydir1
typeset DSTDIR=/tmp/dummydir2
typeset MYLIST=/tmp/mylist.$$
trap 'rm ${MYLIST}' EXIT

cd ${SRCDIR} || exit 1
find . -type f -mtime +180 -print > ${MYLIST}

while read FILE
do
DIR=$(dirname ${FILE})
mkdir -p ${DSTDIR}/${DIR}
cp -p ${FILE} ${DSTDIR}/${DIR}
done < ${MYLIST}

rm $(<${MYLIST}) 2> /dev/null
exit 0

Regards!

...JRF...

[maliaka]

Dennis,

You got it!
Now, it gets a little complicated with the year.

From your first example:

Jan) mon6="July";;

How can I get the previous year?

[Dennis Handly]

>it gets a little complicated with the year. How can I get the previous year?

That's an issue. Also, by using ll on a directory, you get the time of the last modification. So if nobody is fiddling in it, the time should be the time the last file was created.
Also note, if the time on the file is over 6 months old, it is in the Mon Day Year format. So are you going to run this tool on directories that are less than 6 months?

You could look for the ":" and determine it is the current year.
Otherwise you could use the following on old files to get the year:
mon=$(ll -d $directory | awk '{print $6}')
year=$(ll -d $directory | awk '{print $8}')
case $mon in
Jan) mon6="July"; (( year -= 1));;
...
Jun) mon6="December"; (( year -= 1));;
Jul) mon6="January";;
...
Dec) mon6="June";;
esac