General
cancel
Showing results for 
Search instead for 
Did you mean: 

Needed: Large and Fast system backup

kenny chia
Regular Advisor

Needed: Large and Fast system backup

Hi
I need to backup a linux system daily. There are many files and directories (about 2G total) and creating a tar.gz file from them everyday will take up too much CPU resources.

I am thinking of this
1) Create a tar.gz from all the files to be backup

Run a daily program that
1) Search for new or modified files. These files will overwrite or be appended to the tar.gz file
2) Search for deleted files. These files will be deleted from the tar.gz file.

Any ideas on how to implement 1) and 2) ?

Thanks
All Your Bases Are Belong To Us!
4 REPLIES
Stuart Browne
Honored Contributor

Re: Needed: Large and Fast system backup

If creating a tar/gz file is too much CPU over-head, then you've got a seriously under-powered server there. Given that I can do this on my old PPro's without creating much of a serious dent in usage ability, I'd like to know what you're doing to this poor machine!

Tar it's self cannot do file removals based on removed files, so you'd have to create your own routines of which say "is this file still there?" which would be even worse CPU overhead.

For that you it's probably better you use something like 'rdist' or 'rsync' (which are traditionally used for remote-system backup of a given file-set).

As you seem not to be concerned with creating 'backups' on the same system, one of these would be ideal. You could then tar/compress tar of the backup structure for a decent snapshot.
One long-haired git at your service...
Balaji N
Honored Contributor

Re: Needed: Large and Fast system backup

this could be done by maintaining a list of files with their md5 sums. every time generate a list of all files with their md5 sums and if there is a change between these two, then the file is modified and take a backup.

i guess this is a crude way of taking backup and would suggest using rsync.
check out the link for more info.
http://samba.org/rsync/index.html
-balaji
Its Always Important To Know, What People Think Of You. Then, Of Course, You Surprise Them By Giving More.
kenny chia
Regular Advisor

Re: Needed: Large and Fast system backup

rsync looks like a good idea but it doesn't seem to work on tar.gz files. Anyway if I can get a large hard drive. I might implement it without tar.gz
All Your Bases Are Belong To Us!
Balaji N
Honored Contributor

Re: Needed: Large and Fast system backup

yes. that would be the best way to go. rsync directly the files instead of the .tar.gz approach on a new drive.
-balaji
Its Always Important To Know, What People Think Of You. Then, Of Course, You Surprise Them By Giving More.