tar : end of tape

[Monishk]

Hi ,
when i do " tar -cvf /dev/rmt/1m *all files and dirs.in a filesystem* " ; then finally i get a message that "tar: end of tape please enter device/file name .." . i actually needed to know what does that mean ..? is it that only "1m" is full and other "0m,1mnb , 1mn etc." on the tape are still usable to backup or archive some more data using tar..? In other words i also need to know what does 1m , 1mnb , 0m etc. inside /dev/rmt/ work for. i am using an ultrium 200gb tape for backup.

Awaiting your suggetions.

Thanks
Monishk

[Yogeeraj_1 · Accepted solution]

Hi,


On all systems,
/dev/rmt/1m
refers to the 1/4 inch tape drive. 'rmt' mean 'raw magnetic tape' and '1m' is unit #1 with medium density.

The 8 mm tape is accessed as either /dev/rmt/0l (low density, 2.3 GB operation) or /dev/rmt/0m (medium density, 5.0 GB operation).

Normally, the tape drive rewinds the tape after a complete operation. Simply appending an 'n' on the end of the device name specifies the device as a no-rewind drive: /dev/rmt/0mn specifies 'no rewind on raw streaming tape # 0, medium record density'.

This permits multiple files to be placed on one tape cartridge.

What is the size of the total number of files that you are trying to backup?


kind regards
yogeeraj

[Monishk]

HI

I am not sure of the exact size but i suppose it is less than 200Gb which is the tape capacity ;It is very large though. So is there any way by which i can continue to backup the rest of the contents using the same tape despite it giving me "tar:end of tape" . If yes than please let me know about the command and the way i use it this time on .

Also, one more doubt i had is that if i give tar -rvpf /dev/rmt/1m after it has backed up 1 whole filesystem using tar -cvf /dev/rmt/1m as given in my last mail , then it takes hours and hours to actually start the appending ...is the reason for this , is that giving "1m" means it has rewinded back to the start and to append the next filesystem data using rvpf it has to traverse to the last end point of the first archive created using -cvf and then start appending . If this is the case then if i use tar -rvpf /dev/rmt/1mn instead of 1m then will it avoid this time taken by the tape for rewinding back fully untill it starts the actuall appending for the next file contents.

Thanks
Monish

[Peter Nikitka]

Hi,

the message 'end of tape' means there is no more space on the medium to store anything. The tar-command requests what it needs: the mounting of another - different - medium.

But your command makes me wonder; keep in mind:
tar cf /dev/tape dir1 dir1/subdir

will put dir1 on the tape, INCLUDING recursivly all files and dirs below dir1. The second parameter will include dir1/subdir1 a second time.

mfG Peter

[Bill Hassell]

tar is a very primitive tool and cannot use more than one tape. When it reaches the end of the tape, you have to start over and reduce the number of directories you are saving to tape. The various device files in /dev/rmt refer to additional tape drives which may or may not be connected. tar cannot use additional tape drives in a single backup. Use this command to display the device files and their special features:

lssf /dev/rmt/*

It's important to know that tape capacities are inflated by 100% by all manufacturers. The native tape capacity is sometimes listed and it is the only dependable value. The device file 1m will turn on compression and perhaps some of your data will be compressible, perhaps not.

Now appending one backup onto another not only takes a VERY long time, it is the most dangerous technique there is for saving data. It is very easy to make a mistake and destroy the entire tape with no possibility for recovery. With very careful scripting and WEEKS of testing, you may be able to get multiple archives on the same tape but the risks far exceed the minor savings in tape media. And you cannot get a table of contents without hours of reading the entire tape, again with a special script. Yes, the 1m device file rewinds when tar exits while 1mn will leave the tape at it's current position. tar is so primitive that it does not control the tape at all -- it just writes.

Never use tar for large backups. Instead, use fbackup or a commercial backup program. Backup programs are like insurance -- if you have cheap insurance, it may not payoff when you need it. fbackup will handle multiple tapes and will save any file size. It can also recover from tape errors and display a complete index of all the files on all the tapes in just a few minutes. fbackup is aware of tape drives and the very first step for the tape is a rewind. fbackup prevents appending specifically to avoid all the previously mentioned issues but also to provide an accurate index to all the files at the front of the tape.

fbackup is HP-UX proprietary so it will not exchange data between non-HP-UX machines. But as a backup tool, it is the best. Be sure to use a config file for maximum performance:

blocksperrecord 4096
records 64
checkpointfreq 4096
readerprocesses 6
maxretries 5
retrylimit 5000000
maxvoluses 200
filesperfsm 2000

---------------------------------------------

Example for a complete backup starting at / (root):

fbackup -i / -v -c config-file -f /dev/rmt/0m

Display the tape header with dates:

frecover -V - -f /dev/rmt/0m

Display the table of contents:

frecover -I - -f /dev/rmt/0m

[Monishk]

Hi

Thanks a lot for that useful data. I would like to know 1 more thing . Suppose the filesystem tree i want to backup has a directory "A" on which i have mounted another directory "B" present in another system ,in which all data of Dir. "A" is stored to avoid space crunches on the server in which Dir "A" exists . ANd suppose i dont want to backup The whole of Dir "A" , then what is the option i can use to override backing up this "Dir A" and continue with the backup of the rest of the files before and after Dir"A" .

PLease let me know the options for this w.r.t both fbackup and tarring .

Thanks
Monish

[Peter Nikitka]

Hi,

ok., if I understand right, you have something like
A/B/...
A/C/...
A/D/...

and you do not want to backup A/B/... .
I see two possibilities:
1) explicitly name all files/dirs under your parent directory for backup:
cd A
tar cf /dev/tape C D ...
2) exclude explicitly one/some files/dirs (ksh/posix shell only):
cd A
tar cf /dev/tape !(B)
The above pattern just exludes 'B' at expansion

tar cf /dev/tape !(*noback*)
This pattern will report everything but names including the string 'noback'.

mfG Peter

[V. Nyga]

Hi,

with 'man fbackup' you get the informations:
-i means include
-e means exclude
from backup.

You can also use SAM to configure a backup with fbackup and you can create an automatic backup which will be done once a week or once a month.

With 'tar' I believe you can use a 'find' and '|' command to do this (more details you can get with a search in this forum), but as said, fbackup is newer and more comfortable.

HTH
Volkmar

[Bill Hassell]

To answer your question, tar knows nothing about network mounted filesystems so if you backup /A then you will get /A/B and all other subdirectories. This is true for all primitive backup programs such as tar, cpio, pax, dump, etc. The workaround would be to explicitly select the directories you do want like this:

tar cvf /dev/rmt/1m /A/C /A/D /A/X /A/file1 /A/file2

and so on. The command line will likely be quite long and you must be careful not to select /A/B which would be the case if you included /A/* to get all the files in /A.

However, fbackup will NOT backup and network (NFS) filesystems unless you explicitly use the -n option. So running fbackup like this:

fbackup -i /A -v -c config-file -f /dev/rmt/1m

The -i /A will backup all files and directories except those that are located on another machine (on the network).

[Monishk]

Hi

I just saw the man for fbackup and found a
-e option which can be used for defining exceptional files or directories which should not be backed up . SO fbackup can have exceptions in it . BUt yes i dont think submitting exceptions for files or dirs. , which should not be backedup from the backup tree , is not possible in dump , tar or ipio etc.

THanks
Monish

[Peter Nikitka]

Hi,

was something wrong with the solution, I provided?

mfG Peter

[V. Nyga]

Peter,

we are not top enough to get pts.

V.

[Monishk]

Hi Nyga

Actually i got an error when i tried backing up using (!)before directories which i did not want . but still i am keen to know if there is any way to give exceptions in tar , for dirs. which should not be backed up..?

THanks
Monish

[Monishk]

wanted to know a couple of more things :

1) (A) how to do incremental backup using fbackup. Please let me know all the paramters, options and procedures to do the same.

(1)(B) how does fbackup know what are the files previously backed up in the last full backup.Is there any database which fbackup uses to record all these changes OR is it tape specific and we need to use the same tape everytime (while douing incremental) .

(2)How do we restore the whole backup (incremental as well as full) done using fbackup .please let me know the whole procedure with the required options and parameters.

(3) please let me know how to use SAM to configure an automatic backup using fbackup.

Thanks evryone for all your suggetions . They have really helped us .

Regards
Monish

[V. Nyga]

Hi Monish,

first - thanks.

I only can tell you question 3:

You can start SAM? -> 'sam'

There you have to click 'Backup and Recovery' - 'Automated Backups'.
Then under 'Actions' choose 'Add an Automatic Backup'- 'Local Backup Device' (if your tape drive is locally), else choose 'Remote ..'.
'Specify Backup Device' searches for the tape and gives you the Hardware Path.
Choose your tape drive, then you can choose the device (should be '/dev/rmt/1m' like in your tar command).
Then click 'Select Backup Scope' to choose the directories (or files) you want to backup.
Here you can choose 'Specified Files' (then you have to specify the directories you want to want to backup (included Files) and which one don't (excluded Files - optional). Also you can choose to backup over NFS mounts (Cross NFS Mounts) or not.
Or you choose 'Local File Systems only' - then you can't exclude local files or directories, but no NFS mounted files are backuped.
Or you choose 'All File Systems', then all directories are backuped.

Then you 'Select Backup Time' - here you can choose 'Incremental Backups' or not.

'Set Additional Parameters' is for an index log and for mailing the results to 'root' or not.

You can try all possibilities without destroying anything as long as you don't say 'OK' at the end.

Everything is very simple and clearly, I think.

Have fun
Volkmar
P.S.
Also here it's possible that your tape is too small, but with 200 GB I don't think so.

P.P.S.
I never had problems to write to a second tape when tar reached the end of the first one. Only put in a new tape and enter the same device as before.
Everything was readable then.

[Peter Nikitka]

Hi,

which error did you get, using the filename pattern !(xx) ?
I'm in a ksh here, but HP-UX posix shell (should) behave same. Here is an example output:

ef3nip00@forth[1011] mkdir new; cd new
ef3nip00@forth[1012] touch A B Bb c d eee
ef3nip00@forth[1013] ls
A B Bb c d eee
ef3nip00@forth[1014] ls -d !(A)
B Bb c d eee
ef3nip00@forth[1015] ls -d !(B)
A Bb c d eee
ef3nip00@forth[1016] ls -d !(B*)
A c d eee
ef3nip00@forth[1017] ls -d !(?)
Bb eee

HTH.
mfG Peter

[V. Nyga]

Hi Peter,

maybe he has another HP-UX?
For me (11.11) 'ls -d !(s)' doesn't work, too.
Message:
Badly placed ()'s.

HTH
Volkmar

[Peter Nikitka]

Hi,

I wrote
>>
I'm in a ksh here, but HP-UX posix shell (should) behave same.
<<

Since I live on a SUN here, I did this in ksh - please try under ksh@HP-UX!
Perhaps there is a difference between ksh and posix shell here - and it behaves NOT the same...

mfG Peter

[James R. Ferguson]

Hi:

With regard to your last set of questions:

> 1) (A) how to do incremental backup using fbackup.

The manpages for 'fbackup' actually have a reasonably good discusssion of incremental backups. A full backup is denoted by the level number zero (0) whereas incremental backups can have levels of 1-9. With incremental backups, only files in the graph that have been modified since a previous backup of that graph are selected. The timestamp of the last backup is known from a "dates" file stored in the '/var/adm/fbackupfiles/' directory which *you* must create when you setup your backup schemes.

> (1)(B) how does fbackup know what are the files previously backed up in the last full backup. Is there any database which fbackup uses to record all these changes OR is it tape specific and we need to use the same tape everytime (while douing [sic] incremental) .

For a full backup, the graph (file) specification serves to specify the files to be backed up. For an incremental backup, if a file's modification time is found to be later than the last incremental backup as recorded in the '/var/adm/fbackupfiles/dates' files; or the file's inode change time ('ctime') is more recent, then the file is backed up.

'fbackup' always rewinds a tape before it uses it. Therefore, there is *no* way to stack multiple backups on a single tape! When restoring, with 'frecover' you use the appropriate tape for the appropriate timeframe of your restoration.

> (2)How do we restore the whole backup (incremental as well as full) done using fbackup?

You begin by restoring (with 'frecover') the last good full backup and then restore each successively more recent incremental backup.

An example of a full backup specification with a graph file called "full" would be:

# fbackup -f /dev/rmt/0m -0 -u -v -g /var/adm/fbackupfiles/graphs/full -c /var/adm/fbackupfiles/config

The graph file specifies the paths and/or files to (i)nclude and to (e)xclude. The file looks like:

i /
e /cdrom

You can include whole directories and then exclude individual files therein. For instance, I could (i)nclude all files and directories within the '/etc' directory but (e)xclude '/etc/mnttab' for example.

Be sure to specify your own configuration file for 'fbackup'/. I place mine in '/var/adm/fbackupfiles' and name it 'config' as shown. The default values that you will otherwise obtain are out-dated and will generally give very poor performance during backup and recovery. Build a configuration file that looks like:

blocksperrecord 4096
records 64
checkpointfreq 4096
readerprocesses 6
maxretries 5
retrylimit 5000000
maxvoluses 200
filesperfsm 2000

The manpages for 'fbackup(1M)' document the default settings which is what you will get in the *absence* of an explicily defined set.

These parameters are recorded onto the actual backup tape and are thus used for a 'frecover' session too.

Checkpoint records allow the salvage of a backup when a bad tape spot is detected, since the records contain information about the file being backed up. The 'filesperfsm' parameter controls the frequency with which Fast Search Marks (FSM) are written. Both checkpoint and FSM records affect performance. FSMs take a tape drive out of streaming mode thereby adding to backup time. Conversely, however, FSMs improve the time it take to recover a file from tape.

In general, if your backup consists of a high proportion of small files, increase the value for 'filesperfsm'. If your backup consists of a high proportion of large files, then decrease the 'filesperfsm' value.

Regards!

...JRF...