Mistery with my backup on Fridays

[Valkiria Santos]

Hello everybody:

I?ve beem serious and wierd problem with my backup on Friday. I usually start my backup of oracle on Fridays at 23h , after 8 minutes I recieve an error after the list of files backuped: Unexpected write error (errno: 5).Archive aborted.
In syslog.log its logged an critical error from ems monitor about tape drive, in the same time of backup.We changed the tape many times, but don?t solve. But what is really strange is this error just occur only Fridays at the same time!!! On the weeks this do not occur!
The backup don?t is started from crontab, its a script with cpio command. What you suggest? I?ll do some tests...
Can you help me?
Thanks
Valkiria

[Edward Alfert_2]

many companies do different types of depending on the week.

could it be that you do incremental backups monday-thursday and a complete backup on friday... that might explain why things are different on friday... but doesn't explain what specifically is causing it.

can you track it down to 1 file? what is the last file it is able to backup successfully? The file that fails, is it greater than 2gb (cpio cannot backup greater than 2gb).

just a thought

[G.Kumar]

hi,

This can be a Tape drive problem.Are you taking full backup on fridays?
My guess it may be writing some big files on fridays that time gives the error.Incrementel back up may not be reading big files.
Check the drive with some manual backup commands

thanks

GK

[A. Clay Stephenson]

Since this is occurring only on Fridays and I assume you have other backups running normally at the same time other days of the week, I would first examine all the crontabs in /var/spool/cron/crontabs and I woul;d also examine any possible atjobs.

What I suspect is going on is some sort of intrusive device testing routine is being executed at that time.

Hope this helps, Clay

[Bill Hassell]

errno 5 has only one meaning: I/O error. Since cpio (and tar and dump, etc) cannot handle a tape error, they will abort. There is no status provided by these old utilities for the exact nature of the tape error. You'll have to check through your logs to see if any diagnostic errors are being logged. If it occurs only on Friday, is this always the same tape? (yes? use a new tape).

You may want to use the mt option: status to check if the tape is loaded (note: mt status is only available on 11.0 and higher)

[Valkiria Santos]

Well, I don?t have process starded from crontab using tape drive and I don?t have file bigger than 2GB.

thanks,
Val

[Xavier Gutierrez]

Can you tell us what model of tape drive are you using?

Be aware also that maximum capacity is not always met. These devices compress data to a MAXIMUM 2:1 RATIO, but this depends on the type of files you are archiving, if those files are already compressed or not, etc.

Seems that you are reaching the end of tape. I think cpio can't manage more than one tape.

Regards

Xavier

[Celso Medina Kern]

Hi Valquiria,

Did you check if the problem is not being generated by a bad tape used on fridays only?

if this drive works ok in other days it is probably good. Another possibility is some environmental issue, like lack of afterhours temperature control. Tape medias are sensible to it.

As already known, cpio just aborts when an I/O error arises, and errno 5 is write error. It is most likely a media error.

Good luck, tchau.

Celso

[A. Clay Stephenson]

Hi Valkiria,

This is very puzzling but I suspect your tape drive. You say you've changed tapes; I assume you've cleaned the drive.

I think we need to isolate the problem to either a problem with your script or with your hardware/media.

Here is what I suggest you do:
Write about 1GB of data to the tape drive using this method:

dd if=/dev/zero bs=64k count=16384 of=/dev/rmt/0m (or whatever - for this test non-compressed is better)

If you do not have a a /dev/zero create one by 'mknod /dev/zero c 3 0x000003'. This will supply an unlimited stream of ASCII NUL's.

If this test is ok then writes to the tape drive are ok. I would then modify your backup script so that the output device is /dev/null and run your script. It's the death at 8 minutes into your backup that I find most puzzling - that should no be long enough to fill up a any tape. If nothing shows up in these tests then I would do the write test at the same time you normally do your backup.

Regards, Clay

[Darrell Allen]

Hi Valkiria,

I'm sorry I don't have an answer. I'm just trying to "think outside the box" in
hopes we can help you find the answer.

It sounds like you are running the script manually at 23:00 on Friday evenings. Is that correct?

Can you run the same script successfully at other times? Using the same tape that you used on a Friday when it failed?

Do you use the same script to make backups during the week as you use on Friday?

Are you running the script as superuser?

Is the tape being rewound before writing?

Are you using the no-rewind device file?

Could someone or another process be writing to the same tape before you use it on Friday night?

Is the drive an external drive that could be having the power shut off because
someone is leaving for the weekend? (Don't laugh, I've worked at a place that actually had someone who threw the circuit breaker to the lights in our small computer room. I'm glad the lights were the only circuit breaker they had access to.)

Are you on site with the server when you run the backup?

Immediately after the backup fails, can you successfully write something to the tape?

Darrell