Unable to verify data on DDS-3

[Philip Horan]

I have recently installed a new Sony SDT-9000 tape drive on Alpha 1200 running Digital Unix V4.0F. I purchased new DDS-3 media at the same time.

This morning after backup I have run command:

tar tvf /dev/rmt0h

Receive error:

tar: [offset 0]: No space left on device

Any ideas as to what might be causing this and what I can do to resolve?

Many Thanks,
Phil.

[Rob Leadbeater]

Hi Phil,

I wouldn't normally expect to see a "No space" error when reading data from a tape...

However your previous posts show that you've swapped tape drives recently. I'll take a guess that at some point a backup has been done pointing to a /dev/rmt... device that doesn't exist, which has consequently filled up the root file system.

Does the output of "df -k" show any filesystems at 100% ?

Cheers,

Rob

[Philip Horan]

Hi Rob. All that shows 100% is /proc (not sure what that is however) as it has 1024 blocks = 0 Used = 0 Available = 0 Capacity = 100% Mounted On /proc

You are correct regards previous backup. I would have thought the new drive would have the same lable rmt0h? Is this not the case? I did a backup on Thursday with the new device and all was OK (different type of backup)?

More of a windows man myself!!

What could I do to resolve?
Many Thanks,
Phil.

[Kapil Jha]

Hello Phil,

run dsfmgr -k
this will add device file for newly added tape drive and then use new device file to take backup.(ls -lrt /dev last files would be newly created one).

BR,
Kapil

[Philip Horan]

Hi Kapil. Unix did not like the command:
dsfmgr: not found
Regards,
Phil.

[Philip Horan]

OK. I removed entries for *rmt0* form /dev. I recreated using ./MAKEDEV tz1. Tape device shows up. However still get same error when executing tar tvf /dev/rmt0h?

Extract from daily backup script (not written by me) as follows:

[code]
RECOVERY=/appdb/recovery.log; export RECOVERY
TAPEDRIVE=/dev/rmt0h; export TAPEDRIVE
APPDBDIR=/appdb; export APPDBDIR
APPDBRECOVERY=/appdb; export APPDBRECOVERY
APPDIR=/app; export APPDIR

# Stop the BSCqueu.run routine (to free up RECOVERY), if it is running
#

echo `date` " Start of CRON processing"
/app/bin/AUTOBACKUP
echo `date` " End of CRON processing"
[/code]

The daily backup calls script AUTOBACKUP as follows:
[code]

set +x
#
# app Shell Command Script to renew the "RECOVERY.LOG"
# file in /appdb/ after a backup has been made.
#
# VERSION 9.1.2.1 - DBBACKUP 91/10/10 10:32:36

# !!!!! WARNING - date statement uses sccs format
# DO NOT USE GET (Use GETEDIT or Maintain manually)


msg1="app Users on System - Can not Proceed."

there=`ps -ae | grep -c "dbh010.r"`

if [ $there -gt 1 ]
then
clear
echo " "
echo $msg1
mail root < /app/bin/BACKUP_FAIL
echo " "
exit
fi
date=`date +%m%d%H%M`
RECOVERY_SAVE=$appDBRECOVERY/LOG$date.LOG
echo "\f\fapp backup procedure: $RECOVERY renewal"
echo "Recovery file $RECOVERY will be moved to $RECOVERY_SAVE"
#echo "\nConfirm Backup of Database and replacement of '$RECOVERY' : \c"
confirm=YES
#read confirm
there=`ps -ae | grep -c "dbh010.r"`

if [ $there -gt 1 ]
then
clear
echo " "
echo $msg1
mail root < /app/bin/BACKUP_FAIL
echo " "
exit
fi
case $confirm in
y|Y|yes|YES)
if test -r $RECOVERY
then
touch $appDIR/tmp/dbh.lock
cd /appdb1
tar cvB -C /appdb1 . -C /appdb2/appdb1 .
# find DS*.* -print | cpio -ocBvdum > $TAPEDRIVE
# Following added by KH to check for unsuccessful backup to tape
if [ $? = 0 ]
then
echo "\nBackup Successful \n"
mail root < /app/bin/TAPE_SUCCESS
else
echo "\nBackup Failed \n"
mail root < /app/bin/TAPE_FAIL
# exit
fi
# KH - end
mv $RECOVERY $RECOVERY_SAVE
if [ $? = 0 ]
then
echo "\nPrevious $RECOVERY now in $RECOVERY_SAVE\n"
touch $RECOVERY
chmod 777 $RECOVERY
chown appadm $RECOVERY
chgrp app $RECOVERY
rm $appDIR/tmp/dbh.lock
echo " \nNew $RECOVERY now created\n"
echo "\nBackup of Database Complete ..You May Resume Normal app operations now .\n"
mail root < /app/bin/BACKUP_SUCCESS
else
echo "\nUnable to copy $RECOVERY into $RECOVERY_SAVE\n"
rm $appDIR/tmp/dbh.lock
exit
fi
else
echo "\nUnable to find $RECOVERY \n"
mail root < BACKUP_FAIL
exit
fi
;;
n|N|NO|no)
echo "No action taken"
echo "\nDATABASE NOT BACKED UP !!!!!\n\007\007\007"
;;
esac
[/code]

Any help very much appreciated.
Phil.

[Pieter 't Hart]

What do you mean with "Tape device shows up"?
did you check the scsi-id on the drive matches id-0 (rmt0 )!
or did you now create an rmt1 !

[Philip Horan]

Hi Pieter. I followed instructions from previous post:
[link] http://forums12.itrc.hp.com/service/forums/questionanswer.do?threadId=1248318&admit=109447627+1216030720135+28353475
[/link]

scu show edt lists device as:
Bus: 0, Target:1, Lun:0, Type: Sequential Access

My boot file lists device as:
tz1 at scsi0 target 1 lun 0 (LID=0) (compaq SDT-9000)

Just ran: file /dev/nrmt0h

This has returned:
/dev/nrmt0h: character special (9/1027) SCSI #0 SDT-9000 tape #0 (SCSI ID #1) (SCSI LUN #0) offline

Can you assist?
Many Thanks,
Phil.

[Philip Horan]

I have tried: mt online. I receive:
/dev/nrmt0h online 1 failed: I/O error

Kind Regards,
Phil.

[Pieter 't Hart]

in the backup script i do not see any reference for writing to tape
looks like you're writing to a directory.
>>> cd /appdb1
>>> tar cvB -C /appdb1 . -C /appdb2/appdb1 .

also looks lik the os can reach the tapeunit, but the device is off-line.
with DLT drives after loading the tape you must put the unit-online. I don't think the SDT-900 has such a button.
try if if "mt online" command works on this version (we don't have V4.0 anymore so i can't test; mt - Magnetic tape manipulating program)

[Philip Horan]

Hi Pieter. The first script references the tape drive:

RECOVERY=/appdb/recovery.log; export RECOVERY
TAPEDRIVE=/dev/rmt0h; export TAPEDRIVE
APPDBDIR=/appdb; export APPDBDIR
APPDBRECOVERY=/appdb; export APPDBRECOVERY
APPDIR=/app; export APPDIR

It has always worked in the past?
Kind Regards,
Phil.

[Philip Horan]

Now if i type command: file /dev/nrmt0h I receive:

/dev/nrmt0h: character special (9/1027) SCSI #0 SDT-9000 tape #0 (SCSI ID #1) (SCSI LUN #0) errors = 0/1 97000_bpi

Phil.

[Pieter 't Hart]

Sorry Phil,
but you're mistaking.
the only statement refferencing output o the tapedrive is commented out

# find DS*.* -print | cpio -ocBvdum > $TAPEDRIVE

[Philip Horan]

Pieter I am confused. These scripts were written by a third party. Each morning I simply ran: tar tvf /dev/rmt0h this showed data written to the tape from the previous day?

From what I can see there are 2 CRON jobs:
[quote]
CRON

User mtms

50 19 * * 1-4 /mtms/bin/Nightly.jcl > /mtms/tmp/cron.out

00 23 * * 1-5 /mtms/bin/mailcron

00 17 * * 5 /mtms/bin/Weekly.jcl > /mtms/tmp/cron.out

00,15,30,45 00-17 * * 0-4,6 /mtms/usr/bin/monitor_BSC.sh > /dev/null

00,15,30,45 00-16,19-23 * * 5 /mtms/usr/bin/monitor_BSC.sh > /dev/null

00 00 * * 6 /mtms/bin/bscregen.sh

10 00 * * 6 /mtms/bin/mtmsextremove.dat

15 00 * * 6 /mtms/bin/mtmsprtremove.dat

20 00 * * 6 /mtms/bin/mtmswrkremove.dat

25 00 * * 1-5 /mtms/bin/logtiyup.sh

User Root:

15 4 * * * find /var/preserve -mount -mtime +7 -type f -exec rm -f {} \;

20 4 * * * find /tmp -mount -type f -atime +2 -exec rm -f {} \;

30 4 * * * find /var/tmp -mount -type f -atime +7 -exec rm -f {} \;

40 4 * * * find /var/adm/syslog.dated -depth -type d -ctime +5 -exec rm -rf {} \;

20 19 * * 1-5 /mtms/bin/kill_users.jcl

30 19 * * 0-4,6 /mtms/bin/rootcron.sh

30 23 * * 5 /mtms/bin/rootcron.sh

#0 3 * * 4 /usr/sbin/acct/dodisk > /var/adm/diskdiag &

30 2 * * * /mtms/bin/nightly1.jcl

# The next line stops bde

0 19 * * * /mtms/BDE/stop.sh

# The next line monitors and starts bde every 15 minutes

00,10,15,30,45 08-17 * * * /mtms/usr/bin/monitor_bde_up.sh > /mtms/BDE/monitor_log 2>&1

1 1 * * * test -x /usr/sbin/defragcron && /usr/sbin/defragcron -p >>/usr/adm/defragcron.log 2>&1
[/quote]

I have attached the scripts.
Many Thanks,
Phil.

[Philip Horan]

Still having problems. file /dev/rmt0h displays: /dev/rmt0h: character special (9/1026) SCSI #0 SDT-9000 tape #0 (SCSI ID #1) (SCSI LUN #0) offline

What do I need to do to get the tape drive back online and have the ability to read to and write from it?

Many Thanks,
Phil.

[Philip Horan]

Update:
mt online returns:
/dev/nrmt0h online 1 failed: I/O error

Regards,
Phil.

[Pieter 't Hart]

let's start at the hardware.
check scsi-jumper or switch-settings. is it id-1 ?
check termination on the scsi-bus. is there a terminator or is a jumper/switch on the drive so the drive supplies terminiation.

check LED's on the drive, do these say the unit is online? does it need cleaning?

check type / brand of cartridges. are these really DDS-3?
Is it by accident write-protected?

try an older cartridge which you know worked. does this work? (you want to be able to restore an old backup).

[Philip Horan]

Hi Pieter. OK I can now talk to the drive
(new + old media). I was working remotely so was not aware the tape had ejected (offline). Popped the tape back in mt online rane OK.

I ran tar cvf /dev/rmt0h /somefiles followed by mt rewind followed by tar tvf /dev/rmt0h

All commands executed OK.

I am confused regards the CRON jobs (backup routines) if they are not writing to tape?
Many thanks for your time and efforts.
Phil.

[Pieter 't Hart]

About the scripts, a "#" normally denotes a comment-line or any other line that needs not be executed.
unless you use a special shell wich does interpret a line starting with "#", both in autobackup and weeklybackup the line using the tape is "commented out" and not executed.

so in the two lines :
tar -cvB -C /mtms . -C /mtmsdb . -C /mtmsdb1 . -C /mtmsdb2/mtmsdb1 .
and
# find /mtms /mtmsdb /mtmsdb1 -print | cpio -ocBvdum > $TAPEDRIVE

the tar is executed and outputs to a directory and the find is not executed.

NB! i'm also puzzled about the "set +x" at the beginning of the script.
I know the command "set -x" in a shell script, not "set +x"!

[Philip Horan]

Pieter I am confused by it all!! :)
Many Thanks,
Phil

[Philip Horan]

Hi Peter. What I had hoped was that CRON scheduled backup (daily Mon-Thurs / Weekly Fri) that backed up all data + system files to tape so that in the event of a failure I could recover the server from tape media. The script should also kill users at backup commencement to ensure backup does not fail then send email notification upon completion.

As I said previously, running
tar tvf /dev/rmt0h daily looked fine so the script must work?
Regards,
Phil.

[Pieter 't Hart]

maybe the backup to tape does not run under account mtms? (maybe root)
if so you're looking at the wrong jobs.

try if you can
"more /var/spool/cron/crontabs/*"

what mail do you get?
(all mail lines also have "#")

[Philip Horan]

Hi Pieter. I ran: more /var/spool/cron/crontabs it returned

/var/spool/cron/crontabs is a directory

I had a look inside and there are a number of files, what should I be looking for?

Regards email I receive mail with attached file: cron.wri from account mtmsadm

Not sure how mail works on unix (i.e. how it gets pushed out to my exchange server)?

Thanks,
Phil.

[Pieter 't Hart]

hi Phil,
you missed the "/*" (asterix, wildcard) at the end of the command.
Each user has it's own file in this directory for jobs to be run by cron.
so with the "/*" you get a view of the cron entries of all the users (if you may view those files).
especially look at /var/spool/cron/crontabs/root (user root)for entries wich could be backup-actions.

another option is to wait until the tape-drive is used by some process and do a "ps -ef |grep rmt0" to see wich script is running and using the tape unit.

but before going so deep. your first concern is to check if the backup is running correctly now the tape is properly loaded.
locating the right script is interesting but not mandatory at this time.

[Philip Horan]

Hi Pieter. Added the wild card /*. Nothing running that I have not posted previously. What confuses me is that root CRON runs nightly1.jcl & mtmsadm CRON runs Nightly.jcl / Weekly.jcl which in turn reference AUTOBACKUP & WEEKLYBACKUP.

root - nightly.jcl is mostly commented out (or using a shell):
[quote]
umask 002
# CRONTAB="yes"; export CRONTAB
cd /mtms/tmp
if [ -f sys.profile ]
then
. ../sys.profile
fi
# echo "done /mtms/tmp/sys.profile"
if [ -f $MTMSROOT/mtms/std.profile ]
then
. $MTMSROOT/mtms/std.profile
fi
# echo "done /mtms/std.profile"
#if [ -f $MTMSROOT/mtms/user.profile ]
#then
# . $MTMSROOT/mtms/user.profile
#fi

if [ -f std.profile ]
then
. ./std.profile
fi
# echo "done /mtms/tmp/std.profile"
# if [ -f user.profile ]
# then
# . ./user.profile
# fi
# echo "done /mtms/tmp/user.profile"


#
# Stop the BSCqueu.run routine (to free up RECOVERY), if it is running
#

/mtms/bin/BSCqueue
#
# Restart the BSCqueue if required
#
# cd /mtms
# /mtms/crontab.profile
# su - BSC
# wait 20
# exit 0
[/quote]

I pretty much expect the backup to run (it ran OK Thursday (Nighly.jcl). The backup that seems to fail is Weekly.jcl.
Thanks,
Phil.