Extract ammount of tape used (LTO-3 HP Ultrium 960 drive)

[Cosmin Prund]

Hello.

I'm using an Ultrium 960 tape drive to back up about 350Gb of data (on-disk data) to an LTO-3 medium (so 400Gb uncompressed are available).

As part of my backup script I'm writing stuff to a log file and I'd like to log the ammount of tape used, so I can give an warning when 90% of the tape is used (so I can prepare for two-tapes backups).

Unfortunatelly I found no way to extract this information. I tryed using "mt status" and "mt tell" to get the final block number of the backup but this prooved to be inconclusive because the displayed block number seems to be logical, not physical (I filled a tape with 390 GB of stuff from /dev/zero and with 390 GB of stuff I obtained from /dev/urandom and the final block number was the same).

I tryed using LTT to get the information but I didn't find a way to do it. LTT is not a good solution anywhay since it can't be scripted.

Then I found this recent thread "detect shoe-shining" (http://forums12.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1205914398646+28353475&threadId=1212744) and I found a grate way to extract a log page out of the tape drive and decode it to get some information.

Getting the data out of the drive involves issuing:
sg_logs -h /dev/nst0 -p=34
and then using bash/grep/awk/let/bc to extract and transform the information into something meaningfull.

After my nightly backup I found the following:

My "shoe shining" is 1/100Mb - seems ok.
Data rate onto tape after compression: 20.60 Mb/s
Rate of data into the drive: 30.37 Mb/s

From the rate into the drive and the rate onto tape I assume a compression rate of 1.47. Dividing my 350 Gb of data on disk by 1.47 I get 238 Gb of data to tape. Dividing 238 Gb by the tape's uncompressed capacity of 400Gb I get 59% tape usage.

Unfortunatelly this kind of computation can't take into account any kind of overhead so I really can't tell how much tape I'm using.

Finally the question:
Is there a similar way to get more accurate data?

Thanks,
Cosmin Prund

[Richard Bickers]

Hi Cosmin.

You should be able to use LTT for this. You can pull a support ticket using the scripting interface to the standard install. Have a look at the user guide for details.

Here's the syntax for pulling a ticket:

hp_ltt -f ticket p= [directory=] [-format ]

If you select XML output you can parse the ticket from the script and find all the information available to you in an LTO ticket - which covers everything you're looking for as far as I can see.

We can probably help with specifics if you want to go this route.

Regards, Richard (LTT team)

[Cosmin Prund]

Hello. Thanks for your time answering my question.

I managed to get an XML ticket using LTT's menu driven console version. Reading the XML I found something that might tell me what I need. It says:

Cartridge Performance/
Estimated native data rate (last 1 drives) : 19.4 MB/s (199 GB written, 0 KB read)

Does that mean I'm actually at about 50% tape?

This has been the good part. The bad part is that I can't get the XML from the command line, so I can't script it. I tryed several variations of this command line (with or without "-" before parameters - tryed all combinations):

./hp_ltt -f ticket p=/dev/nst0 directory=/rsync/ticket/ format=XML

It returns immediately and generates no output file. I get the following in /opt/ltt/logs/EventLog.ltt:

10:4:4:40:3:3:42:1:1:PlainText:Deleting perl log file failed; errno is 2:1252:0

What's the "perl log file" and can I delete it myself before invoking "hp_ltt"?

Thanks,
Cosmin Prund

(The full logfile follows, but the relevant line seems to be the last line and that's the line I quoted before)

0:1:1:2:4:""
10:1:4:2:0:0:0x20000150:1206002553:1206002553:0:1:8:0:0
10:2:4:40:1:3:100:6:0:0
10:3:4:40:2:3:41:1:1:s_system:common:enu
10:4:4:40:3:3:42:1:1:Timestamp:Thursday March 20 2008 - 10\:42\:33:1252:0
10:1:4:3:0:0:0x20000140:1206002553:1206002553:0:1:8:0:0
10:2:4:40:1:3:120:50:0:0
10:3:4:40:2:3:41:1:1:s_system:common:enu
10:4:4:40:3:3:42:1:1:PlainText:System Information:1252:0
10:1:5:4:3:0:0x20000140:1206002553:1206002553:0:0:8:0:0
10:2:5:40:1:4:100:50:0:0
10:3:5:40:2:4:41:1:1:s_system:common:enu
10:4:5:40:3:4:42:1:1:Version:CentOS release 5 (Final):1252:0
10:5:5:40:4:4:42:1:1:Description::1252:0
10:1:5:5:3:0:0x20000140:1206002553:1206002553:0:0:8:0:0
10:2:5:40:1:4:100:51:0:0
10:3:5:40:2:4:41:1:1:s_system:common:enu
10:4:5:40:3:4:42:1:1:CPUType:unknown:1252:0
10:5:5:40:4:4:43:1:1:CPUCount:8:0:0
10:1:4:6:3:0:0x20000140:1206002553:1206002553:0:0:8:0:0
10:2:4:40:1:3:100:52:0:0
10:3:4:40:2:3:41:1:1:s_system:common:enu
10:4:4:40:3:3:42:1:1:SystemName:bigfoot.sediu.ro:1252:0
10:1:4:7:3:0:0x20000140:1206002553:1206002553:0:0:8:0:0
10:2:4:40:1:3:100:54:0:0
10:3:4:40:2:3:41:1:1:s_system:common:enu
10:4:4:40:3:3:42:1:1:Platform:x86_64:1252:0
10:1:4:8:3:0:0x20000140:1206002553:1206002553:0:0:8:0:0
10:2:4:40:1:3:100:5003:0:0
10:3:4:40:2:3:41:1:1:s_system:common:enu
10:4:4:40:3:3:42:1:1:Ver:Version 4.5 SR1:1252:0
10:1:4:9:3:0:0x20000140:1206002553:1206002553:0:0:8:0:0
10:2:4:40:1:3:100:55:0:0
10:3:4:40:2:3:41:1:1:s_system:common:enu
10:4:4:40:3:3:42:1:1:SystemTime:Thursday March 20 2008 - 10\:42\:33:1252:0
10:1:4:10:0:0:0x20000140:1206002553:1206002553:0:4:8:0:0
10:2:4:40:1:3:120:50:0:0
10:3:4:40:2:3:41:1:1:s_system:common:enu
10:4:4:40:3:3:42:1:1:PlainText:Deleting perl log file failed; errno is 2:1252:0

[Richard Bickers · Accepted solution]

Hi Cosmin.

Sorry for the delay in getting back to you.

The first piece you show is the native transfer rate for writing data to the tape - 19.4 MB/S. I think you're after the amount of tape used which is also in the support ticket under Cartridge Usage. Look for something like this:

Native capacity used : 77.6 GB (38%)

You do still have to have the tape loaded to see this.

Your second question is about being able to pull the ticket from the script. Your output looks to me like you weren't able to reference the drive correctly. You do need to use the right syntax for the drive location.

To get the exact syntax it's best to run LTT from the command line and ask it to do a device scan and it will generate a file called saved_scan.txt with all your drives listed.

Used this syntax to run the scan:

hp_ltt -f scan

And you should find something like this in your saved_scan.txt file:

2/0.4.0:HP:Ultrium 3-SCSI:G54D

Use the numbers at the start of the line for the device address so to pull a ticket:

hp_ltt -f ticket p=2/0.4.0 -format XML

This is all in the user guide with examples though it's probably more cryptic than it might be. Fingers crossed you can make it work OK. This is what the scripting interface is intended for!

Good luck, Richard.

[Cosmin Prund]

Thanks, this really solved it.

And now that I know the answer I find the PDF documentation fairly clear. I'd have two sugestions those, for anyone else reading this thread or for the HP people doing the documentation/application:

(1) Give a clear error message on the command line when the device path is not good. Guess what error message I got from this command:

/opt/ltt/hp_ltt -f ticket p=reallywrong -format XML

(2) In the documentation give an example of Red Hat output. My device path is "60.3.0[6-/dev/sg5]" - my brain failed to realise the square brackets are part of the device path!

Anyway, thanks again!

[Richard Bickers]

Glad it worked out in the end.

Thanks for the feedback on the documentation. We'll take a look at that.

Richard.