Compaq DAT 20/40

[Scott_186]

Hi everyone.. first time poster here..

Device: Compaq DAT 20/40 SDT-10000

I got the HP L&TT app and I've went through all the test and everything seems to check out ok.. Well, my whole purpose of downloading that app was to test the compression.. because like the 187 posts I read before typing this, I understand what the compression should be, and I know what it's not..

Problem: Inside the app I only have 5 choices.. Analysis, Connectivity, Media Validation, Read/Write, Self Test.. I look in the Scripts folder and I see the "Compression Test" file, but how come it doesn't show up inside the app??!?! arrrgg..

[Tina Karasch]

What it should be .... depends on your System and it's SW and HW - setup and off course, your data....

The L&TT application has originally been written with the firmware of the HP Dat40's in mind; not the pre-merge compaq dat 20/40 and it's firmware (inclusion is being worked on) - this is why you can see all at this point compatible tests in the pull down menu - if you go from within L&TT via "options" to "scripts" and it does display the script; you should be able to run it providing the unit has the latest fw and the L&TT version is the latest too.
( www.hp.com/support/tapetools/ )

Besides that you can actually check out what kind of compression you do get, by checking the job-logs of most major backup sw programs.

more info about compression and performance....

http://h20000.www2.hp.com/bizsupport/TechSupport/Document.jsp?objectID=lpg50244

http://www.hp.com/cgi-bin/cposupport/get_doc.pl?SNI=hpsurestor18545&LC=information_storage&Tfile=lpg50167

http://www.hp.com/cposupport/information_storage/support_doc/lpg50460.html

[Scott_186]

Thanks for the reply, but I've already read all of those compression tips and ideas pages.. It's ridiculous, it takes 8 hours to backup 22 gig of data!! WTF! I've never seen something that slow.. Anyway, like you were saying about running a Script in the L&TT, I tryed that.. The script doesn't show up in there.. Whats weird is, the people here are saying the before the 4 servers were upgraded to 2000 from NT4, it backed up just fine..
Anymore ideas?

[Marino Meloni_1]

Yes, another idea,
If this appear after migration to w2k,and if the server is using a Adaaptec HBA, update your SCSI drivers, the first released ones with W2K had some trouble with domain validation and some time the negotiate at 5 MB/S async mode.
another tes is following the 3 link suggested, and to use the tools described, they are very usefulls in order to troubleshoot this kind of issue

[Dave Dewar]

Hi Scott,

I would echo Marino's comments. If you are acheiving 22GB on a 20GB native tape, compression is working fine and your issue is preformance. The win2K original adaptec driver feature is a good candidate. As Marino says the set of performance tools HP provide on the links above are excellent at pinpointing where the issue might lie.

Cheers,

Dave Dewar.

[Scott_186]

thanks guys.. but I've updated the SCSI controller drivers already.. didn't help.. also, it's a 40gig tape.. It looks like ARCserve is running 10 sessions on one drive.. casuing the 3.5gig to become almost 20 gig.. that's not right!! is it?

[Dave Dewar]

Hi Scott,

If you do really have 10 identical Arcserve sessions backing up to the single tape drive then that would slug your performance as well and is obviously not right. I have no diea what type of system you are running on but this would result in a processor and disk load and could mean that the actual tape drive is not getting data as fast as it would like.
I am making the assumption that you only want to backup 3.5GB of data - one backup job. Check you Arcserve scheduling and backup options.
The tape you are using might be called a 40GB tape, but that refers to its capacity at 2:1 compression. The native capacity of a DDS4/DAT40 tape is 20GB.

Cheers,

Dave Dewar

[Scott_186]

Ok.. that makes sence..

So, who's the genious that's gonna tell me how to test my compression? I have the hp L&TT tools.. but I don't have a compression test and it doesn't appear under available scripts to run.. hhmmm

[Dave Dewar]

Hi Scott,

The final link that Tina posted goes to a performance trouble shooting guide.

http://www.hp.com/cposupport/information_storage/support_doc/lpg50460.html

There are also set of tools we have provided to allow you to debug performance issues using the guide. These are available under support/drivers. The explains how to use these tools.

try this link for the tools:

http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?pnameOID=18547&locale=en_US&taskId=135&prodTypeId=12169&prodSeriesId=81997&submit.y=6&submit.x=8&cc=us#Microsoft%20Windows%20Server%202003

These should work for the compaq dat40 drive you have as well, although I am not sure whether we have actually tested it :-).

One of the tools is called HPcreatedata. This allows you to create data files of different compression ratios. You can then use the other tools to send them to the drive and measure performance.

Read the performance troubleshooting guide and try the tools.

Cheers,

Dave Dewar.

[Scott_186]

Ok.. I tested everything.. It all checks out ok! The part that I was most concerned about was the Read/Write speed to the tape itself.. it came back at 6mb/s so that's alright.. Throug-put was good. Subsystem read was just fine.. I don't know what else to do.. I think I'm going to try and stop alot of service tonight to try the backup that way.. any suggestions about anything?

[Dave Dewar]

Hi Scott,

Let me know how you got on. As I said previously I know know much about your system so its difficult to make additional suggestions over and above whats in the performance troubleshooting guide which is pretty exhaustive.
If you are still having problems, give me a quick run down of your system and we will take it from there.
One point though, did you run the tool called PAT to check you disk performance? If your disk subsystem is not fast enough and/or heavily fragmented, then basically the data doesn't get supplied to the drive fast enough and the effective transfer rate with go down.

Cheers,

Dave Dewar

[Scott_186]

Dave.. I ran every tool that was on that page.. Yes, I ran PAT too.. the read/write was 6mb/s so that's ok..

Here's what I got:

4 Windows 2000 servers, all updated from nt4 in January (the last time the drive worked good) and they all have every update ever released.. sp's, drivers, etc.. 1 NT4 server, just doing faxxes.. I installed the Arcserve client agent on each machine last night..today when I got in the backup job was asking for another tape cause it ran out of room! ROOM? It only backed up 2 servers that between the two have maybe have 16 gig.. It's gotta be the compression, but I think it checked out ok with those tools..

[Dave Dewar · Accepted solution]

Hi Scott,

Hmmm, I am still not sure of you system setup here, bear with me if I am being a bit thick :-). My guess from what you are saying is that you have 4 W2k servers, networked and one of these has a Compaq 20/40 DAT drive connected to it. You also have an NT server as well on the same network, making a total of 5 servers.

You are backing up all five servers to the single DAT drive on one of the servers.

You are using Arcserve backup software.

Is this correct so far?

You have patched everything to the latest level.

The performance tools indicate your drive can operate as designed at speeds in excess of 6MB/s when you run them on the server with the tape drive connected.

What is the speed of your network? 100baseT? What configuraton are you running? Are all the servers connected to a single hub/switch?

What happens if you just perform a backup of the tape drive server? Do you get good performance? I wonder whether it is a network issue, or something to do with Arcserve client agents.

I still don't know what SCSI HBA you are using, but am assuming you have patched the drivers up to the latest.

As to why you are apparently getting less than native cpaicty on your tape and few possible answers are:
It is possible that if you have a poorly performing drive to actually waste capacity on the tape as the drive has to do lots of rewrites and so end up with less than native capacity. You have checked this on LTT and everything seems fine.
You could also have software and hardware compression turned on a the same time. This can result in data expansion and again lead to acheiving less than native capacity as observed by the user. You should be able to check this in the Arcserve backup options. Again, I am assuming you have already checked this.
Finally, if the type of data you are backing up is predominantly compressed data, zip, jpeg etc, then data expansion can occur with either software or hardware compression.
Tina's links explain all of this, but you have said you are familar with this, so we can discount it.

Anyway,

Have a good weekend, we can continue this on Monday if you are still having issues.

Cheers,

Dave Dewar.

[Scott_186]

Hi Dave

Yep, what you said about the 5 servers and one DAT drive is correct. We are runing a windows 2000 network (tcp/ip) and all servers are connected to a 100mb switch, oh, the servers are all 100mb too.. You name it though, I got the latest drivers for it.. EVERYTHING inside those servers is up to date, patches and all.. What exactly is a SCSI HMB though??

The server that the DAT sits in, backs up just fine in under an hour..so that is working fine.. Maybe it is the network, but I can't figure it out if it is.. I'm running the backup tonight on the primary server and one win2k server with the agent installed.. I'll let ya know how that goes..

I really appreciate your help Dave..

[Dave Dewar]

Hi Scott,

SCSI host bus adapter (HBA) (scsi controller card conencted to tape drive). Since, you get good performance standalone I think we can discount this.

With a 100baseT link max possible transfer rate is 10MB/s so one would think that would be ok.

I am not very familar with Arcserve client agent backup. I'll find some time to talk to one of our solutions engineers about this today to see if they can provide any pointers.

Cheers,

Dave Dewar.

[Scott_186]

This is my summary from last nights job:
*********************************
Backup Session Cancelled
155,429 directories
745,971 files (21,214.26 mb)
2 Database/Transaction Logs (4.625 mb)
28,375.25 mb written to media
Elapsed time: 3h 7m 5s
Averagae Throughput 151.57 mb/min
*********************************

So this is telling me that there is just not enough room on those tapes or compression is not working properly.. Thing is, this is only 3 of our 5 servers, so now what? New and bigger tape drive?

[Robin Collins]

Hi Scott

Like you I've been to hell and back with Arcserve following upgrade from NT4 to W2K.

If you're running Arcserve 2000 then you need to have Active Directory and Win2k 'domain' controllers for it to behave - weird security mode they built in! I've had to go back to NT4 for the Arcserve boxes 'cos I just do not have the time or resources to switch over to an AD forest.

If you have Antivirus on your backup machine you may want to consider switching it off - I got around 60% speed increase that way.

Compression under Arcserve is very variable and puts a big time overhead for little benefit. I normally get about 1.3 using hardware compression and 1.8 using software but oh boy does it add to the time. However that's with 1.5 million small files, YMMV.

Backup over the network can be very variable with Arcserve - in the end I've put my backup drives onto an old server with 100gb storeage. I then use MS robocopy to mirror the stuff I want to back up onto that machine. This reduces the hit on the network as robobcopy does incremental copy, normally takes about an hour to top up a 65gb mirror. tape drive can then get max thruput on SCSI

I'd suggest you let Win2K compress the folders you are backing up, and just use hardware compression in Arcserve.

Hope this helps a bit

Robin

[Scott_186]

Thanks Robin

When you say let win2k compress the folders itself, do you mean myself compress them or is that an option I have to turn on somewhere?? Arcserve isn't doing any compression right now, it's all hardware, if there is any there anyway..doesn't seem like it..

[Dave Dewar]

Hi Scott,

Looks like Robin has addressed your Arcserve issues.

About compression, do not use multiple compression methods on the same data. Choose one and use only that method. Otherwise you will get data expansion.

Software compression under windows will typically give you a better compression ratio than the hardware compression algorithm in DDS drives. However, you need a server that is fast enough to compress the data as well as sending it to the drive. You should expect to see about a 1.4-1.8 compression ratio for typical user file system backup.

Robin's suggestion about getting all of the data together on the backup server first and then sending it to the tape drive afterwards is a good idea. This is how high end systems work.

Arcserve supports tape spanning, however, if it is unattended backup you want then this is not much use to you unless you have an autoloader.

HP has just released the HP DAT72 product that stores 36GB native, 72GB at 2:1 compression so you could go out and buy this drive.

Cheers,

Dave Dewar

[Scott_186]

Thanks Dave

I think getting all the data on the backup machine before writing to tape, is not gonna happen.. I don't have near enough space available.. Anyway, same results as the night before, it backed up 28 gig and then asked for a new tape.. Either hardware compression is not working or the tapes i'm using are not what they say they are..

I'm using 4mm tape.. that's about all I see, except for the 4 up in the right hand corner.. that should be a 40 gig compressed right?

[Robin Collins]

Hi Scott

To compress the folder in Win2K, right click on the folder, go to properties, advanced, and select 'Compress contents to save space' - this can be helpful with some file types.

I haven't been looking too hard at compression on my system, but a quick check shows 1.42 to 1.54 just using hardware. If you got 28 gb onto a 20/40 tape then you've got 1.4 compression which is exactly what I'd expect.

The compression you are getting will show on the Activity Log tab as a line saying Media Compression Ratio n:nn.

As regards costs, you'll probably find that adding a 160gb IDE drive is a whole lot cheaper than a new DAT drive and lots of new tapes - depends on how you work and time pressures but worth a thought.

Although I do know a lot of folks use 'em autoloaders can be a bit cranky and a number of comments on the old Arcserve forum suggested that you need the really expensive top end jobs for reliability.

Hope this helps anyway

Robin

[Dave Dewar]

Hi,

The tapes you are using are DDS4 20/40GB tapes. As Robin says 28GB is a compression ratio of 1.4 which is what I would expect on typical file system data. Hardware compression is working and it is giving you expected compression ratios.

As I mentioned previously, you could try using software compression instead. The algorithm used by windows will typically give you a bit better compression ratio but you will eat CPU cycles if you are doing it inband (during the backup with Arcserve). Your other option is to create a disk image of you backup job on your backup server and compress this before you start the backup. Then send this down to your tape drive with hardware compression disabled. You will probably get about 1.5-1.8 compression ratio doing this.

I would also agree with Robin that a large IDE drive will be cheaper than another tape drive or an autoloader.

Robin's comment about the dat autoloader reliability is justified to some extent. The reason for this is that some customers use this product as a large tape drive and forget the fact that the duty cycle spec of these products are the same as standalone DDS drives, 20% or less. The product should really be used as an unattended backup device that uses at most two tapes per daily session.

Cheers,

Dave Dewar

[Scott_186]

Thanks again you two.. good info!

But my question remains: Why is it getting to 28 gig written to the tape and then asks for a new BLANK tape? it it's only got 28 gig on there, why is it asking for a new tape? Is it really full or do I still have a problem in there somewhere?

And, How come you say I should buy a new IDE drive? Are you saying copy the data to that as opposed to writing it to tape?? That's probably a stupid question, but I'm just a litt confused on why I'd buy another hdd.

Scott

[Dave Dewar]

Hi Scott,

Ok we have a fundamental misunderstanding about compression here, maybe :-)

The native capacity of a DDS4 tape is 20GB, thats 20 real true GB. The tape will not hold more than 20GB.
When we say the tape can hold say upto 40GB of compressed data we are saying that if we take 40GB of data, say a single large file and compress the file before we put it on tape then with a good compression algorthm and compressible data we might be able to actually remove redundancy from our file and end up with an compressed file that is smaller in size, maybe even as small as 20GB. Hence, we then put this on tape. Winzip is a claasic compression program. Zip up the file and it reduces in size if it contains redudant info. Try zipping a jpeg that is a compressed picture format and there is no redundancy to remove and so the file doesn't reduce in size.

When we do hardware compression in a tape drive the compression occurs before we actually write the data to tape. We take it from the host, store it in a memory buffer, compress it in reasonable sized chunks and write to tape. Uncompressing obviously works in reverse.

Hence, your tape drive is taking 28GB in, compressing it down to about 20GB and writing it to tape. The tape is then full and Arcserve asks you for another tape to complete your backup job. This is called tape spanning and most backup applications support this. Label your second tape "2 of 2" and insert it and Arcserve should complete the backup and your backup image will span across two tapes. You will need both to restore.

Apologies if most of the above is blinding obvious to you already, its sometimes difficult to gauge how much people already know from forums :-)

As to your second question, you don't have to spend any more money at all if you don't want to.
If you don't mind doing tape spanning and getting your 5 servers backed up on two tapes and you are happy with the 2.5-3MB/s you are achieving, then you are all set.
If you want to get your backup on one tape then you need a tape drive that can handle more native capacity, say a DAT72 that has a native tape cap of 36GB or some even bigger.
If you want to get better performance, then I think Robins suggestion of doing an image backup on a new IDE disk on your tape server and maybe partitioning that up to give you several volumes to image to, and then following that up with an backup of the image to your tape drive every so often might give you some performance benefit.

Cheers,

Dave Dewar

[Scott_186]

Thank you guys so much for your help on this..

Cheers..

The only thing left that I need to do is buy a new drive with a large native capacity.. Plain and simple..

Scott

I feel like I should have a certification or something out of this, it's been a very knowlegable couple weeks!!